Declare the merge-bot workflow the release contract already grades - #512
Merged
Conversation
spec/files.json required .github/dependabot.yml on every repo and never required the workflow that merges the pull requests it opens. A grep for merge-bot across the whole baseline returned nothing, while WORKFLOW.md D8.1 grades its behavior, GOVERNANCE.md links it twice as an in-repo path, and audit reports already mark a repo drifting for not having one. So a graded behavioral contract had no corresponding required file, and a repo could carry valid App credentials that nothing consumes. Blog hit it as designed: 23 of 23 applicable baseline files present, configure.sh check exit 0, and then Dependabot opened its first pull request with nowhere to go. The gap is invisible to a presence audit because the missing file is not in the list being checked. Declared at interface fidelity, since a repo legitimately prunes jobs for bots it does not have. The required keys are the two that are universal: merge-dependabot, because dependabot.yml applies to every repo, and disable-auto-merge-on-maintainer-push, which D8.1 names directly. merge-codegen and merge-upstream-version stay optional. Reported by the Blog agent on #456, and confirmed against the fleet: 15 of 21 repos already carry the file, so this ratifies the de facto standard rather than inventing one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the repository’s file-contract spec so the presence/contract of the merge-bot workflow is audited alongside the already-graded release automation requirements.
Changes:
- Adds
.github/workflows/merge-bot-pull-request.ymltospec/files.jsonas aninterface-fidelity required file for all project types. - Enforces a minimal universal workflow contract by requiring the
merge-dependabotanddisable-auto-merge-on-maintainer-pushjob keys.
This was referenced Aug 1, 2026
ptr727
added a commit
that referenced
this pull request
Aug 1, 2026
Closes the governance question #508 raised: may a downstream repo open a pull request against the hub to write its own audit report? ## Why there was a vacuum `AUDIT.md` said the audit is read-only and produces a report under `reports/`, and `AUDIT.md` section 10 said "the hub audits and the agent applies the fixes via target PRs". Neither ever said **who writes the report**, so an agent filled the gap, reasonably. ## The answer: the hub authors it The maintainer's call, and the argument that settles it is **efficiency, not principle**: > consider you want it in develop, and you want to audit it anyway, you are going to run the same effort, and the result would be the same if you agree with their audit That is correct and it dismantles the case I had been making. I argued a submitted report preserves downstream context the hub would otherwise rebuild. But **verifying the judgment dimensions is the audit**: confirming a verdict like "analyzers enforced, `TreatWarningsAsErrors` present" means reading the same files the audit reads. Only the deterministic subset is cheap. So a submitted report saves the **writing up**, which is the trivial part once the audit is done. The independence argument survives as a second reason, since a repo grading itself produces a claim rather than evidence, but on its own it would only have justified verifying a submission rather than declining one. ## A third reason #508 demonstrated **A submitted report is stale by construction.** It is a snapshot of one hub revision arriving at a later one. #508 was authored against hub `6501479` and merged at `839ecde`, and before adopting it I had to reconcile two findings that did not exist when it was written: `OPERATIONS.md` became universal in #507, and the merge-bot workflow was declared in #512. That reconciliation is inherent to the workflow, not incidental. ## What downstream still does, and it matters Unchanged, and it is where downstream context is worth most: - **File findings about the hub as issues.** The opposite of self-certification, and it has surfaced several real hub defects today: the merge-bot workflow that no spec required while `WORKFLOW.md` D8.1 graded it, `copilot-instructions.md` describing a pre-split `AGENTS.md`, `CODESTYLE.md` contradicting the markdownlint config, and D2.2 wording that nearly caused a real regression. - **Apply fixes to its own repo** per section 10. ## Also names the base branch `#508` targeted **`main`**, and I merged it without checking `baseRefName`. That put content on `main` that `develop` lacked, breaking the forward-only invariant, and needed a forward-port in #513. The branching model was already stated in `GOVERNANCE.md`, but not in the place an agent opening a conformance pull request against the hub would be reading. It is now stated here too. ## Verification ``` prose_lint AUDIT.md --check charset --check dupword --check spelling --check charset-unknown clean markdownlint-cli2 (docker, canonical) 0 issues editorconfig-checker (docker, canonical) clean python3 spec/validate.py OK, 21 cataloged ``` Prose only, no engine change. `AUDIT.md` is carried at `intent`, so downstream repos pick this up on their next re-vendor. ## What this does not solve Thirteen of twenty-one cataloged repos still have no committed audit report, tracked in `TODO.md`. This decision means the hub writes them, which is a capacity question rather than a governance one. Accepting unverified submissions would have closed the gap by lowering the bar rather than by doing the audits. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ptr727
added a commit
that referenced
this pull request
Aug 1, 2026
Records a decision that was carried only in conversation, at the maintainer's request. ## The gap `AUDIT.md` already states the **outcome**: the deterministic subset is "owner-initiated, run on demand when onboarding a repo, on suspected drift, or before fleet-wide changes." What it does not state is that **automation was explored and deliberately deferred**, or why. So the next person to notice that 13 of 21 repos have no committed report re-derives the whole analysis, and quite possibly reaches a different answer for reasons already rejected. ## What was considered, and what blocked it Three shapes: 1. A **scheduled hub-driven audit** publishing each report as a workflow artifact. 2. The same, **committing the report back**, the way the codegen bot updates its own files. 3. A **pull-request hook in each downstream repo**, auditing itself against the current hub. Three things blocked all three: - **Noise.** Until the fleet reaches stasis with every repo onboarded, a scheduled run reports mostly noise, because a repo mid-onboarding is *expected* to be non-conformant. Findings that are correct and not actionable train people to ignore the channel. - **Hub stability.** The hub has to be stable before downstreams can audit against it, since a hub change lands as fleet-wide findings the same day. Today is the worked example: `OPERATIONS.md` becoming universal in #507 created 17 findings, and declaring the merge-bot workflow in #512 created 6 more. - **A catch-22 on the downstream shape.** A self-auditing pull-request hook is CI instrumentation, and the repos that most need auditing are exactly the ones that do not carry it yet. The agreed outcome was the on-demand audit `AUDIT.md` describes today. ## Why the entry is actionable rather than a note It records the **preconditions** that would make reopening worthwhile (the fleet onboarded, and the hub going a stretch without carried-content changes) and **which shape to try first**: the artifact, because it produces evidence without committing anything, so it cannot generate review load while the noise level is still unknown. That makes the first experiment cheap and reversible. ## One correction to an existing entry The `reports/` entry read as a defect. It is not. Per the maintainer, the gap is **paced by capacity**, with repos brought up to spec as they are worked on, so the entry now says it records the outstanding set rather than a failure. This matters because #515 just put report authorship with the hub, and an entry that reads as an accusation invites re-opening a decision that was made deliberately. ## Verification ``` prose_lint TODO.md (dash, semicolon, charset, charset-unknown, dupword, spelling, sentence-split) clean markdownlint-cli2 (docker, canonical) 0 issues editorconfig-checker (docker, canonical) clean ``` Backlog only. The `[audit]` reference already pointed at `spec/audit.py`, so the new pointer to the document uses a separate `[audit-doc]` definition rather than overloading it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ptr727
added a commit
that referenced
this pull request
Aug 1, 2026
… the carried-file corrections to main (#517) Promotes twelve commits to `main`. Three of them change **carried** files, so downstream repos read the stale text until this lands, which is what makes the promotion the delivery step rather than bookkeeping. ## What this delivers to the fleet **Carried-file corrections**, picked up by every repo on its next re-vendor: | PR | File | Correction | | --- | --- | --- | | #514 | `.github/copilot-instructions.md` | Described the pre-split `AGENTS.md`, wrong in both halves. `GOVERNANCE.md` holds the rule sections and the two `intent` ones; `AGENTS.md` carries two verbatim sections and none repo-specific | | #514 | `CODESTYLE.md` | Said MD033 flags HTML elements while the config allows `details` and `summary`; also gains the `HISTORY.md` mirror rule, moved from a hub-only spec file so a repo can read the rule it is graded against | | #514 | `WORKFLOW.md` | D2.2 "skipped on smoke" read as the job rather than the validation, and a review acting on it proposed a change that would have coupled the release to smoke | | #503 | `.editorconfig`, `.gitattributes` | 36 comment findings in the files a new repo copies first | | #501 (earlier) | `repo-config/configure.sh` | Already on `main` | **Spec and requirement changes**: - **#507** makes `OPERATIONS.md` `appliesTo: "*"`, keyed to the need rather than to `workflowModel`. Measured: 3 of 21 repos carry it, so **17 will report a new `LETTER file:` finding**. Intended, and a headed stub satisfies it. - **#512** declares `.github/workflows/merge-bot-pull-request.yml` at `interface` fidelity, closing a graded contract (`WORKFLOW.md` D8.1) that no spec required. 15 of 21 already carry it, so this ratifies a de facto standard; the other **6 report a new `DRIFT`**, and those six have Dependabot pull requests piling up with nothing to merge them. - **#505** stops the audit flagging the hub against itself and drops a premise that was false since this repo went public. - **#511** fixes the README shape across repository visibility. - **#515** puts audit-report authorship with the hub and names the base branch, after a downstream conformance PR targeted `main` and I merged it without checking. - **#513** forward-ports that PR's content, restoring the forward-only invariant. **Backlog and reports**: #504, #506, #510, #516 record the README shape rework, the branch-bootstrap step, the declared-description design, and why the audit is on demand. ## Fleet consequence, stated before it lands Two owed follow-ups this promotion creates or carries: 1. **Re-vendor `repo-config/configure.sh`**, already owed from #501, since it is `verbatim` with `appliesTo: "*"` and every repo holding a copy is byte-mismatched. 2. **The next fleet audit reads noisier**, by 17 `OPERATIONS.md` letters and 6 merge-bot drifts. Both are new requirements rather than repo regressions. ## Verification Run on `develop` at `e4c76bc`: ``` python3 scripts/test_prose_lint.py 144 tests, OK python3 scripts/test_repo_gate.py 23 tests, OK python3 scripts/test_pr_review.py 27 tests, OK python3 spec/audit.py --selftest SELFTEST PASS python3 scripts/repo_gate.py eol 0, sha-pin 0 python3 spec/validate.py OK, 21 cataloged prose_lint --check charset --check dupword --check spelling clean editorconfig-checker (docker, canonical) clean ``` Every one of the twelve was merged green with a Copilot review covering its head SHA. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reported by the Blog agent on #456 and confirmed here. This one has teeth: a repo can pass the file-presence audit and still be functionally broken.
The gap
spec/files.jsonrequires.github/dependabot.ymlon every repo (appliesTo: "*") and never required the workflow that merges the pull requests it opens. A grep formerge-botacross the whole baseline returns nothing.Everything else assumes it exists:
WORKFLOW.md:197D8.1 is a graded behavioral contract for it, and audit reports grade repos on it.reports/vscode-server-dotnetcore/audit.mdalready marks a repofail/driftfor "no merge-bot workflow".GOVERNANCE.md:54links it as an in-repo path and makes it the mechanism the dual-target bot model depends on.GOVERNANCE.md:298names it as one of two documentedcancel-in-progress: falseexceptions.WORKFLOW.md:199ties the tracker's branch prefix to "the merge-bot's hard-coded<prefix>-<base>head/base pairs, or auto-merge silently never fires."A graded contract with no required file.
How it presented
Blog carried all 23 applicable baseline files and
configure.sh checkexited 0, which is the definition of conformant. Dependabot then opened its first pull request and it had nowhere to go. The gap is invisible to a presence audit because the missing file is not in the list being checked, and the symptom appears days later as bot pull requests quietly accumulating.The
repo-setupdimension also asserts the App secrets are provisioned while nothing checks the consumer, so a repo can hold valid App credentials that nothing consumes.The change
{ "path": ".github/workflows/merge-bot-pull-request.yml", "fidelity": "interface", "contract": { "requiredJobKeys": ["merge-dependabot", "disable-auto-merge-on-maintainer-push"] }, "intentRef": "WORKFLOW.md#d8---bots--automation", "appliesTo": "*" }interfacefidelity, because a repo legitimately prunes jobs for bots it does not have. Blog vendoredmerge-dependabotanddisable-auto-merge-on-maintainer-pushand droppedmerge-codegenandmerge-upstream-version, having neither a codegen workflow nor an upstream tracker, and a job keyed on a bot that never opens a pull request would be dead code reading as coverage.The two required keys are the universal ones:
merge-dependabot, sincedependabot.ymlapplies to every repo, anddisable-auto-merge-on-maintainer-push, which D8.1 names directly. The other two stay optional.Verified the contract is actually enforced rather than decorative:
audit.py:701readsitem.get("contract")andcheck_interfaceat:340iteratesrequiredJobKeys.Fleet impact, measured
15 of 21 repos already carry the file, so this ratifies a de facto standard rather than inventing one.
Those six have Blog's silent pile-up today.
One thing worth deciding separately
Absence of an
interfaceunit reports asDRIFT, notLETTER. Confirmed live against a missing repo:audit.py:690explains why: "a workflow's naming is more variable than a carried config, so absence is a hint to verify." That is reasonable in general and consistent withtest-pull-request.yml, the other universally-required interface file.For this file it may understate the case, since absence is not a naming variation, it is the bot not running. Whether an
interfaceunit atappliesTo: "*"should reportLETTERon absence is a separate engine question, deliberately not changed here.Verification
The first draft used
intentRef: WORKFLOW.md#d8---bot-pull-requests, an anchor that does not exist. Nothing caught it, since anchors in the spec are unvalidated. Corrected to#d8---bots--automation, matchingproject-types.json, and the unvalidated-anchor gap is worth its own look.🤖 Generated with Claude Code