fix(scripts): read revert subjects and assert the canary's replay attributions - #2843
Conversation
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Claude finished @kyle-sexton's task in 1m 38s —— View job Security review complete
No security issues found. Summary of what was reviewed:
This canary runs post-merge on |
|
Claude finished @kyle-sexton's task in 5m 1s —— View job Code review complete
ScopeReviewed all four changed files against FindingsOne inline finding posted on Everything else checked out on read-through:
Not independently verifiedThis checkout is shallow (2 commits of history), so I could not run |
|
Last security-reviewed head: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d8e43cfd31
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Claude has reviewed this PR 5 times. The lane skips further automatic reviews after 5; deleting this comment resets the count. |
…e replay's attributions Closes #2837. Closes #2833. declares_removal() accepted three intent forms, none of which a deliberate revert can carry here: the repo is squash-only with squash_merge_commit_title PR_TITLE, and the required Conventional-Commits title gate admits `revert:` but nothing a `Revert "…"` subject could match. It now also accepts the Conventional-Commits revert type, anchored at the start of the subject and requiring the lowercase token, its optional scope and/or `!`, its colon, and a non-empty description -- never a substring search for "revert". verify_known_incidents decided a `fires` row purely on exit status while its note claimed a specific culprit and line count that nothing checked. A row may now carry `[<culprit-full-sha>=<lines>,...]` after its sha, and the replay asserts the run's findings are exactly that set. Counts come from a new FINDINGS_SINK that report_finding writes, not from scraping the human report; nothing else sets it, so --commit and range mode are unchanged. The canary workflow header claimed there is no `pull_request` trigger when its own `on:` block has one. The conclusion holds for a different reason -- both scan steps are gated on the event name and the lane sits outside ci-status -- so only the stated reason is corrected. No trigger and no `if:` changed. Also fail-closed mk_repo in the test harness: it is called through a command substitution, so a failure returned "" and `git -C ""` then staged and committed the developer's own work as `test <t@t.test>`. It now yields a nonexistent path so git fails loudly instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ending on ambient config Refs #2833. The first CI run of this branch went red because the runner attributed 298 lines to eda5ae5 where the authoring machine measured 301. attribute_file called bare `git diff --unified=0`, so it inherited whatever `diff.algorithm` the caller's config carried: git's default `myers` gives 298, a globally-set `diff.algorithm = histogram` gives 301. The algorithm changes which lines a hunk calls deleted, and therefore the per-culprit counts the canary thresholds on -- so the same commit could cross the 200-line threshold on one machine and stay below it on another, and the header's "fires on 5 commits over 500, 1%" calibration only ever described one algorithm. attribute_file now pins `--diff-algorithm=myers -M`, and the file enumeration pins `-M`. Both are git's defaults, so what CI detects is unchanged; a local run now matches CI rather than the reverse. `-M` closes the same exposure for rename detection, which the header already calls load-bearing. The recorded figure becomes 298, with the refutation of 301 attached in both the script header and the corpus so it is not "corrected" back by someone re-measuring under histogram. This was surfaced by #2833's new exact-count replay assertions, which turned a silent config-dependent divergence into a red build on their first run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
d8e43cf to
167d5aa
Compare
|
Claude finished @kyle-sexton's task in 3m 34s —— View job Code review complete
ScopeReviewed the full diff against FindingsOne inline finding, posted on
Everything else checked out on read-through
Not independently verifiedDid not re-run |
|
Claude finished @kyle-sexton's task in 1m 19s —— View job Security review complete
No security issues found. ScopeThis is a What changed
AssessmentSkip-gate criterion 2 applies: no security-relevant surface. Both |
|
Both review lanes raised the same finding at A A leading if [[ "$rest" == \[* ]]; then
[[ "$rest" == \[*\]* ]] ||
die "unterminated attribution field for $sha in $INCIDENTS_FILE (no closing ']'): $rest"
...
fiReproduced against the real corpus with the closing bracket stripped from the Two malformed shapes added to |
…ot absent Refs #2833. Raised on #2843 by two independent review lanes. A `fires` row whose attribution field opened with `[` but never closed it failed the `[[ "$rest" == \[*\]* ]]` glob, so `attribution` stayed empty, the whole remainder became free-text `note`, and the row fell back to passing on exit status alone. That reintroduces the exact pre-#2833 gap this replay exists to close, by the one route nobody would think to look at, and it contradicts the contract documented directly above it: a malformed field is exit 2, never a FAIL and never a pass. A leading `[` now COMMITS the row to carrying an attribution, and an unterminated field takes the malformed path. The four malformed shapes already pinned all carried a closing `]`, so this shape was untested. Two more are added -- `[<sha>=40` and a bare `[`. The shipped corpus was never at risk (t_shipped_data_files_are_wellformed's regex covers it), but that is a separate layer and does not hold for a custom SILENT_REVERT_INCIDENTS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 28s —— View job Code review complete
ScopeThis is a What changedThis commit fixes the exact bracket-detection gap that both Codex and this lane's two prior passes raised on
Verification (static read-through)
No new findings. The previously-raised finding (bracket-detection trapdoor) is fixed correctly and is now covered by a permanent test. Not independently verifiedCould not execute |
|
Claude finished @kyle-sexton's task in 53s —— View job Security review complete
No security issues found. ScopeThis is a What changed
AssessmentSkip-gate criterion 2 applies: no security-relevant surface. |
… canary Refs #2833. Raised in fresh-context verification of #2843. Pinning the diff flags left the other half of the measurement exposed: attribute_file still called bare `git blame`, so `blame.ignoreRevsFile` -- an ordinary setting in any repo carrying a bulk-reformat commit -- still decided the per-culprit counts the replay now asserts on. Measured on cc58cbc with that setting naming bfb66be, its attribution collapses 853 -> 259 while eda5ae5's rises 298 -> 322. On a synthetic fixture it is worse than a wrong number. With the pin absent and that config present, the detector reports NO FINDING AT ALL on a genuine silent revert -- a false green, which is the precise failure this canary exists to remove, reached through the developer's own gitconfig. It has to be the --no-ignore-revs-file OPTION. `-c blame.ignoreRevsFile=` does NOT clear it: the documented "an empty file name resets the list" applies to the option, and the -c form was measured leaving the hostile value fully in effect. The obvious symmetry with the -c pins is wrong here, so the comment says so. t_counts_are_immune_to_ambient_git_config pins the property directly: the same fixture scanned twice, once under a hostile GIT_CONFIG_GLOBAL setting blame.ignoreRevsFile, diff.algorithm and diff.renames, asserting identical exit status and identical per-culprit findings. Confirmed discriminating -- with the blame pin stripped it fails, and it fails because the hostile run reports nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 6m 1s —— View job Code review complete
ScopeThis is a What this commit doesExtends the ambient-config immunity already established for Read-through verification
No new findings. Not independently verifiedCould not execute |
|
Claude finished @kyle-sexton's task in 1m 5s —— View job Security review complete
No security issues found. ScopeThis is a What changed
AssessmentSkip-gate criterion 2 applies: no new security-relevant surface.
|
## Summary
`ci-status` is the single check the org `ci-gate` ruleset keys on, and
its own comment calls its
`needs` list "the single source of truth for the lane list". Nothing
enforced the other direction. A
job defined in `.github/workflows/ci.yml` but absent from that list
still runs, still reports, and
still turns red in the run list — while `ci-status` reports success and
the merge proceeds. The
`hook-utils-windows` comment already states the doctrine in prose ("a
lane missing from that list is
informational no matter how loudly a comment here calls it a gate");
prose is not a gate.
The gap was computed, not read. Against `origin/main` at `2d20a277`
(i.e. after #2841 landed and
added two lanes of its own):
```console
$ bash derive-gap.sh
defined jobs (incl. ci-status): 37
ci-status.needs entries: 34
--- defined but NOT in ci-status.needs (excluding ci-status itself) ---
managed-scope-sync
state-key-sync
--- in ci-status.needs but NOT defined ---
```
(The issue's "38 defined / 34 named" was an eyeball count on a pre-#2841
tree; the derived numbers
differ, the two missing lanes do not.)
## Fix
**Both missing lanes gate — neither omission was deliberate.**
| Job | Decision | Why |
| --- | --- | --- |
| `managed-scope-sync` | **Add to `needs`** | Structurally identical to
the four sync lanes already in `needs` (`hook-utils-sync`,
`parse-concern-value-sync`, `resolve-convention-pattern-sync`,
`standards-contract-sync`): same `--check` / lib self-test /
`--check-bump` triple. Not advisory, not schedule- or event-scoped — the
`if: github.event_name == 'pull_request'` sits on the bump **step**,
never on the job, so the job runs and reports on every PR and every
push. `scripts/cross-plugin-source-registry.txt` already advertises it
as the dedicated check for `lib/managed-scope.sh`. |
| `state-key-sync` | **Add to `needs`** | Same shape, same registry
claim for `lib/state-key.sh`. |
No job in this workflow is intentionally omitted, so this PR ships
**zero** opt-out annotations.
`cross-plugin-source-drift` (already in `needs`) covers a *drifted copy*
of either cluster, so this
was never a total hole — but it does not cover the `--check-bump` half
(lib changed, carrying plugin
version not bumped), which only the dedicated lanes run.
**The guard.** `scripts/check-lane-coverage.sh --check` proves the
defined-job set and
`ci-status.needs` are equal in both directions, and is itself wired into
`ci-status.needs` as
`lane-coverage-gate` (self-covering: its own absence from `needs` would
be caught by itself). It
reports four classes:
- `UNGATED LANE` — defined, not in `needs`, not annotated. The class
#2856 filed.
- `DANGLING NEED` — a `needs` entry naming no defined job.
- `STALE OPT-OUT` — a job annotated as deliberately ungated that *is* in
`needs`.
- `BARE OPT-OUT` — `# lane-coverage-ok:` with no reason after the colon.
**Required, not advisory** — deliberately. An advisory lane-coverage
check would be a green-and-silent
surface whose entire purpose is detecting green-and-silent surfaces,
which
`docs/conventions/liveness-assertion/` names as non-conforming. The
false-positive risk that normally
argues for advisory is absent here: the gate reads one file, takes no
diff, no base ref and no
network, and every YAML shape it does not model (flow-sequence `needs:
[a, b]`, scalar `needs:`, an
aggregate with no `needs:`, an unmodelled 2-space key under `jobs:`)
exits **2 (inconclusive)** rather
than 0 — reporting coverage from a file it did not parse would be the
gate committing the very defect
it detects.
**The opt-out** is the same annotated-exemption shape `#
silent-skip-ok:` uses for
`scripts/check-silent-skips.sh` — a `# lane-coverage-ok: <reason>`
comment in the contiguous 2-space
block immediately above the job key, or trailing the key itself. The
reason lives next to the thing it
excuses, in the file a reviewer is already reading, with no separate
list to drift. It carries a stale
guard (the annotation cannot outlive what it excuses) and a bare
annotation **fails** rather than
passing as "annotated", so it can never become a silent off switch.
No job's logic, `runs-on`, triggers, or `if:` conditions changed. The
`ci.yml` diff is
**31 insertions, 0 deletions**: three `needs:` entries and one new job
block.
## Verification
**The guard fires on the real defect and stops firing once fixed** —
constructed and run, not reasoned
about. On the unfixed `ci.yml` (pre-edit, at `origin/main`):
```console
$ bash scripts/check-lane-coverage.sh --check ; echo "EXIT=$?"
UNGATED LANE: job 'managed-scope-sync' is defined in .github/workflows/ci.yml but absent from ci-status.needs, so it cannot gate a merge. ...
UNGATED LANE: job 'state-key-sync' is defined in .github/workflows/ci.yml but absent from ci-status.needs, so it cannot gate a merge. ...
check-lane-coverage: 2 coverage defect(s) in .github/workflows/ci.yml
EXIT=1
```
After the wiring, the same derivation returns an empty gap and the gate
is green:
```console
$ bash derive-gap.sh
defined jobs (incl. ci-status): 38
ci-status.needs entries: 37
--- defined but NOT in ci-status.needs (excluding ci-status itself) ---
--- in ci-status.needs but NOT defined ---
$ bash scripts/check-lane-coverage.sh --check
check-lane-coverage: .github/workflows/ci.yml — all 37 lane(s) reachable from ci-status.needs
```
## Test plan
`scripts/check-lane-coverage.test.sh` — 20 cases, synthetic workflow
fixtures built per case, all
`ALL PASS`:
- a job absent from `needs` fails **1** and names the job; every job
present passes **0**
- annotated opt-out passes **0**; trailing-comment form passes **0**
- opt-out with no reason fails **1** (`BARE OPT-OUT`)
- opt-out on a job that *is* in `needs` fails **1** (`STALE OPT-OUT`)
- an annotation separated from its key by a blank line does **not**
exempt the job
- `needs` entry naming no defined job fails **1** (`DANGLING NEED`)
- flow-sequence `needs`, scalar `needs`, absent `needs`, empty `needs`,
unmodelled 2-space key,
unknown aggregate id, missing file, bad usage, no `jobs:` mapping — all
exit **2**, never 0
- the repository's own `ci.yml` exits **0**
Fixtures are plain files under `mktemp` addressed by absolute path — no
scratch git repo, so the
`git config user.email` caller-config-clobber class (#2839) cannot recur
here.
Also run clean locally: `shellcheck` and `actionlint` on the touched
files;
`check-shell-portability.sh --paths` (both new scripts);
`check-silent-skips.sh`;
`check-discriminating-test-skips.sh`; `check-orphaned-fixtures.sh`;
`check-changelog-parity.sh
--check` and `--check-bump origin/main` (no plugin touched, so no
version bump is owed);
`check-stale-base-overlap.sh --check origin/main`; `affected-tests.sh`
(both new files map to a suite
— no unmapped-file error); `affected-tests.test.sh` (32/32) and
`check-docs-only.test.sh` (21/21),
the two suites the selector picked for the `ci.yml` change.
An independent fresh-context verifier re-derived both sets with its own
script, rebuilt the
fired/not-fired evidence from scratch, and diff-checked that no job's
logic, triggers, or `if:`
conditions changed.
## Related
- Fixes #2856
- Refs #2834, #2841 — #2841 is where "a new job is automatically
required" was tested and found
false; it added `windows-path-emit-gate` and `windows-path-emit-windows`
and wired both by hand.
This PR makes the hand-wiring mechanical. Branched off `origin/main`
after #2841 merged.
- Refs #532 — `docs/conventions/liveness-assertion/`, the false-green
class this defect belongs to.
- Does not touch `scripts/check-silent-revert.sh` or
`scripts/silent-revert-incidents.txt` (#2843).
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
The header rewritten earlier in this PR replaced a false claim ("there is
no `pull_request` trigger") with one that was itself slightly untrue: that
the trigger "exists solely to run the detector's own unit tests", gating
"both scan steps".
On a `pull_request` event the lane runs three steps -- Check out, Test the
silent-revert detector, and Replay the recorded incidents. The replay is a
corpus reproduction against real history, not a unit test. And of the two
steps carrying `if: github.event_name != 'pull_request'`, one resolves the
pushed range rather than scanning.
The load-bearing claim was true and is unchanged: no PR is ever inspected
for silent reverts. This just stops the correction from repeating, in
smaller form, the defect it was written to fix.
Comment-only. The executable YAML stays byte-identical to main -- verified
by md5 of the non-comment lines and by a zero count of changed
non-comment lines.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018S8a1S71VxhLTRWBtMuEvp
|
Claude finished @kyle-sexton's task in 56s —— View job Code review complete
ScopeThis is a What this commit doesCorrects the header prose this PR itself rewrote earlier (in Verified by reading the current file (
Confirmed comment-only: every changed line in No new findings. This is a small, accurate self-correction to prose this same PR introduced — verified against the actual workflow structure, not just read for plausibility. Not independently verifiedDid not execute |
|
Claude finished @kyle-sexton's task in 42s —— View job Security review complete
No security issues found. ScopeThis is a What changedTwo comment blocks in the workflow header are reworded — from "the AssessmentSkip-gate criterion 2 applies: no security-relevant surface.
|
…overclaiming which The note above the diff asserted that EVERY config knob able to move a count was pinned. Three were not, and one of them was a false green. --no-ext-diff. `diff.external` is what difftastic's and delta's own install instructions tell people to put in their global config. It replaces git's diff output wholesale, so no @@ headers reach attribute_file and every commit attributes nothing. Measured on the real corpus: cc58cbc under `[diff] external = /usr/bin/true` exits 0 with an empty findings sink, against 853 + 298 on a clean config. Same shape as the blame.ignoreRevsFile false green the previous commit fixed. Pinned only on the hunk-producing diff -- measured inert on the --name-only enumeration, which runs no diff driver. --no-textconv, on the diff AND the blame. A `.gitattributes` entry like this repository's own `*.md diff=markdown` names a driver; a global `diff.markdown.textconv` turns that name into a content transformer, and --no-ext-diff does NOT cover it. Pinning only the diff is worse than pinning neither: attribute_file computes -L ranges from the diff's view and hands them to blame, so two disagreeing views silently re-cut an 80-line single-culprit finding into 60 + 20. git blame applies textconv by default, which its own -h and manual page never mention. --no-show-signature. The ruleset requires signed commits, so every commit scanned on main is signed. `log.showSignature = true` prints the verification verdict on stdout ahead of the output, so `%B | head -1` yields `Good "git" signature for ...` and both SUBJECT-anchored intent forms stop matching. The one deliberate revert in main's history then fires. It moves no count and fails toward firing, so it is a spurious detection rather than a missed one -- but it is still a wrong answer, and the report prints gpg noise where the subject belongs. `-l0` is deliberately NOT added. It would override git's own `diff.renameLimit` default rather than pin a caller's override of it -- a behaviour change on every machine including CI, and one that removes a bound on an O(N^2) cost. Filed separately instead. The note is rewritten to claim only what is testable and true: identical per-culprit counts regardless of the CALLER's git configuration, which is exactly what t_counts_are_immune_to_ambient_git_config checks. It now names each pin's measured load-bearing call site, and states plainly what is NOT covered -- the repository's own tracked .gitattributes, where `-diff` on *.lock and `binary` on assets make deletions contribute zero and no command-line flag overrides them, and git's own defaults as distinct from a caller's overrides. A PR about a header stating untrue things must not ship a new superlative it cannot defend. Also ends the `declared` note with a newline. Without it, range mode glued the next commit's verdict onto it (`...revert typeok 04a2ec1 fix(...)`), which also hid that line from any `^ok ` match. Verified: --verify-known-incidents reproduces all four recorded rows unchanged, so no calibration figure moves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018S8a1S71VxhLTRWBtMuEvp
…each pin discriminates Two gaps, both of which let a case report `ok` while testing nothing. FAIL-OPEN HARNESS. mk_repo yields MK_REPO_FAILED when mktemp or git init fails, and `(cd "$missing" && ...)` exits 1 -- byte-identical to the detector firing. Every case asserting rc=1 therefore passed on a fixture that was never built. Measured by forcing mk_repo to fail: nine cases reported `ok`, including all four false-positive-resistance cases -- the ones that must fire. mktemp really did fail transiently during #2837's development, so this is a measured mode, not a hypothetical one. run_canary now checks the fixture is usable first and fails with a named reason, and rc becomes 99, outside the detector's 0/1/2 contract so no assertion can mistake it for an expected status. Same forced failure now yields 0 passed, 94 failed. UNDISCRIMINATED PINS. The immunity case proves the shipped detector does not move under hostile config; it cannot prove any INDIVIDUAL pin is what holds it still, and a pin nobody can show is load-bearing is one a future tidy deletes as noise. Four cases now strip exactly one flag from a copy of the detector and assert the outcome MOVES: -M pure `git mv` of a 500-line file under `diff.renames = false`: pinned rc 0, stripped rc 1 with a spurious <culprit> 500. Measured load-bearing at the --name-only enumeration only; stripping it from attribute_file's pathspec-limited diff changes nothing. --no-ext-diff 300-line drop under `diff.external`: pinned rc 1 with findings, stripped rc 0 with an empty sink -- a false green. --no-textconv 300-line drop under a line-count-changing `diff.markdown.textconv`, asserted against BOTH the both-stripped copy and the blame-only-stripped copy, since the mixed case is the one that silently reattributes rather than erroring. --no-show-signature a signed `revert:` subject under `log.showSignature = true`: pinned rc 0 and declared, stripped rc 1. strip_pin fails loudly when its sed matches nothing, because a no-op edit would leave the "stripped" copy identical to the shipped one and the case would pass while comparing a script against itself -- the same fail-open shape as an unbuilt fixture. The signature fixture needs a genuinely signed commit: with commit.gpgsign false git has nothing to verify, prints nothing extra, and the case would pass either way. It grafts a gpgsig header onto the commit object directly, which needs no signing program, key, keyring, agent or network. An ephemeral ssh key does not work -- ssh-keygen refuses a private key whose permissions it deems too open, and Windows temp directories hand out exactly such ACLs. The grafted signature need not verify; git prints its verdict either way, and an unverifiable one prints more noise, not less. The immunity case also gains `diff.external` in its hostile config, so the property claim and the pin set stay in step. Also covers the `declared` note's trailing newline: only range mode shows the gluing, because --commit mode stops after the one commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018S8a1S71VxhLTRWBtMuEvp
|
Claude finished @kyle-sexton's task in 1m 19s —— View job Security review complete
No security issues found. ScopeThis is a What changed
AssessmentSkip-gate criterion 2 applies: no security-relevant surface.
|
The `--no-textconv` discrimination case passed vacuously and then failed outright on CI, with all four arms reporting the identical count. Two fixture defects, both of which made the experiment never run rather than making the pin inert: - The transformer only PREPENDED lines. Prepending shifts a hunk's starting offset but leaves its length alone, so every arm reports the same count no matter which pins are stripped. Replaced with `sed p`, which doubles every line and therefore changes the count. - The transformer was a helper script inside the fixture directory. git spawns a textconv through the platform's native process API, which on Windows cannot resolve the MSYS-style path `mktemp -d` returns, so the driver failed to spawn and every arm reported zero. `sed p` is a bare PATH-resolved command with no script file and no absolute path. The case now also proves the driver is live independently of the canary: a raw unbounded `git blame --line-porcelain` is run with and without the driver config, and the case fails with a distinct message if the filtered line count is not double the unfiltered one. That failure says "fix the fixture", not "the pin is inert" -- the two were previously indistinguishable. The fixture gained 150 lines of earlier content ahead of the culprit's block so misaligned -L ranges can slide off one commit's lines onto another's. At that size the mixed arms fall below the 200 threshold and go silent (rc 0, empty sink), which is the real harm the pin prevents: a true finding suppressed, not merely miscounted. Co-authored-by: Claude <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 51s —— View job Security review complete
No security issues found. ScopeThis is a What changedFixes
AssessmentSkip-gate criterion 2 applies: no security-relevant surface.
|
…ripped arm Three findings from an independent audit of the pin cases, none of which changed the verdict that the --no-textconv pin is load-bearing, but all of which weaken what the case can claim. `strip_pin` guarded against a sed that matched nothing by comparing the whole file. That guard is satisfiable by comment collateral alone: the flag names these expressions match also appear in the prose above the calls they target, so reflowing the blame call at line 467 would still rewrite the comment at line 456, the copy would still "differ", and the arm would silently compare the shipped detector against a comment-only edit. That is exactly the fail-open the guard exists to close. The comparison now ignores comment lines, so only a CODE change counts as a match. Verified by construction: a sed restricted to comment lines makes the whole-file comparison report "differs" and the code-only comparison report "identical", which now fails the case. The both-stripped arm asserted only that its output differed from the pinned run, while the comment table above it states a specific number. `sed p` duplicates every line, so an unpinned diff and an unpinned blame agree on a count exactly twice the truth. The arm now asserts that doubling, which is the figure the table names. Inequality would also be satisfied by a count that is merely different, which is how a case that measured something drifts into a case that only observed a change. The pin table in the detector claimed a measurement that does not reconcile with its own premise: a textconv prepending ten lines was said to re-cut an 80-line single-culprit finding into 60 plus 20, but a ten-line shift reconstructs to 70 plus 10. The fixture that produced it no longer exists, so rather than restate an unreproducible figure the comment now points at the shipped test case, which owns the numbers and re-measures them on every run. Co-authored-by: Claude <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 44s —— View job Security review complete
No security issues found. ScopeThis is a What changed
AssessmentSkip-gate criterion 2 applies: no security-relevant surface.
|
Both sides harden the same canary against different false greens, and both survive here intact. This branch pins the detector's git invocations (--no-ext-diff, --no-textconv, --no-show-signature) so ambient user configuration cannot move the numbers, teaches the canary to read a `revert:` subject as declared intent, and adds bracketed attribution assertions so a recorded incident that still fires but attributes differently is caught rather than passed. Main brought #2873, which fixes #2855: a partial re-land of a recorded incident used to pass the replay, so the canary could report health while the repository was still missing merged work. Its mechanism is the `marker` rows in scripts/silent-revert-incidents.txt and the `--verify-restoration` mode that reads them. The two conflicts were both additive. In the incidents file, every `fires` row now carries both this branch's attribution field and main's marker rows, and the header keeps both explanations. In the test file, both sides' cases are present and both are called; the merged suite runs 101 cases, which is exactly the 27 cases the merge base had plus the 27 this branch added plus the 47 main added. The attribution figures recorded here were re-measured under the pinned detector rather than carried over: 346 for f603880, 451 for 9239f15, and 853 plus 298 for cc58cbc. The 298 is the myers figure the pins make reproducible; the same attribution reads 301 under diff.algorithm = histogram, which is the divergence these pins exist to close. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwdkpWf6bptu3AqTMoeg2H
Catching the branch up to the current main before push. The commit this brings in adds a docs-hygiene skill and touches none of the canary's files, so the reconciliation resolved in the previous merge stands unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwdkpWf6bptu3AqTMoeg2H
|
Claude finished @kyle-sexton's task in 2m 8s —— View job Security review complete
No security issues found — skip gate applied (criterion 3: this head already has a successful security review that still applies). Why this head is skippedThe last security-reviewed head was Verified this concretely rather than assuming it from the merge commit messages:
So there is nothing new to review here. Every commit that makes up this PR's actual |
…tector PR #2843 pinned attribute_file's diff and blame flags to git's own defaults. Three of the figures the calibration prose quotes move under those pins, because the original calibration was taken on a machine carrying diff.algorithm = histogram: #2640 reads 447 rather than 390, #2135 reads 323 rather than 340, and #2642's share of #2633's squash reads 298 rather than 301. A calibration comment that states a number the shipped detector no longer produces is the defect this branch exists to fix, so every sentence carrying a figure was re-measured rather than patched. The overlap argument survives and is stated more strongly. The smallest true finding is 298 and both cleared fires score above it, at 323 and 447, so the populations invert rather than merely abut. The 200-versus-300 sentence inverted outright and was rewritten from measurement. It claimed the finding survived a threshold of 300 by a single line; under the pins that finding is 298 and disappears instead. The commit set at 200 and 300 is still identical, because cc58cbc keeps its 853-line finding, but the finding set is not, and running the replay at 300 reports cc58cbc as firing but not as recorded. Three further corrections the re-measurement surfaced. The corpus endpoint is now named as a sha rather than written as "the last 500 first-parent commits", which is a moving window that falsifies itself on the next merge. Detection and disposition are now distinguished: five commits cross the threshold but two are cleared by the acknowledgment file, so a reader sees three. And the two recall gaps are acknowledged rather than implied, so the corpus figures read as floors by construction: attribute_file swallows git's stderr (#2880), and paths marked -diff or binary in .gitattributes contribute nothing (#2883). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwdkpWf6bptu3AqTMoeg2H
…tor's counts (#2847) Closes #2846. ## Summary The calibration comments in `scripts/check-silent-revert.sh` carry the argument that justifies the 200-line threshold. Two of their sentences depended on which finding is the smallest, and #2832 had already falsified both by recording a fourth true finding. This PR repairs them — and re-derives every figure they rest on against the detector as PR #2843 pins it, because three of those figures move under the pins. Found by fresh-context verification of #2832, after it had merged. ## Why the numbers moved PR #2843 pins `attribute_file`'s git invocations to git's own defaults (`--diff-algorithm=myers`, `--no-ext-diff`, `--no-textconv`, `--no-ignore-revs-file`, `-M`). The original calibration was taken on a machine carrying `diff.algorithm = histogram`, and the algorithm choice changes which lines a hunk calls deleted. Every figure below was re-measured against the pinned detector and reproduced byte-identically with `GIT_CONFIG_GLOBAL` emptied, which is the property `t_counts_are_immune_to_ambient_git_config` asserts. | commit | PR | pre-pin | pinned | class | | --- | --- | ---: | ---: | --- | | `cc58cbc53` | #2633 | 853 | **853** | incident | | `cc58cbc53` | #2633 (#2642's share) | 301 | **298** | incident | | `9239f1541` | #2641 | 451 | **451** | incident | | `f603880da` | #2639 | 346 | **346** | incident | | `6f0a31109` | #2640 | 390 | **447** | cleared | | `91e77fc16` | #2135 | 340 | **323** | cleared | ## Fix **The overlap argument survives; every sentence stating it was re-derived.** The smallest true finding is 298 and both cleared fires score above it, at 323 and 447. The relationship is an inversion, not a narrow gap — so `NO THRESHOLD SEPARATES THEM` is now true by a wider margin than the six-line version it replaces. THRESHOLD stays 200. **The 200-vs-300 sentence inverted and was rewritten from measurement, not patched.** The old text said "at 300 the 301-line finding survives by a single line". Under the pins that finding is 298, so at 300 it VANISHES. The COMMIT set at 200 and 300 is still identical — `cc58cbc53` keeps its 853-line finding — but the FINDING set is not, and that is the sharper argument against tuning. Measured: ``` $ SILENT_REVERT_THRESHOLD=300 scripts/check-silent-revert.sh --verify-known-incidents FAIL cc58cbc fires, but NOT as recorded recorded attribution: bfb66be 853 eda5ae5 298 what the detector reported: bfb66be 853 EXIT=1 ``` Without #2833's attribution expectations that row would have passed on the surviving 853-line finding and announced a reproduction it never performed. The passage now says that, and cites `t_replay_asserts_the_recorded_attribution`, which pins the same two-culprit shape. **Detection and disposition are now distinguished.** The corpus sentence said the canary "fires on 5 commits" and left a reader to assume that is what CI shows. It is not: `6f0a31109` and `91e77fc16` are in `scripts/silent-revert-acknowledged.txt`, so `scan_commit` clears each before it attributes a line. Five commits cross the threshold; three print `SILENT REVERT SUSPECTED`. The prose now states both and says which one the threshold is calibrated against. **The corpus endpoint is pinned.** "the last 500 first-parent commits of main" is a moving window that falsifies itself on the next merge — the same defect class this PR closes. It now reads "the 500 first-parent commits of main ending at `738791c45`". **Two recall gaps are acknowledged rather than implied.** - `attribute_file` swallows `git` stderr on both commands that produce a count, so a failed diff or blame is indistinguishable from nothing-to-attribute and can only subtract. Every corpus enumeration is therefore a floor, and the prose is worded so the caveat is structural rather than an appended qualifier (#2880). - Paths the repository's `.gitattributes` marks `-diff` or `binary` produce no hunks, so their deletions attribute to zero on every machine including CI (#2883). This is a RECALL gap, not a calibration one: no path of that class appears in any commit whose figure is quoted, and the largest such deletion anywhere in the sweep was 72 lines from a `package-lock.json` — well under the threshold. **Also corrects a mischaracterization of #2656** that #2832 introduced, which said that issue recorded the event "rather than as a silent revert". Its Evidence section opens with *"#2633 was a stale-base squash that silently reverted two merged features."* It recorded it exactly as a silent revert — what was coverage-shaped was what it **asked for**. Now quoted rather than paraphrased. ## Not fixed here The fresh-context verifier confirmed each of these; every one sits in prose this PR does not own, and each needs a rewording rather than a renumber. - **"main's 1527-commit history"** (twice). Reproduces at no named endpoint — measured 1546 at `738791c45`, 1549 at `origin/main`. The claims it supports are unaffected and do reproduce: `Revert "` = 0, `revert:` = 1, that one being `1d1fca6e8` (#1839). Renumbering it would be falsified by the next merge, which is the same moving-window defect this PR removes elsewhere. - **"the replay exited 0 for 31h28m"**. The duration reproduces exactly, but it is the content-absence window; the replay itself only existed for about 6h13m of it. #2873's prose, and the same conflation appears once in `silent-revert-incidents.txt`. - **"a four-row corpus"**. There are 5 `marker` rows, and 3 `fires` + 1 `clean` expectation rows; the sentence's own unit is one read per marker, so 5. #2873's prose. - **The repo-wide-grep counterfactual.** Its present-tense half holds, but at the tree where both markers were actually missing, a repo-wide grep would have falsely cleared only the README half — the CHANGELOG copy that makes the claim true today was added by the restore commit itself. #2873's prose. - **The `clean` row's "129 lines"** reads 136 under the pins. Issue #2865 owns that row; correcting it here would collide. ## Related - PR #2843 — merged ahead of this one; it added the pins that move three of the figures here, and its pin table records the pre-pin and pinned columns side by side. This PR layers prose on top of it and changes no pin. - PR #2873 — merged ahead of both; added the restoration markers and the `marker) continue ;;` arm. Untouched here and verified intact after the rebase. - Refs #2880 — `attribute_file` swallows git stderr, which is why the corpus figures are worded as floors rather than exact counts. Acknowledged here, not fixed. - Refs #2883 — paths marked `-diff` or `binary` contribute zero to attribution. Acknowledged here as a recall gap, not fixed. - Refs #2865 — owns the `clean` row whose "129 lines" reads 136 under the pins. Deliberately left to that lane. - Refs #2832 — the corpus attribution correction that added the fourth finding and falsified the two sentences this PR repairs. ## Verification - Every figure re-measured against the shipped detector on this branch, twice — once inheriting ambient config and once with `GIT_CONFIG_GLOBAL` emptied — with identical results. - `scripts/check-silent-revert.test.sh` (101 passed, 0 failed) and both replay modes run green against the merged content. - An independent fresh-context verifier re-measured every number in the calibration comments without access to this reasoning, running the full 500-commit corpus sweep rather than per-commit checks alone. Every figure this PR states reproduced. It raised two defects in the new prose, both fixed here: "the number a reader sees in CI is 3" read as findings when it means commits (three commits, four findings between them), and "roughly once a month" was 12-19x off — the corpus spans 7.9 days with four of its five crossings inside 76 minutes, so that rate claim was removed rather than renumbered, because the corpus measures a burst and no per-month figure is defensible from it. Its full verdict, including drift it confirmed in prose this PR does not own, is recorded in the PR comments. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…sub-threshold miss (#2905) Closes #2865. ## What this fixes `scripts/silent-revert-incidents.txt` pins a `clean` row that is supposed to be the closest a non-incident got to the 200-line threshold without crossing it — the row that breaks first if a threshold change starts taxing ordinary development. The pinned commit was not that, and its note named a number that is not a pull request. **Defect 1 (the re-pin).** Measured over the file's own 500-commit corpus (`7b47d2253~500..7b47d22`) at the pinned invocations (#2843, `GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1`), the pinned `c8470efd0` scores **136** blamed lines — sixth-closest of the eight commits in the 100–199 band. The true closest miss is **`9a2307c43` (#2189) at 195 lines** from `3584ae1fa` (#2183) — a margin of 5 lines, not the ~71 the old row implied. `9a2307c43` is now the lead `clean` row. **Defect 2 (the wrong PR number).** The old note credited the deleted content to #2679, which is a closed issue in this repository, not a pull request (`gh pr view 2679` cannot resolve it). The blamed lines trace to `6370a44e7`, the squash commit that landed #2715 — `gh api .../commits/6370a44e7/pulls` returns only #2715, and the commit's own body says `Builds on merged #2715 → #2692 → #2690`. The row is kept as a second guard with its note corrected, rather than dropped: it is still a verified-legitimate quiet commit, and keeping it costs a few lines of prose. **Defect 3 (the "once a month" rate)** was already fixed by #2847, which removed the rate claim from `scripts/check-silent-revert.sh` entirely. Nothing in this PR touches it. ## What this does NOT do Neither clean-row figure is CI-asserted, before or after this change. `clean` rows carry no bracketed attribution field (#2879) — their assertion is the absence of findings, which has no per-culprit count to pin — so the 195 and 136 are hand-measured prose, not watched numbers. The old row's recorded 129 drifting to a measured 136 under the pinned invocations without anything going red is exactly that gap, and the section comment now states it so a reader does not mistake the re-pin for an assertion. Both counts are written as floors ("no fewer than") because `attribute_file` drops `git blame`'s stderr (#2880), so any line blame fails on is silently not counted. ## Verification - Spot-checked both figures against PR #2843's pinned invocations before editing: `9a2307c43` reproduces **195** (75 lines `song-forms-examples.md`, 67 `box-model.md`, culprit `3584ae1fa`), `c8470efd0` reproduces **136** (109 lines `persist-findings.md`, culprit `6370a44e7`). - `bash scripts/check-silent-revert.test.sh`: **101 passed, 0 failed** on this branch. - `scripts/check-silent-revert.sh --verify-known-incidents`: exit 0 — all three `fires` rows reproduce their attributions exactly, and both `clean` rows stay quiet. - `scripts/check-silent-revert.sh --verify-restoration`: exit 0 — all 5 markers present. - A fresh-context verifier independently swept all 500 corpus commits twice (complete coverage: 487 `ok` + 2 `acknowledged` + 11 finding commits = 500) and reproduced every figure in the file. Its verdict: **195 at `9a2307c43` is the highest sub-threshold score** — the next highest is 188 (`3d69448cb`) — so the lead `clean` row pins the true closest miss. The two acknowledged commits were re-run with the ack file disabled and score 447 and 323, both above the threshold, so neither could displace it. ## Related - #2843, #2847, #2873 — the three PRs that reshaped the canary and this file ahead of this change; the figures here are measured under #2843's pinned invocations. - #2879 — records that `clean` rows carry no bracketed attribution field, which is why neither figure in this PR is CI-asserted. - #2880 — records that `attribute_file` drops `git blame`'s stderr, which is why both counts are written as floors. - #2831 / #2832 — the same wrong-PR-number defect shape, corrected earlier on the `fires` rows. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LwdkpWf6bptu3AqTMoeg2H --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…omma (#3081) Closes #2874 Closes #2875 ## Summary The silent-revert canary's self-proof lane could report a green replay after verifying nothing. `verify_known_incidents` used `while read` without keeping a last line that lacks a trailing newline, and it had no row-count floor, so an empty file, a comments-only file, or a one-row file whose only row was dropped all printed "Canary reproduces every recorded incident" and exited 0. The same read idiom dropped the last acknowledgment-file row (fail-closed). Attribution grammar from #2843 is already on main; a trailing comma in `[sha=n,]` was still accepted because word splitting under `IFS=','` drops a trailing empty field, against the file's own "exactly that grammar / exit 2" doctrine. ## Fix In `scripts/check-silent-revert.sh`: - `verify_known_incidents` and `ack_reason` now use `while read … || [[ -n "$var" ]]` so a final unterminated row is processed rather than dropped. - `verify_known_incidents` counts verified rows and dies (exit 2) when the count is zero — an empty or comments-only incident file is a broken input, not a pass. - `parse_attribution_field` rejects a field that starts with, ends with, or contains a doubled comma before splitting, so a trailing comma is exit 2 like every other empty-entry position. ## Test plan `bash scripts/check-silent-revert.test.sh` — 112 passed, 0 failed. New cases pin the empty-file / comments-only floor, newline vs no-newline two-row parity (fixtures written with printf so the trailing newline is real), a one-row file with no trailing newline, an unterminated last ack row, and all four comma positions. ## Verification - `bash scripts/check-silent-revert.test.sh` — 112 passed, 0 failed - `shellcheck -x scripts/check-silent-revert.sh scripts/check-silent-revert.test.sh` — clean - New coverage: - empty and comments-only incident files exit 2 (not the success banner) - a two-row file without a trailing newline reaches the same FAIL as the same file with one (fixtures differ by exactly one trailing byte) - a one-row file with no trailing newline still verifies that row - an acknowledgment file whose last row has no trailing newline still clears - all four empty-entry comma positions (`[sha=n,]`, `[,sha=n]`, `[sha=n,,sha=m]`, `[,]`) exit 2 ## Related Refs #2843 (attribution grammar already merged; this PR only closes the remaining reader defects). Refs #2833 (why the attribution field exists). Refs #2808 (canary and incident corpus). --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
…on git errors (#3084) Closes #2879 Closes #2880 ## Summary The silent-revert canary's `clean` rows named a specific sub-threshold line count in their notes, but the grammar rejected an attribution field on those rows, so the figure was prose. Pinning the detector's diff flags moved 129 to 136 on the previous closest-miss row and nothing went red. Independently, `attribute_file` discarded git stderr on both commands that produce a count, so a failed diff or blame was byte-identical to "this file had nothing to attribute" and every corpus figure was a lower bound. ## Fix In `scripts/check-silent-revert.sh`: - A `clean` row may carry the same bracketed `[<sha>=<n>]` field a `fires` row does. The replay asserts the run's largest in-window sub-threshold attribution is exactly that set. The row's pass/fail is unchanged: nothing above the threshold is still what makes it clean. - `attribute_file` checks git diff/blame status separately from output. A non-zero status is exit 2 ("the canary could not run") with the captured stderr on the die message. Successful runs still discard stderr. `git-diff(1)` of two commits exits 0 on success even when the files differ (`--exit-code` is what would turn a difference into status 1), so any non-zero status here is a real failure. `git-blame(1)` has no analogous "found something" status. - `attribute_file` is no longer piped into awk, so `die` cannot land in a subshell and be swallowed back into a quiet zero. The two shipped clean rows now record their measured maxima: `3584ae1fa…=195` on `9a2307c43` and `6370a44e7…=136` on `c8470efd0`. Not in scope: `-diff`/`binary` paths (#2883) and `diff.renameLimit` (#2884). ## Verification - `bash scripts/check-silent-revert.test.sh` — 121 passed, 0 failed - `bash scripts/check-silent-revert.sh --verify-known-incidents` — both clean rows print `largest sub-threshold attribution reproduced exactly` - `bash scripts/check-silent-revert.sh --verify-restoration` — 7 markers present - `shellcheck -x scripts/check-silent-revert.sh scripts/check-silent-revert.test.sh` — clean - New coverage: - a clean row whose recorded count or culprit has moved fails (`stays clean, but NOT as recorded`) — the 129 → 136 shape - a firing commit labeled clean still fails on status - a failed blame or content diff is exit 2, including when reached through the replay - successful attribution still discards git stderr - every shipped clean row carries a well-formed attribution field ## Related Refs #2833 (why the attribution field exists on `fires` rows). Refs #2843 (pinned diff/blame flags that moved the unasserted clean-row figure). Refs #2874 / #2875 (reader defects just merged; this PR starts from that main). Refs #2883 (recall gap for `-diff`/`binary` paths — not this PR). Refs #2884 (`diff.renameLimit` — not this PR). Refs #2808 (canary and incident corpus). --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
…etection (#3088) Closes #2883 Closes #2884 ## Summary `scripts/check-silent-revert.sh` attributed deleted lines from `git diff` hunks, then blamed those ranges. Two holes sat next to that design: paths the repo's own `.gitattributes` mark `-diff` or `binary` emit `Binary files differ` and contribute zero lines (#2883), and git's default `diff.renameLimit` of 1000 can decompose a large basename-changing content-touched relocation into adds plus deletes -- a false positive on a clean machine, no hostile config required (#2884). #2843 was scoped to pins only and correctly left both out. ## Fix In `scripts/check-silent-revert.sh`: - When the path-limited content diff produces no hunks and a `Binary files` header, recover old-side ranges from a **blob-to-blob** diff. That invocation has no path, so no attribute. Git's content heuristic still treats a genuine binary as binary. `--text` / `-a` is not used: gitattributes(5) Unset on `diff` generates `Binary files differ`, and `--text` would force hunks through that -- a 200 KB random blob is ~800 "lines" and would fire at the shipped threshold of 200 from a commit that reverted nothing. - Pass `-l0` on the `--name-only` enumeration in `scan_commit` only. git-diff(1): `-l` gates the exhaustive O(N²) rename pass; "a value of 0 is treated as unlimited". git-config(1) `diff.renameLimit` defaults to 1000 and is equivalent to `-l`. Exact-rename and basename-preserving pre-passes are not limit-gated; the exposure is relocate more than ~1000 files AND change their content AND change their basenames. `-l0` is inert on `attribute_file` (pathspec-limited, no pair can form). Enumeration stderr stays visible so a skipped exhaustive pass still announces itself if a future bound is too low. Not `--text`. Not a pin of a default. `-l0` overrides git's own bound; that is the honest fix #2884 describes. ## Verification - `bash scripts/check-silent-revert.test.sh` — 130 passed, 0 failed (was 121) - `bash scripts/check-silent-revert.sh --verify-known-incidents` — all five rows reproduced exactly (853 / 451 / 346 / 298 / 195 / 136 unchanged) - `bash scripts/check-silent-revert.sh --verify-restoration` — 7 markers present - `shellcheck -x scripts/check-silent-revert.sh scripts/check-silent-revert.test.sh` — clean - New coverage: - `-diff` lock file (320 lines) + binary png churn + 25-line text file: lock and text fire, png does not - stripping the `Binary files` recovery makes the lock deletion a FALSE GREEN at threshold 200 - detector code does not pass `--text` - a wholly deleted `-diff` lock file is attributed (empty-blob branch) - a 320-line ASCII PDF-like blob marked `binary` stays clean - a 200 KB binary png re-export stays clean - 1001-file basename-changing content-touched relocation stays clean (`-l0`) - stripping `-l0` under `diff.renameLimit=1` makes a 12×20 relocation fire ### Full-scan runtime (#2884 acceptance) Measured on this machine (git 2.43.0) over the 500 first-parent commits of `origin/main` ending at `8b61b2b1` (`ab012e219..HEAD`), plus the 1001-file basename-changing content-touched fixture — the commit that finally is not a no-op for `-l0`. | run | elapsed | result | | --- | --- | --- | | 500-commit scan, origin/main script | 131.606 s | exit 1 (known incidents fire; expected) | | 500-commit scan, this PR (`-l0` + blob recovery) | 132.253 s | exit 1, same findings | | 1001-file fixture, `-M` (git default limit) | 0.003 s | `1001 A` + `1001 D`, warning: exhaustive rename detection skipped | | 1001-file fixture, `-M -l0` | 0.104 s | `1001 R`, no warning | The full scan moved by 0.6 s (0.5%). That is not material. `-l0` is the landing; `-l5000` is not needed. The 1001-file exhaustive pass costs 104 ms on this machine — the bound git's default exists to impose is not paying for anything at this corpus size. ## Related Refs #2843 (pins only; `-l0` and `-diff` were correctly excluded). Refs #2847 (header calibration; this PR updates the #2883/#2884 limitation notes that lane wrote). Refs #2879 / #2880 (just merged; this PR starts from that main). Refs #2691 / #2808 (canary and incident corpus). --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Kyle Sexton <kyle-sexton@users.noreply.github.com>
Closes #2837. Closes #2833.
Summary
Three defects in the silent-revert canary, plus one unfiled harness-safety fix
in a file this change already owns. Branched off
2de57a379(#2832), which hadalready merged.
Fix
1.
declares_removal()could not read the only revert subject that merges here (#2837)The detector accepted three intent forms. This repo is squash-only with
squash_merge_commit_title: PR_TITLE, so the squash subject is the PR title,and
.github/workflows/pr-title.ymlgates every title through a requiredConventional-Commits check whose default type list is all-lowercase — it admits
revert:and contains nothing aRevert "…"subject could match. So adeliberate revert reached
mainwearing a subject the detector could not read.declares_removal()now also accepts the Conventional-Commits revert type,anchored at the start of the subject, requiring the literal lowercase token,
its optional
(scope)and/or!, its colon, and a non-empty description:Never a substring search for "revert" — kept exactly as constrained as the three
forms beside it.
Before / after on the repo's one real deliberate revert
1d1fca6e8—revert: remove Cursor dual-target marketplace manifests (#1835) (#1839)Before (at
2de57a379, shipped detector, thresholds unmodified):After:
Suppression is not widened over anything the corpus records
revert-prefixed subject —f603880dafix(disk-hygiene): …,9239f1541feat(disk-hygiene): …,cc58cbc53fix(repo-fleet-hygiene): …,c8470efd0docs(conventions): …. All four rowsstill hold (green run below).
6f0a31109fix(repo-fleet-hygiene): …,91e77fc16fix(hook-utils): …— so no ack rowgoes dead now that
declares_removal()short-circuits ahead ofack_reason().unchanged; none of those five is revert-prefixed.
New tests pin both directions.
revert:,revert(scope):,revert!:andrevert(scope)!:suppress;feat: do not revert the alpha guard (#99),reverted: drop the alpha guard (#99),Revert: drop the alpha guard (#99)anda bare
revert:with no description all still fire. The pre-existing case onlycovered a body mention of "revert"; the subject is what the new form reads, so
that is where a substring bug would widen.
2.
verify_known_incidentsasserted only "something fired" (#2833)A
firesrow passed onscan_commit's exit status while the note beside itnamed a specific culprit and a specific line count that nothing checked. On
cc58cbc53, whose deletions trace to two culprits, losing theeda5ae5edattribution entirely would still have printed
okon the survivingbfb66beb8finding — the canary announcing a reproduction it did not perform.
A
firesrow may now carry a bracketed attribution expectation after its sha:and the replay asserts the run's findings are exactly that set — same
culprits, same per-culprit counts, no extras, no omissions. Full 40-character
culprit shas only, the same discipline
silent-revert-acknowledged.txtuses.Counts come from a new
FINDINGS_SINKfile thatreport_findingappends<full-culprit-sha> <count>to, not from scraping the human report — the reportprints a 9-character abbreviation, which is not enough sha to assert on. Nothing
else sets
FINDINGS_SINK, so--commitand range mode are byte-identical.Every recorded figure was measured, not transcribed from the notes — the
sink was wired first and the observed values recorded:
Three of the four agree with the notes #2832 corrected; the fourth is 298 rather
than 301, for the reason in section 5. A malformed field is exit 2 (cannot
run), never a FAIL and never a pass — a silently misread expectation is the same
false green this file exists to remove. The field is optional so a row can be
pinned before its attribution is measured, but a new well-formedness assertion
requires every shipped
firesrow to carry one.3. The workflow header stated a reason that was not true (unfiled)
.github/workflows/silent-revert-canary.ymlasserted "There is nopull_requesttrigger, so it can never gate a PR" — while its ownon:blockhas a paths-filtered
pull_requesttrigger, added by #2808 to run the detector'sunit tests, and explained at length 20 lines further down in the same file. The
conclusion holds for a different reason: both scan steps are gated
if: github.event_name != 'pull_request', and the lane sits outsideci.ymland its
ci-statusaggregate. Wording corrected to the actual reason.No trigger and no
if:changed.git diffon that file is comment linesonly — one hunk,
@@ -13,6 +13,9 @@, entirely inside the#header.4. The detector's line counts depended on ambient git config (unfiled, found by #2833's new assertion)
The first CI run of this branch went red, and the failure is the most valuable
thing in this PR. The runner reported the
eda5ae5edattribution as 298lines where my machine measured 301:
Cause:
attribute_filecalled baregit diff --unified=0, so it inheritedwhatever
diff.algorithmthe caller's config carried. I havediff.algorithm = histogramset globally; CI has nothing set and therefore usesgit's default
myers. The algorithm changes which lines a hunk calls deleted,so it changes the per-culprit counts this canary thresholds on. Reproduced
directly:
Three lines is harmless in itself. The principle is not: the same drift can
carry a count across the 200-line threshold, so a commit could fire on one
machine and stay silent on another, and the header's "fires on 5 commits over
500 — 1%" calibration only ever described one algorithm.
attribute_filenow pins--diff-algorithm=myers -Mexplicitly, and the fileenumeration pins
-Mtoo. Both are git's defaults, so this does not changewhat CI detects today — CI already had no
diff.algorithmset. It makes alocal run match CI, not the reverse.
-Mcovers the same exposure for renamedetection, which the existing header calls "load-bearing rather than
incidental":
diff.renames = falsein a developer's config would decompose agit mvinto delete + add and make relocating a large recent file fire.The recorded figure is now 298, and the prose figures in the script header
and the corpus are corrected with the refutation attached, so nobody
re-measuring on a histogram machine "corrects" it back to 301.
Worth stating plainly: nothing asked for this. #2833's exact-count assertion
turned a silent, config-dependent divergence into a red build on its first run —
which is precisely the argument for asserting attributions instead of exit
status.
5. The test harness could commit the developer's work as
test <t@t.test>(unfiled)Found the hard way while developing this.
mk_repois called asrepo="$(mk_repo)", so areturn 1inside the command substitution cannot abortthe suite — the caller just gets
"". And""is not inert:git -C ""isdocumented as a no-op, so the next
add -A+commitstaged and committed myuncommitted work into the checkout, authored
test <t@t.test>. Such a commitcannot be pushed here — it fails
required_signatureswithno_user.mk_reponow yields a path derived fromSELF_DIRthat does not exist, so everygit call against it fails loudly and the assertions go red — fail-closed, which
is what a harness that cannot build its fixture should do. Scoped to this file
only; if the same
repo="$(mk_repo)"shape exists in sibling harnesses that is aseparate follow-up.
Verification
scripts/check-silent-revert.sh --verify-known-incidents, run on a machine withdiff.algorithm = histogramset globally — it now agrees with CI exactly,because the flags are pinned:
Injected failure against the shipped corpus (
853changed to852in acopy) — the mechanism is proven on real rows, not only on synthetic fixtures:
The commit still fires — exit status alone would have passed this row. Note the
row is red on a one-line discrepancy in one of two attributions, which is
exactly the regression #2833 describes.
scripts/check-silent-revert.test.sh: 49 passed, 0 failed, includingreplay fails when the finding is attributed to a different culprit,replay fails when the recorded line count no longer reproduces, andreplay fails when one of two recorded attributions stops reproducing. So theassertion is proven by permanent tests, not only by a one-off injection.
6. Review follow-up: an unterminated attribution field read as absent
Both automated review lanes independently flagged the same real gap, and they
were right. A
firesrow whose field opened with[but never closed it failedthe
[[ "$rest" == \[*\]* ]]glob, soattributionstayed empty, the remainderbecame free-text
note, and the row fell back to passing on exit status alone —reintroducing the exact pre-#2833 gap by the one route nobody would look at, and
contradicting the contract documented directly above it.
A leading
[now COMMITS the row to carrying an attribution; unterminated takesthe malformed path. Reproduced against the real corpus with the closing bracket
stripped from the
f603880darow:The four malformed shapes already pinned all carried a closing
], so this onewas untested;
[<sha>=40and a bare[are now pinned too. The shipped corpuswas never at risk —
t_shipped_data_files_are_wellformed's regex covers it —but that is a separate layer and does not hold for a custom
SILENT_REVERT_INCIDENTS.7. Verification follow-up:
git blamewas still ambient-config dependentFresh-context verification of section 4 found that fix was only half of one.
Pinning the diff flags left
attribute_file'sgit blamecall bare, soblame.ignoreRevsFile— an ordinary setting in any repo carrying abulk-reformat commit — still decided the per-culprit counts the replay now
asserts on. On the real corpus, with that setting naming
bfb66beb8:blame.ignoreRevsFilebfb66beb8…eda5ae5ed…On a synthetic fixture it is worse than a wrong number. With the pin absent and
that config present, the detector reports no finding at all on a genuine
silent revert:
culprit 40culprit 40culprit 40That is a false green reached through the developer's own gitconfig — the
precise failure this canary exists to remove.
The obvious fix does not work, and the comment says so.
-c blame.ignoreRevsFile=does not clear it: the documented "an empty filename resets the list" applies to the option, and the
-cform was measuredleaving the hostile value fully in effect (853 → 259 with the reset supposedly
applied). Only
--no-ignore-revs-fileactually resets. The symmetry with the-cpins above is wrong here and is deliberately not used.t_counts_are_immune_to_ambient_git_configpins the property: the same fixturescanned twice, once under a hostile
GIT_CONFIG_GLOBALsettingblame.ignoreRevsFile,diff.algorithmanddiff.renames, asserting identicalexit status and identical per-culprit findings. Confirmed discriminating —
with the blame pin stripped it fails, and it fails because the hostile run
reports nothing at all.
shellcheck,actionlint,typos,bash -nandscripts/check-shell-portability.shall pass.Test plan
Run from a clean checkout of this branch, with
unset GIT_DIR GIT_WORK_TREE:bash scripts/check-silent-revert.test.sh— the detector's own unit suite.Expect 49 passed, 0 failed. Covers all four
revert:spellings, thefour negative subject cases (
feat: do not revert ...,reverted:,Revert:, barerevert:), the wrong-culprit / wrong-count / missing-attribution replay regressions, all six malformed-field shapes, and
t_counts_are_immune_to_ambient_git_config.bash scripts/check-silent-revert.sh --verify-known-incidents— the realcorpus. Expect exit 0 with all three
firesrows reportingattribution(s) reproduced exactlyand thecleanrow staying clean.scripts/silent-revert-incidents.txttoinject a wrong culprit sha (leaving the count correct, so the commit still
fires) and, separately, a wrong line count. Each must exit 1 with
fires, but NOT as recorded. Restore the file afterwards.diff.algorithm = histogramin global git config and repeat steps 1-2.The numbers must not move — that is what the new flag pins buy.
git diff origin/main...HEAD -- .github/workflows/silent-revert-canary.ymlmust show comment-only changes; no executable YAML line may differ.
Related
silent-revert canary: verify_known_incidents proves only that a commit still fires, not which culprit or how many lines #2833 was raised in its review.
1d1fca6e8) used as the real-historyfixture for silent-revert canary: declares_removal() does not recognize a
revert:subject, the only revert spelling the PR-title gate permits #2837.