Skip to content

silent-revert canary: paths marked -diff or binary in .gitattributes contribute zero to attribution #2883

Description

@kyle-sexton

Deliberately excluded from #2843, which was scoped to pins only. This is a different exposure class and needs a different fix, so it is filed separately.

What happens

scripts/check-silent-revert.sh attributes deleted lines by asking git diff for the old-side hunk ranges of every modified/deleted file and then blaming those ranges. Any path whose .gitattributes say -diff or binary produces no hunks, so it contributes zero lines to attribution — on every machine including CI, with no unusual configuration involved.

The repo's own tracked .gitattributes puts a large, high-value file class in exactly that state (lines 40-61):

package-lock.json   -diff
packages.lock.json  -diff
*.lock              -diff
*.png    binary
*.jpg    binary
*.jpeg   binary
*.gif    binary
*.ico    binary
*.webp   binary
*.pdf    binary
*.zip    binary
*.gz     binary
*.7z     binary
*.woff   binary
*.woff2  binary
*.ttf    binary
*.otf    binary

Measured

Fixture (git 2.55.0.windows.3): one commit adds content, the next deletes 320 lines from a foo.lock, 320 from an img.png, and 25 from a plain.txt. The canary sees only the 25.

=== CANARY RUN threshold 20 ===
  lines lost   25  (threshold 20, window 40 commits)

  by file:
      25  plain.txt

At the shipped threshold of 200 the commit reports ok and exits 0. 640 deleted lines were attributed to zero.

The mechanism, precisely

The crux is where the path dies, and it is not where you would guess.

It is NOT excluded from enumeration. scan_commit's git diff --name-only -z -M --diff-filter=MD lists all three paths normally:

=== ENUMERATION: git diff --name-only -z -M --diff-filter=MD ===
foo.lock
img.png
plain.txt

It dies at ZERO HUNKS. attribute_file's content diff emits a binary-files line instead of hunks:

=== CONTENT DIFF foo.lock (attribute_file flags) ===
diff --git a/foo.lock b/foo.lock
index 0ec6411..93c7a61 100644
Binary files a/foo.lock and b/foo.lock differ

=== hunk count foo.lock ===
0

attribute_file therefore derives no -L ranges, hits its [[ "${#ranges[@]}" -gt 0 ]] || return 0 guard, and never runs blame at all.

Blame itself is unaffected. git blame --no-ignore-revs-file --no-textconv --line-porcelain -L3,322 HEAD^ -- foo.lock returns 320 content lines correctly attributed to the culprit; the same holds for img.png. The data is fully available — the detector just never asks for it.

So the accurate statement is "enumerated but contributes zero at attribution", not "invisible to enumeration". Any fix that reasons about the enumeration site will miss it.

No command-line flag overrides a -diff attribute — and --text MUST NOT be adopted

The pins that landed in #2843 do not help here: --no-ext-diff and --no-textconv leave hunks at 0. The only flag that produces hunks is --text / -a:

=== --text on foo.lock: hunk headers ===
@@ -2,320 +1,0 @@ HEADER LINE OF THE LOCK FILE AAAAAAAAAAAAAAA

Do not fix it that way. Measured on a genuinely binary blob — 204,800 bytes of random data carrying the binary attribute, re-exported once (one routine image churn, no revert of anything):

=== hunks without --text: ===
0
=== hunks WITH --text: ===
3
=== OLD-SIDE DELETED LINES that --text would feed to blame: ===
808
=== attribution --text WOULD produce (blame those ranges) ===
808 lines attributed to the culprit

808 lines attributed to a single recent culprit, from a commit that reverted nothing — four times the 200-line threshold. As shipped the same commit reports ok.

That figure is a property of the blob, not a coincidence: roughly 1 byte in 256 of random data is 0x0A, so a 200 KB binary asset is ~800 "lines" to a --text diff whatever it contains. (An earlier lane measured 817 on its own random blob; the difference is just the sample.) Every image, font, and archive in the repo becomes a several-hundred-line false positive the moment it is regenerated.

So --text converts a recall gap into a false-positive machine, and the whole design rests on the FALSE-POSITIVE STRATEGY section's premise that the canary fires roughly once a month. This paragraph exists so nobody reaches for -a as the obvious one-word fix.

Scope: a pure RECALL gap, NOT a calibration problem

This is the important framing and it should survive into whatever fix lands.

Zero lock-file and zero binary-attribute paths appear among the modified-or-deleted paths of ANY calibration commit. Verified across all six:

commit MD paths lock paths binary-attribute paths
f603880 10 0 0
9239f15 10 0 0
cc58cbc 6 0 0
6f0a311 4 0 0
91e77fc 55 0 0
c8470ef 12 0 0

(c8470ef has 14 changed paths total — 12 M plus 2 A — of which the --diff-filter=MD enumeration presents 12.)

So no derived figure in the file header was ever measured through this hole. The correct claim is "the detector cannot see one file class", not "the calibration is suspect". None of the 853 / 451 / 346 / 298 / 390 / 340 attributions move.

Impact worth naming

A silent revert of a dependency change is invisible to the canary today. package-lock.json is 58,940 bytes on main; a squash that dropped a sibling PR's lockfile update entirely would score 0 and report ok. Given that #2691's incident class is "a squash merge dropped work a sibling PR had just merged", a dependency bump is a plausible member of that class, and it is the one member the canary structurally cannot detect.

Exposure class — different from #2843

Worth stating because it changes what the fix looks like:

Different exposure class, different fix. A pin cannot address it.

Note for the triager

Do not write prose about this into scripts/check-silent-revert.sh's header. #2847's lane owns that header and will place this next to the existing corollary at lines 106-109 ("content reverted from OUTSIDE that window is missed by design"), which is the same class of stated-limitation claim. Coordinate there rather than adding a second, competing limitations note.

Related

Siblings from the same review pass: #2874, #2875, #2837, #2833, #2846, #2855, #2865, #2843, #2847, #2691, #2656.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions