Surfaced by fresh-context verification during #2847's calibration
re-measurement. Not blocking that PR, and deliberately not fixed in it.
The mechanism
attribute_file discards git's stderr on both of the commands that produce a
count — the per-file diff that enumerates deleted line ranges, and the blame
that attributes them:
git diff --unified=0 --no-color --diff-algorithm=myers -M \
"$parent" "$commit" -- "$file" 2>/dev/null |
...
git blame --no-ignore-revs-file --line-porcelain \
"${ranges[@]}" "$parent" -- "$file" 2>/dev/null |
If either fails for a file, it contributes no attributed lines. That outcome is
byte-identical to the legitimate result "this file had no lines attributable to
a recent culprit". The failure has no distinct signature.
Consequence: corpus counts are lower bounds
Because the failure mode is silent and one-directional — a swallowed error can
only remove attributed lines, never invent them — every enumeration over the
corpus is a lower bound, not an exact count. Statements of the shape
over the last 500 first-parent commits of main, the canary fires on N commits
are really "the canary reported findings on N commits". Any commit whose
blame errored is silently absent from that N.
This is the single way a corpus enumeration in the calibration prose could be a
false confirmation, so it is worth stating precisely rather than leaving the
reader to assume exhaustiveness.
What is currently covered, and what is not
Measured over the 500 first-parent commits ending at origin/main, with the
pinned flags:
- Enumeration path: clean across all 500. Running the enumeration diff with
stderr captured rather than discarded produced no output on any commit — no
inexact rename detection was skipped, no errors. Max paths presented by any
single commit is 326, against git's default diff.renameLimit of 1000.
- Blame path: clean on the largest calibration file.
cc58cbc53's
2178-line audit-fleet.sh produces 38 hunk ranges and a 76-argument blame
invocation; run with stderr captured it exits 0 and emits nothing. This was
the specific worry about Windows argv limits, and it is not hit.
- Not covered: blame stderr on the 494 non-calibration commits. The sweep
that established the corpus figures ran with stderr discarded, as shipped.
So the current figures are very likely exact — but "likely exact" and "proven
exact" differ, and the difference is the point of this issue.
Why this is not simply "add 2>&1"
The suppression is presumably deliberate: blame and diff are noisy on paths
that legitimately do not resolve, and a canary that prints git noise on every
run gets ignored. The fix probably is not un-suppressing stderr wholesale but
distinguishing expected silence from failed silence — for example, checking
the exit status of the diff and blame separately from their output, and making
a non-zero status a hard error (exit 2, "the canary could not run") rather than
a quiet zero.
That matches the script's own stated contract:
Exit 0 clean, 1 findings, 2 the canary could not run (never a quiet pass).
A blame that fails and yields zero attributions is exactly the "quiet pass"
that contract forbids, reached by a path the contract does not currently guard.
Severity
Low-to-moderate, and bounded. No observed miss; direction is one-way; the
uncovered surface is a subset of one path. Worth fixing because the canary's
entire value proposition is that its failure mode is not silent success, and
this is a silent-success path inside the detector itself.
Not to be fixed in #2847
That PR re-derives calibration prose only. Its prose will be worded so it does
not rest on exhaustiveness it cannot prove.
Surfaced by fresh-context verification during #2847's calibration
re-measurement. Not blocking that PR, and deliberately not fixed in it.
The mechanism
attribute_filediscards git's stderr on both of the commands that produce acount — the per-file diff that enumerates deleted line ranges, and the blame
that attributes them:
If either fails for a file, it contributes no attributed lines. That outcome is
byte-identical to the legitimate result "this file had no lines attributable to
a recent culprit". The failure has no distinct signature.
Consequence: corpus counts are lower bounds
Because the failure mode is silent and one-directional — a swallowed error can
only remove attributed lines, never invent them — every enumeration over the
corpus is a lower bound, not an exact count. Statements of the shape
are really "the canary reported findings on N commits". Any commit whose
blame errored is silently absent from that N.
This is the single way a corpus enumeration in the calibration prose could be a
false confirmation, so it is worth stating precisely rather than leaving the
reader to assume exhaustiveness.
What is currently covered, and what is not
Measured over the 500 first-parent commits ending at
origin/main, with thepinned flags:
stderr captured rather than discarded produced no output on any commit — no
inexact rename detection was skipped, no errors. Max paths presented by anysingle commit is 326, against git's default
diff.renameLimitof 1000.cc58cbc53's2178-line
audit-fleet.shproduces 38 hunk ranges and a 76-argument blameinvocation; run with stderr captured it exits 0 and emits nothing. This was
the specific worry about Windows argv limits, and it is not hit.
that established the corpus figures ran with stderr discarded, as shipped.
So the current figures are very likely exact — but "likely exact" and "proven
exact" differ, and the difference is the point of this issue.
Why this is not simply "add
2>&1"The suppression is presumably deliberate: blame and diff are noisy on paths
that legitimately do not resolve, and a canary that prints git noise on every
run gets ignored. The fix probably is not un-suppressing stderr wholesale but
distinguishing expected silence from failed silence — for example, checking
the exit status of the diff and blame separately from their output, and making
a non-zero status a hard error (exit 2, "the canary could not run") rather than
a quiet zero.
That matches the script's own stated contract:
A blame that fails and yields zero attributions is exactly the "quiet pass"
that contract forbids, reached by a path the contract does not currently guard.
Severity
Low-to-moderate, and bounded. No observed miss; direction is one-way; the
uncovered surface is a subset of one path. Worth fixing because the canary's
entire value proposition is that its failure mode is not silent success, and
this is a silent-success path inside the detector itself.
Not to be fixed in #2847
That PR re-derives calibration prose only. Its prose will be worded so it does
not rest on exhaustiveness it cannot prove.