Follow-up to #2831 / #2832, caught by fresh-context verification of that PR
after it had already merged.
That PR corrected the incident corpus to record that #2633's squash dropped
work from two siblings, adding a fourth true finding: 301 lines traced to
eda5ae5ed (#2642). It did not carry that fact through to two calibration
sentences that depend on which finding is the smallest, and both are now false
on main.
The two false sentences
scripts/check-silent-revert.sh:
State the uncomfortable part plainly: 340 is the largest false positive and
346 is the smallest true one. NO THRESHOLD SEPARATES THEM. Picking a
number in that 2% gap would be overfitting to this corpus...
scripts/silent-revert-acknowledged.txt:
At 340 lines this is the largest measured false positive, six lines below
the smallest real incident (346), which is why no threshold can separate
the two shapes...
Measured
Every commit in the calibration set, scanned at the shipped settings with the
acknowledgment file bypassed so the cleared fires are visible:
| commit |
PR |
findings (lines) |
class |
cc58cbc53 |
#2633 |
853, 301 |
incident |
9239f1541 |
#2641 |
451 |
incident |
f603880da |
#2639 |
346 |
incident |
6f0a31109 |
#2640 |
390 |
verified-legitimate |
91e77fc16 |
#2135 |
340 |
verified-legitimate |
The smallest true finding is 301, not 346. Both verified-legitimate fires —
340 and 390 — score above it. So the relationship is not a narrow gap that
a threshold might split; it is an inversion, and the two ranges overlap
outright. "Six lines below the smallest real incident" is off by sign as well
as magnitude: 340 sits 39 lines above it.
This does not change the threshold
The conclusion the sentences were arguing for is unaffected, and in fact
strengthened — if no threshold separated the two shapes when the gap was six
lines, it certainly does not now that the ranges overlap. THRESHOLD stays at
200, below all five figures, and the disposition path (the
Intentional-removal: trailer and silent-revert-acknowledged.txt) remains
what makes the canary livable rather than the number.
One detail worth recording while correcting this: 200 and 300 still fire on the
identical five commits, but at 300 the 301-line finding survives by a single
line. That is a sharper argument against tuning the threshold than the
now-false "2% gap" was.
Fix
Prose only. Rewrite both sentences to state the measured overlap, and leave
check-silent-revert.sh's executable body byte-identical — no threshold,
window, or recorded expectation moves.
Related
Follow-up to #2831 / #2832, caught by fresh-context verification of that PR
after it had already merged.
That PR corrected the incident corpus to record that #2633's squash dropped
work from two siblings, adding a fourth true finding: 301 lines traced to
eda5ae5ed(#2642). It did not carry that fact through to two calibrationsentences that depend on which finding is the smallest, and both are now false
on
main.The two false sentences
scripts/check-silent-revert.sh:scripts/silent-revert-acknowledged.txt:Measured
Every commit in the calibration set, scanned at the shipped settings with the
acknowledgment file bypassed so the cleared fires are visible:
cc58cbc539239f1541f603880da6f0a3110991e77fc16The smallest true finding is 301, not 346. Both verified-legitimate fires —
340 and 390 — score above it. So the relationship is not a narrow gap that
a threshold might split; it is an inversion, and the two ranges overlap
outright. "Six lines below the smallest real incident" is off by sign as well
as magnitude: 340 sits 39 lines above it.
This does not change the threshold
The conclusion the sentences were arguing for is unaffected, and in fact
strengthened — if no threshold separated the two shapes when the gap was six
lines, it certainly does not now that the ranges overlap. THRESHOLD stays at
200, below all five figures, and the disposition path (the
Intentional-removal:trailer andsilent-revert-acknowledged.txt) remainswhat makes the canary livable rather than the number.
One detail worth recording while correcting this: 200 and 300 still fire on the
identical five commits, but at 300 the 301-line finding survives by a single
line. That is a sharper argument against tuning the threshold than the
now-false "2% gap" was.
Fix
Prose only. Rewrite both sentences to state the measured overlap, and leave
check-silent-revert.sh's executable body byte-identical — no threshold,window, or recorded expectation moves.
Related
301-line finding.
sentences.