Repository navigation
test(ci): the ledger-writing path nobody ran, and the mirror of the first finding (#2492) - #2494
Merged
Merged
Conversation
…irst finding (#2492) --loud found eight sites where nothing required a gate to be silent. Five are closed here. FOUR were the success return of --update-baseline, right after "baseline written: N entries": check_json_parses, check_seal_coverage, check_specs_generate, check_withdrawn_live. Nothing in the tree ran the ledger-writing path at all -- a run that wrote the ledger and then reported failure was invisible to every control. Each new case asserts the exit AND the effect. Exit alone would pass a run that returned 0 without writing; the marker alone would pass one that wrote and then reported failure, which is the mutation that found this. check_specs_generate already had an --update-baseline case, but it asserts the REFUSAL to grow the ledger; the success path needed a tree where the ledger shrinks. The FIFTH was check_gate_preconditions.py -- the file that enforces this discipline for six other gates -- printing "OK: 9 precondition(s) across 6 gates fail loudly" and exiting 1 with nothing noticing. Its control now runs the whole program on an all-healthy tree and asserts exit 0 and the count, because an exit 0 from a table that silently emptied would satisfy the code alone. Three remain and are a different class: report modes behind a flag CI never passes (--all, --allow-missing-tools, a counts-table branch). Named with their reason rather than folded into the count. Refs #2492
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
Four gates shared an unguarded site: the success return of --update-baseline. Exit alone passes a run that returned 0 without writing; the marker alone passes one that wrote and then reported failure, which is the mutation that found the gap. One gate already had an --update-baseline case asserting the REFUSAL to grow the ledger -- the opposite branch. Having "a case for the command" is not having covered it. Plus: distinguish report modes from verdicts before counting them. Three survivors sit behind flags CI never passes and each file says so in its own words. Eight survivors was accurate; "one class of four, one mirror, three report modes" is what a reader can act on. And: the tool groups, the reading classifies. --loud made the shape visible by putting the same line in four reports; it could not tell that those four were one fix. Refs #2492
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-23 03:14:33 UTC
Summary
Seal Status
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--loudfound eight sites where nothing required a gate to be silent. Five are closed here; the three that remain are a different class and are named with their reason.Four were one shape: the success return of
--update-baselineRight after "baseline written: N entries" — in
check_json_parses,check_seal_coverage,check_specs_generate,check_withdrawn_live.Nothing in the tree ran the ledger-writing path at all. A
--update-baselinethat wrote the ledger and then reported failure was invisible to every control.Each new case asserts the exit and the effect: exit alone would pass a run that returned 0 without writing, and the marker alone would pass one that wrote and then reported failure — which is the mutation that found this.
check_specs_generatealready had an--update-baselinecase, but it asserts the refusal to grow the ledger. The success path needed a tree where the ledger shrinks.The fifth was mine, and the worst
check_gate_preconditions.py— the file that enforces this discipline for six other gates:Its control now runs the whole program on a tree where every gate is healthy and asserts exit 0 and the count, because an exit 0 from a table that silently emptied would satisfy the code alone.
The three that remain are report modes, not verdicts
check_specs_parse.py:302--allcheck_elab_ratchet.py:440--allow-missing-toolscheck_specs_generate.py:375Covering them means asserting that report modes exit 0. Worth doing, not the same finding, so left with its reason rather than folded into the count.
Method note
The four
--update-baselinefixes came from one shape read once, not four investigations.--loudgrouped them by putting the same line in four reports; the reading turned "eight survivors" into "one class and three exceptions".Refs #2492