Repository navigation
fix(gates): the last survivor was a frame, not a limit (#2492) - #2502
Merged
Merged
Conversation
The last surviving mutant in the suite was check_elab_ratchet.py's "no baseline" branch, declared UNCOVERED with this reason: "Reaching it needs `t27c fpga-build --smoke` to SUCCEED and then find no baseline, and a smoke build that succeeds needs the real spec tree -- not something an empty directory can be given." Every clause is true. The conclusion does not follow. A control does not have to use an empty directory: the note reasoned inside the frame of run_on_empty_tree(), the helper this file is built around, and never asked whether a stage could keep the real corpus and empty only the thing under test. REAL_CORPUS does that -- cwd=ROOT so the smoke build finds real specs and succeeds, T27_ELAB_ROOT a planted tree with a built t27c, an empty generated/ and no baseline. The branch is reached in under a second and the real tree is only read (git status clean before and after). UNCOVERED 1 -> 0. Two negative controls for the new runner (VACUOUS and WRONG), plus a real 1.2 MB corpus planted in both end-to-end trees, plus an UNRUN verdict when no corpus is present -- a stage that cannot run proved nothing. Refs #2492
Contributor
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The suite's final surviving mutant was
check_elab_ratchet.py's "no baseline" branch, declaredUNCOVEREDwith a reason that had stood for a week:Every clause is true. The conclusion does not follow. A control does not have to use an empty directory. The note reasoned entirely inside the frame of
run_on_empty_tree()— the helper that file happens to be built around — and never asked whether a stage could keep the real corpus and empty only the thing under test.The stage
REAL_CORPUS:cwd=ROOTsot27c fpga-build --smokeresolves the real specs and succeeds,T27_ELAB_ROOTpointed at a planted tree holding a built compiler, an emptygenerated/and no baseline. The gate gets past the smoke build and then finds nothing to compare against.Reached in under a second. The real tree is only read —
git statusclean before and after, asserted while developing.What came with it
run_on_empty_treeUNRUNverdict when no corpus is present, matching this file's existing doctrine: a row that cannot run proved nothing, and calling it a pass is the vacuous green the file exists to catchUNCOVERED1 → 0, kept as a constant rather than deleted, because the count is asserted in the OK line and a reader who watches it fall learns something a removed line cannot tell themResult, and its boundary
90 mutants across 13 gates — 36 silent, 21 loud, 33 invert — every one killed by the gate's own control.
What that does not say: that the gates are correct, or that their checks are the right checks. It says no mutant in these three families survives, and the families are narrow by construction. A fourth operator would be a fourth question, and the honest prior after this campaign is that a new question finds something — twice now it found the instrument.
Refs #2492