fix(coverage-report): inventory_guard recovered 0 of 23 rules and called it all clear - #16
Merged
Merged
Conversation
…led it all clear
The third script in coverage-report/scripts/ to state a result nothing
re-derived, and wrong for the same reason as the first. It was written against a
verifier whose obligations lived in `failures.append("literal")` sites. The
registry refactor moved twenty-two of the twenty-three into an explicit `RULES`
table. It did not know that shape, recovered zero rules, found nothing
unrecognisable in the nothing it had found, and printed "no rule is written in a
shape the inventory cannot recover" while exiting 0.
Its whole purpose is to turn a rule the inventory cannot see into a red run. It
had become a script that cannot see any rule and reports that as success.
It reads the registry through mutation_report._sites now, the one place that
shape is parsed. Learning it a second time here is how the two tools would come
to disagree about what a rule looks like, which is the original defect with an
extra step.
It also refuses, with its own exit code, when discovery comes back empty. A
guard may not have "I found nothing" as a success path.
tests/test_inventory_guard_runs.py runs it. It asserts the obligations were
recovered rather than only that the exit status was zero, since a guard that
recovers nothing reports no unrecoverable rule. The refusal is exercised by
injection: pointing the CLI at a synthetic tree makes mutation_report die
importing the verifier, which also exits non-zero, and non-zero for the wrong
reason would leave the test passing over a refusal that never ran. There is a
control asserting the real run does not refuse.
Found by asking which files no test and no workflow reaches. Six python files
under the repository qualify; this was the one whose output is a safety claim.
1033 to 1037 passed, 1 skipped. Reverting the registry knowledge turns the
recovery assertion red. Ruff clean on src/ and tests/; coverage-report/ carries
11 pre-existing findings on both trees.
Signed-off-by: Louielunz <48041247+lywinged@users.noreply.github.com>
|
❔ Contributor Check: UNKNOWN
Automated check by AgenTrust Contributor Check. |
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by asking which files no test and no workflow reaches. This is the one whose output is a safety claim.
What it printed
There are 23 obligations.
mutation_report.pyprintssites: 23 (0 append, 1 inline, 22 registry).This module was written against a verifier whose obligations lived in
failures.append("literal")sites. The registry refactor moved 22 of the 23 into an explicitRULEStable. It did not know that shape, so it recovered nothing, found nothing unrecognisable in the nothing it had found, and reported that as success.Its own docstring says what it is for:
It had become a script that cannot see any rule, and says so as a pass.
Same refactor, same blindness, third script
mutation_report.pycheck_canonicalizer.pyinventory_guard.pycoverage-report/scripts/was referenced by no test and no workflow, which is howREPORT.md's figures drifted for months. Two of the three have since been given a test. This was the third.The change
It reads the registry through
mutation_report._sites, the one place that shape is parsed. Learning it a second time here is how the two tools would come to disagree about what a rule looks like, which is the original defect with an extra step.It refuses when discovery comes back empty, with its own exit code:
After:
The test
tests/test_inventory_guard_runs.py.It asserts the obligations were actually recovered, not only that the exit status was zero. A guard that recovers nothing reports no unrecoverable rule, so exit status alone is exactly the reading that was wrong before.
The refusal is exercised by injection, not through the CLI. Pointing the CLI at a synthetic tree makes
mutation_reportdie importing the verifier, which also exits non-zero, and non-zero for the wrong reason would leave the test passing over a refusal that never ran. There is a control asserting the real run does not refuse, so a guard that refused unconditionally would not pass.Measurements
4b21262inventory_guardrecoverscoverage-report/scripts/with a testruff check src/ tests/Reverting the registry knowledge turns the recovery assertion red.
coverage-report/carries 11 pre-existing ruff findings on both trees, untouched.Correction to an earlier version of this description
It said three vector generators are reached by nothing. That is wrong and I withdraw it.
tests/test_generators_reproduce_fixtures.pyruns all three and compares byte for byte, 17 tests, and it is better than what I was proposing: it discovers generators byrglob("gen_*.py")rather than listing them, and compares the two name sets before comparing any bytes.That discovery is exactly why my scan missed it. The scan matched module stems textually against the test sources, and a generator found by glob never appears there as a literal. A search for "what does nothing reference" cannot see a reference that is computed, which is the same class of blindness as the defect this PR fixes, arriving in the instrument I used to find it.
The genuine remaining orphans are
coverage-report/scripts/pair_mutation.pyandtriple_mutation.py. Nothing imports either;mutation_report.pyonly names them in prose. Separate change.Cut from
main, independent of #12, #13, #14, #15 and #17.