Skip to content

fix(coverage-report): inventory_guard recovered 0 of 23 rules and called it all clear - #16

Merged
lywinged merged 1 commit into
mainfrom
fix/inventory-guard-is-blind-and-unrun
Aug 29, 2026
Merged

lywinged merged 1 commit into
mainfrom
fix/inventory-guard-is-blind-and-unrun

Conversation

@lywinged

@lywinged lywinged commented Aug 27, 2026 •

Copy link
Copy Markdown
Owner

Found by asking which files no test and no workflow reaches. This is the one whose output is a safety claim.

What it printed

$ python coverage-report/scripts/inventory_guard.py .
[inventory] 0 rule(s) in the recognised `append("literal")` shape
[inventory] 1 code(s) emitted through inline literal lists

[guard] no rule is written in a shape the inventory cannot recover
exit=0

There are 23 obligations. mutation_report.py prints sites: 23 (0 append, 1 inline, 22 registry).

This module was written against a verifier whose obligations lived in failures.append("literal") sites. The registry refactor moved 22 of the 23 into an explicit RULES table. It did not know that shape, so it recovered nothing, found nothing unrecognisable in the nothing it had found, and reported that as success.

Its own docstring says what it is for:

Instead of asking what can the inventory recognise? it asks is there anything here the inventory cannot recognise?, and treats any such thing as a failure demanding either the canonical idiom or an explicit declaration. That converts an unrecognised rule from a silent omission into a red run.

It had become a script that cannot see any rule, and says so as a pass.

Same refactor, same blindness, third script

script what it reported fixed in
mutation_report.py "every obligation is held" over 1 site of 23 #5, test in #8
check_canonicalizer.py named two codes that had been renamed away #10
inventory_guard.py "no unrecoverable rule" over 0 rules of 23 here

coverage-report/scripts/ was referenced by no test and no workflow, which is how REPORT.md's figures drifted for months. Two of the three have since been given a test. This was the third.

The change

It reads the registry through mutation_report._sites, the one place that shape is parsed. Learning it a second time here is how the two tools would come to disagree about what a rule looks like, which is the original defect with an extra step.

It refuses when discovery comes back empty, with its own exit code:

A guard whose whole purpose is to turn a silent omission into a red run may not have "I found nothing" as a success path.

After:

[inventory] 0 rule(s) in the recognised `append("literal")` shape
[inventory] 1 code(s) emitted through inline literal lists
[inventory] 22 rule(s) in the `RULES` registry
[inventory] 23 obligation(s) recovered in total

The test

tests/test_inventory_guard_runs.py.

It asserts the obligations were actually recovered, not only that the exit status was zero. A guard that recovers nothing reports no unrecoverable rule, so exit status alone is exactly the reading that was wrong before.

The refusal is exercised by injection, not through the CLI. Pointing the CLI at a synthetic tree makes mutation_report die importing the verifier, which also exits non-zero, and non-zero for the wrong reason would leave the test passing over a refusal that never ran. There is a control asserting the real run does not refuse, so a guard that refused unconditionally would not pass.

Measurements

4b21262 this branch
obligations inventory_guard recovers 0 of 23 23 of 23
scripts in coverage-report/scripts/ with a test 2 of 3 3 of 3
suite 1033 passed, 1 skipped 1037 passed, 1 skipped
ruff check src/ tests/ clean clean

Reverting the registry knowledge turns the recovery assertion red. coverage-report/ carries 11 pre-existing ruff findings on both trees, untouched.

Correction to an earlier version of this description

It said three vector generators are reached by nothing. That is wrong and I withdraw it. tests/test_generators_reproduce_fixtures.py runs all three and compares byte for byte, 17 tests, and it is better than what I was proposing: it discovers generators by rglob("gen_*.py") rather than listing them, and compares the two name sets before comparing any bytes.

That discovery is exactly why my scan missed it. The scan matched module stems textually against the test sources, and a generator found by glob never appears there as a literal. A search for "what does nothing reference" cannot see a reference that is computed, which is the same class of blindness as the defect this PR fixes, arriving in the instrument I used to find it.

The genuine remaining orphans are coverage-report/scripts/pair_mutation.py and triple_mutation.py. Nothing imports either; mutation_report.py only names them in prose. Separate change.

Cut from main, independent of #12, #13, #14, #15 and #17.

…led it all clear

The third script in coverage-report/scripts/ to state a result nothing
re-derived, and wrong for the same reason as the first. It was written against a
verifier whose obligations lived in `failures.append("literal")` sites. The
registry refactor moved twenty-two of the twenty-three into an explicit `RULES`
table. It did not know that shape, recovered zero rules, found nothing
unrecognisable in the nothing it had found, and printed "no rule is written in a
shape the inventory cannot recover" while exiting 0.

Its whole purpose is to turn a rule the inventory cannot see into a red run. It
had become a script that cannot see any rule and reports that as success.

It reads the registry through mutation_report._sites now, the one place that
shape is parsed. Learning it a second time here is how the two tools would come
to disagree about what a rule looks like, which is the original defect with an
extra step.

It also refuses, with its own exit code, when discovery comes back empty. A
guard may not have "I found nothing" as a success path.

tests/test_inventory_guard_runs.py runs it. It asserts the obligations were
recovered rather than only that the exit status was zero, since a guard that
recovers nothing reports no unrecoverable rule. The refusal is exercised by
injection: pointing the CLI at a synthetic tree makes mutation_report die
importing the verifier, which also exits non-zero, and non-zero for the wrong
reason would leave the test passing over a refusal that never ran. There is a
control asserting the real run does not refuse.

Found by asking which files no test and no workflow reaches. Six python files
under the repository qualify; this was the one whose output is a safety claim.

1033 to 1037 passed, 1 skipped. Reverting the registry knowledge turns the
recovery assertion red. Ruff clean on src/ and tests/; coverage-report/ carries
11 pre-existing findings on both trees.

Signed-off-by: Louielunz <48041247+lywinged@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

❔ Contributor Check: UNKNOWN

Check Result
Profile UNKNOWN
Credential LOW
Overall UNKNOWN

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:UNKNOWN Contributor check flagged UNKNOWN risk label Aug 27, 2026
@lywinged
lywinged merged commit 71eecfb into main Aug 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:UNKNOWN Contributor check flagged UNKNOWN risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant