Skip to content

feat(tri): gates mutate --loud, the direction six days of mutants never took (#2492) - #2493

Merged
gHashTag merged 2 commits into
masterfrom
feat/loud-mutation-operator
Aug 23, 2026
Merged

gHashTag merged 2 commits into
masterfrom
feat/loud-mutation-operator

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

For six days every mutant this command made turned a failure into a pass. Every one asked the same question — can this gate still fail? — so a control made entirely of cases demanding RED satisfied all of them, and the opposite defect went unmeasured across thirteen gates.

--loud

return 0 → return 1: does anything notice a gate that fails on a clean tree?

First run, seven of thirteen gates have survivors — #2492 carries the table and the class.

Verified end to end on the worst one, which is mine

check_gate_preconditions.py — the file that enforces this discipline for six other gates:

OK: 9 precondition(s) across 6 gates fail loudly; 1 known-uncovered
exit 1
control: exit 0 -- nothing noticed

It prints a clean bill of health and reports failure. Exact mirror of this campaign's first finding — check_catalog_integrity printing OK while exiting 0 — and it survived every iteration since, because the tool and the blind spot shared a direction.

Only a bare return 0 counts

A ternary can yield 0 on one arm and a verdict on the other, so forcing the whole line to 1 is the silent operator seen backwards and would be scored against the wrong control. A test asserts the two directions never claim the same line, and that return 1, return 0 if ok else 1 and return fails are all excluded.

Not fixed here

Eight sites across seven gates is separate work. Fixing them in the change that introduced the tool that found them would leave nobody able to tell whether the tool or the fixes were doing the work — and the tool is the part that has to survive being wrong.

Refs #2472, #2492

…er took (#2472)

Every mutant this command made turned a failure into a pass, so it only
ever asked "can this gate still fail?". A control made entirely of cases
demanding RED satisfies all of them, and the opposite defect -- a gate
that fails on a clean tree -- went unmeasured.

--loud flips `return 0` to `return 1`. First run: seven of thirteen gates
have survivors.

Verified end to end on check_gate_preconditions.py, which is mine:

  OK: 9 precondition(s) across 6 gates fail loudly; 1 known-uncovered
  exit 1
  control: exit 0 -- nothing noticed

The gate prints a clean bill of health and reports failure. Exact mirror
of this campaign's first finding, and it survived every iteration since
because every mutation until today pushed the same way.

The survivors are one class: four are the success return of
--update-baseline, right after "baseline written: N entries". Nothing
exercises the ledger-writing path's exit code. check_vector_data is the
exception, and only because a case added three days ago for an unrelated
reason happens to run --update-baseline.

Only a BARE `return 0` counts. A ternary can yield 0 on one arm and a
verdict on the other, so forcing the whole line to 1 is the silent
operator seen backwards; a test asserts the two directions never claim
the same line.

Not fixed here: eight sites across seven gates is separate work. Fixing
them in the change that introduced the tool that found them would leave
nobody able to tell which was doing the work.

Refs #2472
…res one way (#2492)

For six days every mutant this campaign made turned a failure into a
pass, so a control built entirely of cases demanding RED satisfied all of
them. --loud asks the other question and found survivors in seven of
thirteen gates -- the worst being the file that enforces this discipline
for six others, which prints a clean bill of health and exits 1 with
nothing noticing.

The generalisation is not about mutation testing: any instrument that
perturbs a system does so along some axis, and the axes you did not
choose are invisible exactly the way a passing test is.

Plus: note when a clean score is accidental, and land the tool with the
finding rather than with the fixes -- otherwise nobody can tell which was
doing the work.

Refs #2472, #2492
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 03:02:40 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 2
PRs with All Checks Green 4
READY 3
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@gHashTag
gHashTag merged commit 8bcc50a into master Aug 23, 2026
27 of 29 checks passed
@gHashTag
gHashTag deleted the feat/loud-mutation-operator branch August 23, 2026 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant