Skip to content

feat(tri): gates mutate --dir; testing a published prediction (#2492) - #2509

Merged
gHashTag merged 1 commit into
masterfrom
feat/mutate-dir
Aug 23, 2026
Merged

gHashTag merged 1 commit into
masterfrom
feat/mutate-dir

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

The twelfth post claims that controls miss degenerate inputs — the empty case, the equal case, the one off the edge — because a control author plants examples of a fault, and degenerate inputs are examples of nothing. It says outright that this predicts a hole in other suites and does not measure one.

So: measure one. tri gates mutate learned --dir, and the campaign's own question was pointed outside its repository for the first time.

The prediction could not be tested, and the reason is a finding

The second repository has 54 Python tools and not one declared negative control — no --self-check, no --selftest, under any spelling. There is nothing to ask "did the control notice?" of. That is the state this repository was in before the campaign, and it is a larger fact than the prediction was.

The only other controlled gate in reach is the one written yesterday from these lessons. Its boundary column reads 0/0 — the file contains no comparison at all — so the prediction is untestable there too, honestly and for a boring reason.

The run found something else

That gate — nine control cases, written with more discipline than anything else this week — had a surviving silent mutant: main()'s no-argument branch. Every case passes a build path, so nothing ran it the way a broken caller would.

It matters because of where the gate is wired. The unattended publisher calls it inside an if !, so a lost argument reaching return 0 reads as a passing check and the cron publishes having compared nothing. The vacuous pass, inside the guard written against the loss it exists to prevent.

Beside it: the usage line was __doc__.splitlines()[-4], which printed the --history line rather than the usage line. A message whose content is an index into the prose above it is wrong the moment the prose is edited — and it already was.

Two rulers widened to make the run possible

  • the file filter matched check_ and not check-, so aimed elsewhere it found nothing and printed an empty table — which reads exactly like a clean suite
  • code() hard-coded tools/, so with --dir it would have run a different file of the same name, or none

Both are the shape this campaign keeps meeting: an instrument that works because everything it has ever seen was arranged one way.

Safety

The gate scripts refuse a path override on purpose — a flag that aims a live gate somewhere harmless is a way to make it pass. This is the auditor, not a gate. The dirty-tree refusal follows the directory, so an interrupted run is still recoverable with git checkout in the right repository. Default behaviour is unchanged: verified identical output on the 13-gate suite, and 13 unit tests green.

Refs #2492

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 07:19:26 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 2
PRs with All Checks Green 4
READY 3
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag merged commit 1dd2718 into master Aug 23, 2026
22 of 24 checks passed
@gHashTag
gHashTag deleted the feat/mutate-dir branch August 23, 2026 07:20
@gHashTag

Copy link
Copy Markdown
Collaborator Author

Correction, one iteration later. The claim in this PR — "54 Python tools and not one declared negative control" — is literally true and its meaning is false. Retracted and measured properly in #2511.

The denominator is 6, not 54: only 7 of the 54 appear in any workflow, and 6 both run in CI and can exit non-zero. And three of those six are well controlled — including one whose test docstring reads "Values, not verdicts. The structural check's flop counter returned zero for every design on earth and stayed green for weeks because the only thing its self-test asserted was pass-or-fail."

I searched for --self-check and concluded absence. A control can be a workflow job with fixtures or a test file, and this tool's own EXTERNAL_CONTROL table exists because five gates here keep their control in a different file. I searched for the mechanism I had built rather than for the property I cared about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant