Skip to content

fix(gates): a control for verify_trainer_c; the selector was right by accident (#2492) - #2524

Merged
gHashTag merged 2 commits into
masterfrom
fix/trainer-c-control
Aug 23, 2026
Merged

gHashTag merged 2 commits into
masterfrom
fix/trainer-c-control

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

Two measurements this iteration, and the honest one is the boring one.

The assert blind spot costs nothing here

is_gate_by_property recognises return, sys.exit and raise SystemExit. A tool that fails only through an uncaught AssertionError is invisible to it, and I expected that to be costing coverage.

Measured across every CI-invoked tool in tools/: zero fail only that way — every one has an explicit or ternary exit as well. The hole is real and empty. Saying so is worth more than the fix I was about to write.

But the selector has a false positive, and it fired

gft_backprop_microcode.py was classified as a gate on the strength of

return 0 if ys[0] >= ys[1] else 1

which is a class label from a classifier, not a verdict. verdict_literals was built to recognise return statements that carry verdicts; using it to answer "can this file exit non-zero" conflates a returned value with an exit code.

The file is a gate anyway — sixteen asserts — so the classification is right by accident, which is the least useful way to be right. Recorded rather than patched: narrowing the predicate risks the false negative, and the one file affected is already correctly in the list.

A control for the last of the four verifiers

verify_trainer_c.py — the one whose skip() every other copied — now scores 1/1 silent, 1/1 loud, 3/3 invert, with a planted divergence in its C arm and the clean direction asserted beside it.

The plant scopes its edit to the text after def run_c(, because the same plant in fuzz_trainer.py hit run_model three functions earlier and a case then passed on a divergence planted where it was never meant to be.

Not covered, and named: the per-architecture "C produced N of M steps" branch, which needs a C arm that truncates rather than one that disagrees.

Where the count stands

Gates with no control in any form: 1 — and it is the file the selector was right about by accident.

The rule

A predicate reused outside the question it was written for will be right often enough to look correct. verdict_literals answers "is this return a verdict?"; it does not answer "can this program fail?", and the difference only shows up on a classifier that returns 1 for a class.

Refs #2492

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 10:23:05 UTC

Summary

Status Count
Total Open PRs 7
PRs with Failing Checks 3
PRs with All Checks Green 4
READY 3
FAILING 3
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag merged commit 9ad8eff into master Aug 23, 2026
24 of 26 checks passed
@gHashTag
gHashTag deleted the fix/trainer-c-control branch August 23, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant