Skip to content

fix(control): drop the gate's stale bytecode before importing it (#2472) - #2489

Merged
gHashTag merged 4 commits into
masterfrom
fix/wp18-stale-bytecode
Aug 23, 2026
Merged

gHashTag merged 4 commits into
masterfrom
fix/wp18-stale-bytecode

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

wp18_selftest_gate.py does import wp18_conformance_gate as G. Python keys a .pyc on (source mtime in whole seconds, source size), and editing the gate's verdict from else 2 to else 0 preserves the size — so an edit-run-edit loop inside one second hands this control the previous state's bytecode.

Measured here, while testing this very gate

Five hand mutations in quick succession left a .pyc that made the control report five failures with git status clean and the source sha matching HEAD. I spent a round chasing a phantom regression on a tree byte-identical to master.

Worse: every "mutant killed" verdict taken before that discovery was suspect, because the control could have been red for the wrong reason. All of them were re-run after clearing the cache. They hold — but that is a re-measurement, not the original one.

The same defect, one layer over

tri gates mutate clears this between mutants; that fix landed two days ago after a reviewer found it in the neighbouring module. A person at a terminal does not clear it, and neither did this control.

Three lines remove the class: drop the gate's cached bytecode before importing it.

Verified

mutate, run (leaves a mutant .pyc), revert, run immediately  ->  exit 0, tree clean
drift -> else 3   still killed
drift -> else 0   still killed
else 2 (no-op)    not caught -- the control does not cry wolf

Refs #2472

wp18_selftest_gate.py does `import wp18_conformance_gate as G`. Python
keys a .pyc on (source mtime in whole seconds, source size), and editing
the gate's verdict from `else 2` to `else 0` preserves the size, so an
edit-run-edit loop inside one second hands this control the PREVIOUS
state's bytecode.

Measured here, while testing this very gate: five hand mutations in quick
succession left a .pyc that made the control report five failures with
`git status` clean and the source sha matching HEAD. I spent a round
chasing a phantom regression on a tree that was byte-identical to master.

tri gates mutate clears this between mutants -- that fix landed two days
ago after a reviewer found it. A person at a terminal does not, and
neither did this control. Three lines remove the class.

Verified: mutate, run (leaving a mutant .pyc), revert, run immediately ->
exit 0 on a clean tree. And the guard does not weaken the control: both
drift mutations are still killed.

Refs #2472
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 02:15:10 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 2
PRs with All Checks Green 4
READY 3
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

…inal (#2472)

tri gates mutate was taught to clear Python's bytecode cache two days
ago. I then hit the identical defect by hand, on the same gate, while
checking a recommendation: five quick mutations left a stale .pyc and the
control reported five failures on a tree matching HEAD exactly.

Fixing a hazard inside one tool leaves it live wherever a person does the
same thing manually -- and the manual path is the one used while
investigating, which is when a false red costs most.

Plus: every verdict taken before finding the broken instrument is
suspect, not wrong, and the write-up should say the numbers are a
re-measurement. And refuting a recommendation needs more rigour than
accepting one -- three mutations of increasing narrowness plus a no-op as
a control on the control.

Refs #2472
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 02:15:33 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 2
PRs with All Checks Green 4
READY 3
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

A grep for files importing a sibling module found three that matter and
none was guarded. wp18_selftest_gate.py was fixed in the previous commit;
this adds the other two.

  wp18_gate_selfconsistent_selftest.py  imports wp18_conformance_gate
  verify_exhaustive.py                  imports ternary_model

The second is the worst of the three: verify_exhaustive is the gate that
proves four backends agree with a SEPARATELY WRITTEN model, so a stale
model means the comparison is against a version nobody is looking at.

Proven on it, both directions: a same-size edit to ternary_model.py gives
exit 1, and reverting then running immediately gives exit 0. Without the
guard the second run reads the broken model.

Refs #2472
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 02:22:36 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 2
PRs with All Checks Green 4
READY 3
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

After fixing the stale-bytecode trap in the control where I hit it, one
grep for the SHAPE of the defect -- files importing a sibling module --
found two more. The one nobody had tripped over was the worst:
verify_exhaustive imports the separately-written model it compares four
backends against, so a stale .pyc means the fourth opinion is not the one
anyone is reading.

Plus: prove the fix on the instance you did NOT stumble into, or you have
only confirmed the diagnosis you already had. And note that every
mutation a mutation-tester makes is size-preserving by design, which
makes a mutation harness and this cache natural enemies.

Refs #2472
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 02:23:04 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 2
PRs with All Checks Green 4
READY 3
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag merged commit 2e0a0d3 into master Aug 23, 2026
25 of 27 checks passed
@gHashTag
gHashTag deleted the fix/wp18-stale-bytecode branch August 23, 2026 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant