Skip to content

test(ci): the agreeing direction, so a louder gate cannot pass (#2472) - #2491

Merged
gHashTag merged 2 commits into
masterfrom
test/agreement-silence-case
Aug 23, 2026
Merged

gHashTag merged 2 commits into
masterfrom
test/agreement-silence-case

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

Yesterday's §28 was: a control made entirely of cases demanding a failure is blind to every mutation that makes a gate louder. Applied across all thirteen controls, one gate answered.

check_duplicate_agreement had two controls and neither required silence

Both assert returncode == 1. A gate rewritten to report a split on a tree where every copy agrees satisfies both.

Measured before the fix — two such mutations of the one-group branch:

                          gate   --self-check   --self-check-drop
constant false             1          0                 1
comparison against zero    1          0                 1

Caught only by --self-check-drop, which exists to exercise a different branch entirely. Coverage by a sibling's accident is not coverage, and nothing in a green report distinguishes the two.

The new case plants agreement

Same fixture for both copies, so they genuinely agree; it asserts one behaviour and exit 0.

constant false             --self-check 1   killed
comparison against zero    --self-check 1   killed
no-op                      --self-check 0   silent

My own comment broke my own harness

The first version of that comment quoted the branch verbatim. str.replace(needle, mut, 1) takes the first occurrence, prose comes before code, so the mutation hit the comment — the gate ran unmutated and the case reported itself blind. I nearly wrote up "the silence case does not work".

It describes the branch now instead of quoting it, and says why. Settled by mutating by line number.

What the sweep did not do

The per-file count of silence-shaped strings is a proxy. One file was read and confirmed; the other twelve were scored by a regex and left there. Twelve unexamined counts is a different claim from twelve clean results, and the note says which this is.

Refs #2472

check_duplicate_agreement had two controls and neither required silence:
both assert returncode == 1. A gate rewritten to report a split on a tree
where every copy agrees satisfies both.

Measured before the fix: two such mutations were caught ONLY by
--self-check-drop, which exercises a different branch entirely. The
primary control passed both. Coverage by a sibling's accident is not
coverage.

The new case plants agreement -- same fixture for both copies -- and
asserts one behaviour, exit 0. Both mutations are now killed by the
primary control and a no-op mutation stays silent.

My own comment broke my own harness on the way: it quoted the branch
verbatim, a text-replacing mutation hit the COMMENT instead of the code,
the gate ran unmutated, and the case reported itself blind. It describes
the branch now and says why.

Refs #2472
…ate (#2472)

A comment explaining a new silence case quoted its branch verbatim. A
text-replacing mutation then hit the comment instead of the code, the
gate ran unmutated, and the case reported itself blind. str.replace takes
the first occurrence and prose comes before code.

Describe the branch instead, or mutate by line number.

Plus: applying section 28 across all thirteen controls found one gate
whose two controls both demand red, so a gate that reports a split on
agreeing copies satisfies both. It was caught only by a sibling control
written for something else -- coverage by accident, which a green report
does not distinguish from the real thing.

And: the sweep was a regex proxy over twelve files and a reading of one.
The write-up says so, because a sweep reported without that sentence
reads as twelve clean results.

Refs #2472
@github-actions

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-08-23 02:54:04 UTC

Summary

Status Count
Total Open PRs 6
PRs with Failing Checks 2
PRs with All Checks Green 4
READY 3
FAILING 2
PENDING 0

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=e905ba8eeb53 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag merged commit 704942b into master Aug 23, 2026
20 of 22 checks passed
@gHashTag
gHashTag deleted the test/agreement-silence-case branch August 23, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant