Skip to content

test(analog): sar_comparator_psp mutation census -- 6 of 9 survivors killed, 2 accepted (Closes #12099) - #12115

Merged
gHashTag merged 1 commit into
masterfrom
lane/analog-comparator-psp-mutants-12099
Oct 24, 2026
Merged

gHashTag merged 1 commit into
masterfrom
lane/analog-comparator-psp-mutants-12099

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

Summary

Mutation lane for specs/analog/sar_comparator_psp.t27. Closes #12099 (part of epic #10486; this PR does not close the epic).

Run Mutants Killed Survived Hung Unviable
Before (census on the unmodified spec, 04:58-05:00 +07, exit 0) 41 26 9 5 1
After (gate with --accepted, 05:07-05:09 +07, exit 0, "Gate passed.") 41 32 2 6 1
  • 3 tests appended at the end of the spec, so the accepted line numbers hold: the "no crossing" branch of the time
    readers (-1 when no output reaches half the supply, 131.5 for a jump at the last sample); the absolute value of a
    signed integer; the mirrored unequal pair, whose flip point moves to the other side (+70064 and -70062 at this seal),
    which reaches the first bisection branch with a non-zero result.
  • docs/analog/mutation-accepted-sar_comparator_psp.txt: the 2 remaining survivors (both equivalent, with the
    arithmetic) and the 6 hung mutants (loops that cannot move their bracket).
  • docs/analog/mutation-sar_comparator_psp-2026-10-25.md: the record, with where each number comes from.
  • Seal re-saved with the master-built t27c (tri seals changed).
  • Of the 9 survivors: 6 killed, 1 turned into a hung mutant (hi + lo > 1 in the first bisection), 2 accepted.

Checks run

  • tri test specs/analog/sar_comparator_psp.t27: 16 pass, 0 fail, 0 vacuous.
  • tri mutate spec --file specs/analog/sar_comparator_psp.t27 --accepted docs/analog/mutation-accepted-sar_comparator_psp.txt --jobs 3 --timeout 30: 8 not killed, 8 accepted, 0 accepted lines now killed.
  • tri coverage 'specs/analog/*.t27' --strict: gaps 0 coverage 130/130.

Not claimed

The 2 accepted survivors are classified by arithmetic, not proved by a tool; which test killed which mutant was not
measured; the pinned flips are the model's own words, not ngspice values (the tests compare the mirror, not ngspice).

🤖 Generated with Claude Code

…killed, 2 accepted with arithmetic (Closes #12099)

`tri mutate spec` on specs/analog/sar_comparator_psp.t27 (master-built t27c, jobs 3, timeout 30): 41 mutants,
26 killed, 9 survived, 5 hung, 1 unviable. After this change: 32 killed, 2 survived, 6 hung, 1 unviable;
the gate with --accepted passes (one survivor became hung: the first bisection with a swapped bound).

Three tests appended at the end of the spec (the accepted line numbers hold): the "no crossing" branch of the
time readers, the absolute value of a signed integer, and the mirrored unequal pair, whose flip point moves to
the other side (+70064 and -70062) and so reaches the first bisection branch with a non-zero result.
16 tests pass, 0 vacuous.

docs/analog/mutation-accepted-sar_comparator_psp.txt lists the 2 survivors (both equivalent) and the 6 hung
mutants, each with its arithmetic. The record is docs/analog/mutation-sar_comparator_psp-2026-10-25.md. The
seal is re-saved with the master-built t27c; strict coverage stays gaps 0, 130/130.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-24 22:12:44 UTC

Summary

Status Count
Total Open PRs 46
PRs with Failing Checks 40
PRs with All Checks Green 6
READY 4
FAILING 40
PENDING 0
NO CHECKS YET 0

These columns do not partition: 4 + 40 + 0 + 0 = 44, and there are 46 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=55c0f1b1b32d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@tri-claw

tri-claw Bot commented Oct 24, 2026

Copy link
Copy Markdown

Summary by t27-bees

Checked by the t27 compiler

t27c parse, parse-complete and typecheck on each changed .t27 file at a9b0c88. Nothing was generated, built or run. A pass means the file is well formed, not that it is right.

  • specs/analog/sar_comparator_psp.t27: parses and typechecks.

Read by the model

This PR adds three tests to specs/analog/sar_comparator_psp.t27 to kill 6 of 9 surviving mutants from mutation testing, accepts 2 equivalent mutants, and documents 6 hung mutants. The seal is regenerated with the new test count (13→16).

Worth a look

  • specs/analog/sar_comparator_psp.t27:700 — z[(SC_N * SC_NN + SC_DP) as usize] = vdd; indexes a 1128-element array. The comment claims this is "the last sample," but the constant values aren't visible in the diff. If SC_N * SC_NN + SC_DP ≠ 1127, the test sets the wrong element and the timing assertion after == 131.5 becomes invalid.

  • specs/analog/sar_comparator_psp.t27:701,702 — assert after == 131.5; and assert sp_res(&z) == 131.5; use exact f64 equality. While 131.5 is exactly representable, any change to the crossing interpolation logic (even a correct one) will break these tests.

  • specs/analog/sar_comparator_psp.t27:752,753 — assert_eq(fa, 70064); and assert_eq(fb, 0 - 70062); pin exact model outputs measured once in this lane. The documentation acknowledges these are "the model's own words at this seal, not ngspice values." Any implementation change to sp_flip or the bisection logic will break these tests even if the behavior is still correct.

  • specs/analog/sar_comparator_psp.t27:754 — assert sp_abs_i(fb) < 71580.0; compares against an ngspice reference (71580) that the model deliberately undershoots by ~2.1%. If the model is later improved to match ngspice more closely, this assertion will fail despite the change being an improvement.

  • docs/analog/mutation-accepted-sar_comparator_psp.txt — The accepted mutants are tied to line numbers (179, 217, 141, 180, 200, 208). The file notes: "The line numbers hold only while the spec lines above are unchanged: a moved line leaves its mutant unaccepted and the gate fails." Any edit above line 217 in this spec will require regenerating this file.


Read: 4 files, +195 -11, the whole diff. Head a9b0c88. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

@gHashTag
gHashTag merged commit 2a36da6 into master Oct 24, 2026
37 checks passed
@gHashTag
gHashTag deleted the lane/analog-comparator-psp-mutants-12099 branch October 26, 2026 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

analog: pin the 9 surviving mutants of sar_comparator_psp.t27 (Refs #10486)

1 participant