Skip to content

test(analog): sar_cdac mutation census -- 8 of 13 survivors killed, 5 accepted (Closes #12098) - #12114

Merged
gHashTag merged 1 commit into
masterfrom
lane/analog-cdac-mutants-12098
Oct 24, 2026
Merged

gHashTag merged 1 commit into
masterfrom
lane/analog-cdac-mutants-12098

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

Summary

Mutation lane for specs/analog/sar_cdac.t27. Closes #12098 (part of epic #10486; this PR does not close the epic).

Run Mutants Killed Survived Hung Unviable
Before (census on the unmodified spec, 04:28-04:32 +07, exit 0) 148 122 13 12 1
After (gate with --accepted, 04:51-04:55 +07, exit 0, "Gate passed.") 148 130 5 12 1
  • 6 tests appended at the end of the spec, so the accepted line numbers hold: the closed form for a bridge of two units
    and a dummy unit; the first-code-wins rule of worst_gain, worst_inl, worst_arith and worst_ng_vs_f64 on exact
    ties (built where TOL is absorbed by rounding), a zero grid, and the off-grid edge of ties_clear.
  • docs/analog/mutation-accepted-sar_cdac.txt: the 5 remaining survivors (2 equivalent, 3 not constructed) and the 12
    hung mutants, each with its arithmetic.
  • docs/analog/mutation-sar_cdac-2026-10-25.md: the record, with where each number comes from.
  • Seal re-saved with the master-built t27c (tri seals changed).

Checks run

  • tri test specs/analog/sar_cdac.t27: 19 pass, 0 fail, 0 vacuous.
  • tri mutate spec --file specs/analog/sar_cdac.t27 --accepted docs/analog/mutation-accepted-sar_cdac.txt --jobs 3 --timeout 30: 17 not killed, 17 accepted, 0 accepted lines now killed.
  • tri coverage 'specs/analog/*.t27' --strict: gaps 0 coverage 130/130.

Not claimed

The 5 accepted survivors are classified by arithmetic, not proved by a tool; which test killed which mutant was not
measured; the model's tie rules are not compared with ngspice. sar_comparator_psp stays with #12099.

🤖 Generated with Claude Code

… accepted with arithmetic (Closes #12098)

`tri mutate spec` on specs/analog/sar_cdac.t27 (master-built t27c, jobs 3, timeout 30): 148 mutants,
122 killed, 13 survived, 12 hung, 1 unviable. After this change: 130 killed, 5 survived, 12 hung,
1 unviable; the gate with --accepted passes.

Six tests appended at the end of the spec (the accepted line numbers hold): the closed form for a bridge of
two units and a dummy unit, and the first-code-wins rule of the five worst-code searches on exact ties built
where TOL is absorbed by rounding, plus a zero grid and the tie gate's off-grid edge. 19 tests pass, 0 vacuous.

docs/analog/mutation-accepted-sar_cdac.txt lists the 5 survivors (2 equivalent, 3 not constructed) and the
12 hung mutants, each with its arithmetic. The record is docs/analog/mutation-sar_cdac-2026-10-25.md. The seal
is re-saved with the master-built t27c; strict coverage stays gaps 0, 130/130.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-24 21:58:11 UTC

Summary

Status Count
Total Open PRs 45
PRs with Failing Checks 40
PRs with All Checks Green 5
READY 3
FAILING 40
PENDING 0
NO CHECKS YET 0

These columns do not partition: 3 + 40 + 0 + 0 = 43, and there are 45 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=55c0f1b1b32d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@tri-claw

tri-claw Bot commented Oct 24, 2026

Copy link
Copy Markdown

Summary by t27-bees

Checked by the t27 compiler

t27c parse, parse-complete and typecheck on each changed .t27 file at 6c8a834. Nothing was generated, built or run. A pass means the file is well formed, not that it is right.

  • specs/analog/sar_cdac.t27: parses and typechecks.

Read by the model

Worth a look

  • specs/analog/sar_cdac.t27:1070-1090 — The closed-form bridge test uses CL = 31 + 1 dummy = 32 but UNITS and SIDE are imported constants; if those arrays ever change length or values, the hand-computed ratios (33/1118, 528/1118, etc.) become invalid and the test will fail silently because it only checks against fixed literals.

  • specs/analog/sar_cdac.t27:1120-1130 — The worst-gain tie test constructs kb with every element 1073741824.0 (2^30) and asserts gain_lsb(&kb, 0) == gain_lsb(&kb, 1023) in magnitude; if gain_lsb ever changes to use a different reference or scaling, the tie condition breaks and the mutant at line 594 (>= vs >) would survive unaccepted.

  • specs/analog/sar_cdac.t27:1140-1155 — The worst-INL tie test sets only kb[17] and kb[19] to large opposite values; it assumes codes 511 and 512 are the symmetric pair around mid-scale. If UNITS or SIDE bit ordering changes, those codes may no longer be the tie pair and the test would not kill the >= mutant at line 581.

  • specs/analog/sar_cdac.t27:1180-1190 — The worst-arithmetic zero-grid test expects worst_arith to return code 0 with error 0 for both z1 and z0; this relies on the initial best = 0 and best_at = 0 in the function. If the initialization changes (e.g., best = -inf), the e > best - TOL mutant at line 609 would no longer be killed by this test.

  • docs/analog/mutation-accepted-sar_cdac.txt:17-18 — Two "not constructed" survivors at line 633 (>= and - TOL) depend on the claim that "no stored-word set builds such a pair against the fixed ngspice table." The ngspice table NG_HALF is imported; if that table is ever updated with higher-precision values, these mutants could become killable and the accepted file would then block the gate (accepted line now killed).


Read: 4 files, +278 -9, the whole diff. Head 6c8a834. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

analog: pin the 13 surviving mutants of sar_cdac.t27 (Refs #10486)

1 participant