Skip to content

test(analog): pin 4 of 12 survivors of sar_kick with 3 new tests, accept the 8 left - #12110

Merged
github-actions[bot] merged 1 commit into
masterfrom
lane/analog-kick-mutants-12097
Oct 24, 2026
Merged

github-actions[bot] merged 1 commit into
masterfrom
lane/analog-kick-mutants-12097

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

Closes #12097. Part of epic #10486 (see the epic; not closed here).

What changed

  • specs/analog/sar_kick.t27: 3 tests appended after the last original test (66 lines inserted, nothing above the old end moved, so the accepted line numbers hold).
  • docs/analog/mutation-accepted-sar_kick.txt (new): the 8 equivalent survivors, each with the arithmetic, and the 5 hung drop-step mutants.
  • docs/analog/mutation-sar_kick-2026-10-25.md (new): the record.
  • .trinity/seals/analog_AnalogSarKick.json: re-saved with the master-built t27c (tri seals changed).

Numbers

Run Mutants Killed Survived Hung Unviable
Before (census in this lane on the unmodified spec, 04:04-04:07 +07, exit 0) 94 77 12 5 0
After (gate on this tree, 04:19-04:21 +07, exit 0) 94 81 8 5 0

Gate: 13 not killed (8 survived, 5 hung), 13 accepted, 0 accepted lines now killed ("Gate passed."). Commands and the table of the 12 survivors are in the record.

  • tri test specs/analog/sar_kick.t27: 13 pass, 0 fail, 0 vacuous.
  • tri coverage 'specs/analog/*.t27' --strict: gaps 0 coverage 130/130.

Not claimed

  • Which test killed which of the 4 killed mutants was not measured, only the totals.
  • The 8 accepted survivors are classified by reading the arithmetic, backed by strict assertions on the pinned data; not proved equivalent by a tool.
  • The second added test reports 6 runtime asserts against 10 written; the cause was not established.
  • The model is not compared with ngspice here.

Siblings still open: #12098 (sar_cdac), #12099 (sar_comparator_psp).

🤖 Generated with Claude Code

…ept the 8 left (Closes #12097)

Census on the unmodified spec (master-built t27c, lane base 2e52102):
94 mutants, 77 killed, 12 survived, 5 hung, 0 unviable. After this change:
81 killed, 8 survived, 5 hung, 0 unviable; the gate passes with 13 accepted
and 0 accepted lines now killed.

- 3 tests appended after the last original test, so no line above moved:
  an index without records runs the whole scan grid; a decision exactly on
  the threshold follows the 0-above / 1-at-or-below rule (pinned records,
  plus and minus 1 nV neighbours); the grid end and the series-resistor
  bracket leave no tie or extra point.
- docs/analog/mutation-accepted-sar_kick.txt: 8 equivalent survivors with
  their arithmetic, 5 hung drop-step mutants.
- docs/analog/mutation-sar_kick-2026-10-25.md: the record, with what is not
  claimed.
- Seal re-saved with the master-built t27c; tri test 13/13, 0 vacuous;
  tri coverage strict 130/130.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-24 21:24:29 UTC

Summary

Status Count
Total Open PRs 46
PRs with Failing Checks 40
PRs with All Checks Green 6
READY 3
FAILING 40
PENDING 0
NO CHECKS YET 0

These columns do not partition: 3 + 40 + 0 + 0 = 43, and there are 46 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=55c0f1b1b32d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@tri-claw

tri-claw Bot commented Oct 24, 2026

Copy link
Copy Markdown

Summary by t27-bees

Checked by the t27 compiler

t27c parse, parse-complete and typecheck on each changed .t27 file at f74f072. Nothing was generated, built or run. A pass means the file is well formed, not that it is right.

  • specs/analog/sar_kick.t27: parses and typechecks.

Read by the model

This PR adds 3 tests to sar_kick.t27 to kill 4 of 12 mutation survivors, then accepts the remaining 8 as equivalent and 5 as hung (drop-step loops). The mutation gate passes with 81 killed, 8 survived, 5 hung.

Worth a look

  • specs/analog/sar_kick.t27 (test 2): The test "a_decision_exactly_on_the_threshold..." has 10 written assert/assert_eq statements but only 6 are counted at runtime; the PR notes the 4 comparing constant array elements are not in the count and the cause was not established. If those 4 are optimized away or never executed, the test's coverage of the threshold tie rule is weaker than it appears.

  • docs/analog/mutation-accepted-sar_kick.txt:10-11 (line 170 equivalence): The equivalence argument for the two flip-cmp mutants at line 170 depends on the latch offset (0.153 mV) being strictly between the bracket voltages (9.4155, 4.086, -0.8945 mV). This is only backed by the new test 3's assertions on the pinned data; if the model or data changes, the acceptance becomes invalid.

  • docs/analog/mutation-accepted-sar_kick.txt:2-3 (line-number fragility): The file explicitly states line numbers hold only while lines above are unchanged; any edit above line 306 invalidates the acceptance file and fails the gate. This couples spec maintenance to the mutation baseline.

  • .trinity/seals/analog_AnalogSarKick.json: All four gen_hash_* values and spec_hash changed with the seal re-save (new built_by hash 19a89f1e3). The generated C, Rust, Verilog, and Zig outputs differ from the previous seal—verify the regenerated code is intended.

  • specs/analog/sar_kick.t27:577-579 (test 3 assertions): The test asserts rs_off(0) > lat and rs_off(1) < lat to prove the bracket is at i == 1, but rs_off is not shown in the diff. If rs_off reads a different table or computes differently than implied, the equivalence justifications for lines 168, 170, and 177 could be wrong.


Read: 4 files, +192 -9, the whole diff. Head f74f072. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

analog: pin the 12 surviving mutants of sar_kick.t27 (Refs #10486)

1 participant