Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .trinity/seals/analog_AnalogSarKick.json
Original file line number Diff line number Diff line change
@@ -1,29 +1,29 @@
{
"built_by": "t27c-bootstrap@0.5.2+42a4860c4",
"built_by": "t27c-bootstrap@0.5.2+19a89f1e3",
"closure": [
{
"spec": "specs/analog/linear.t27",
"spec_hash": "sha256:3ddef7be183aba583ed285334ee389b48a65b41335e98914514235db7c4e5d62"
}
],
"config": "gen=c,rust,verilog,zig;tests=zig",
"gen_hash_c": "sha256:77daac412fa8244abc763daf5b7b7fa39d95b2f71547dd4670078b0b83a0bfec",
"gen_hash_rust": "sha256:a986bdfd81d3c5ed719ec1d54de5db8b60b82b165761b5ee4781b989792e8fca",
"gen_hash_verilog": "sha256:d10e1ae69bac9c40b293e22a482359faa8f908efacc8a685ef3e9fcdb09c74c2",
"gen_hash_zig": "sha256:e63b5a59b13bc07961afb614f789de6f28afd716149333decbf35b8129d9de3a",
"gen_hash_c": "sha256:7af4d66ccee10e7678b697900958ab787e8c49b47c99201ffb099d3ab450f061",
"gen_hash_rust": "sha256:cdd887fa849c13bd4548d696c89299c73faf09eb67149d6c9427a36542821a8f",
"gen_hash_verilog": "sha256:39effc8225c260e7664e7b035e292aee25b44fb732268544e5e6c396afa90163",
"gen_hash_zig": "sha256:3540385c12e14288aa01c0f815472d897554b8a5ad633a8f69e95875f5e82c3d",
"module": "AnalogSarKick",
"ring": 12,
"seal_schema": 2,
"sealed_at": "2026-10-20T03:12:04Z",
"sealed_at": "2026-10-24T21:19:04Z",
"sealed_by": "t27c-bootstrap@0.5.2",
"spec_hash": "sha256:b38a9a0bd22837f7ca1d5609756c522cd791f7b0f11115b7863e04cd4e15bf10",
"spec_hash": "sha256:2e9f43fd473f52b67a7d36cab8e118b47e0d93e306e74bdc8511a2d53b037aef",
"spec_path": "specs/analog/sar_kick.t27",
"tests": {
"failed": 0,
"failing": [],
"forced": false,
"passed": 10,
"total": 10,
"passed": 13,
"total": 13,
"vacuous": 0
},
"toolchain": {
Expand Down
22 changes: 22 additions & 0 deletions docs/analog/mutation-accepted-sar_kick.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Mutants of specs/analog/sar_kick.t27 that no test can or must kill (issue 12097).
# Format: path:line [kind] and a note, as `tri mutate spec --accepted` reads it (rules: specs/tri/mutate/survivors.t27).
# Measured 2026-10-25 with the master-built t27c (b9d83f9ef2) on the unmodified spec (lane base 2e52102792): 94 mutants, 77 killed, 12 survived, 5 hung, 0 unviable.
# The tests added in 12097 kill 4 of the survivors (81 killed, 8 survived, 5 hung, 0 unviable); the 8 and the 5 are all listed below.
# The line numbers hold only while the spec lines above are unchanged: a moved line leaves its mutant unaccepted and the gate fails.
# New tests go at the end of the spec for that reason.
#
# Equivalent: the mutated body gives the same result on every input of this spec.
specs/analog/sar_kick.t27:168 [flip-cmp] equivalent: rs_zero_ohm reads no input and returns at i == 1, because the offset crosses the latch's own between the 1300 and 2592 ohm rows (strict on both sides, asserted in the last test), so i == NRS never runs
specs/analog/sar_kick.t27:170 [flip-cmp] equivalent: the tie oa == lat or ob == lat is not met: oa is 9.4155 mV at i == 0 and 4.086 mV at i == 1, ob is 4.086 mV and -0.8945 mV, the latch's offset is 0.153 mV (asserted strictly in the last test)
specs/analog/sar_kick.t27:177 [swap-arith] equivalent: the line is the "no row brackets" value and is never reached, the scan returns at i == 1 on the pinned data
specs/analog/sar_kick.t27:275 [flip-cmp] equivalent: the extra point j == SCAN_N is -4.0 mV, where every index with records has no fewer errors than its best (asserted for 7 .. 10 in the last test) and every index without records has 0, so `e < best` is false there
specs/analog/sar_kick.t27:277 [flip-cmp] equivalent: at e == best the assignment writes the value best already holds
specs/analog/sar_kick.t27:302 [flip-cmp] equivalent: the extra point j == SCAN_N is -4.0 mV, where the pooled count of indices 9 and 10 is no lower than the pooled best (asserted in the last test), so `e < best` is false there
specs/analog/sar_kick.t27:305 [flip-cmp] equivalent: at e == best the assignment writes the value best already holds
#
# Hung: a mutant that makes a loop endless, so no test can go red; only the clock ends the mutant.
specs/analog/sar_kick.t27:175 [drop-step] hung by construction: the row loop of rs_zero_ohm never ends
specs/analog/sar_kick.t27:258 [drop-step] hung by construction: the record loop of late_errors never ends
specs/analog/sar_kick.t27:278 [drop-step] hung by construction: the first scan loop of late_thr never ends
specs/analog/sar_kick.t27:291 [drop-step] hung by construction: the second scan loop of late_thr never ends
specs/analog/sar_kick.t27:306 [drop-step] hung by construction: the scan loop of pooled_min_errors never ends
95 changes: 95 additions & 0 deletions docs/analog/mutation-sar_kick-2026-10-25.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Mutation census of specs/analog/sar_kick.t27, 2026-10-25

Issue #12097, part of epic #10486 (weak spots of the analog v0.1 release). The tool, `tri mutate spec`, never edits the
spec, and the checkout stayed clean during the runs. Where each number comes from:

- **Before** (94 mutants, 77 killed, 12 survived, 5 hung, 0 unviable) is a census run in this lane on the unmodified
spec: lane base master `2e52102792`, master-built `t27c` (build `b9d83f9ef2`, seals stamp
`built_by: t27c-bootstrap@0.5.2+19a89f1e3`), `--jobs 3 --timeout 30`, 04:04-04:07 +07, exit 0, load 9.96, 12 GiB
free. The 12 survivor lines are read off that output.
- **After** is the gate run on the tree of this change, same binary and flags plus `--accepted`: 04:19-04:21 +07,
exit 0, "Gate passed.". The tests appended to the spec add no mutants (the tool mutates the functions, not the
tests), so the mutant count is 94 in both runs.

## What was run

Before, the census:

```
tri mutate spec --file specs/analog/sar_kick.t27 --t27c <master-built t27c> --jobs 3 --timeout 30
```

After, the gate (the first seconds of that window are `tri seals changed`):

```
tri mutate spec --file specs/analog/sar_kick.t27 --t27c <master-built t27c> --jobs 3 --timeout 30 \
--accepted docs/analog/mutation-accepted-sar_kick.txt
```

## Census

| Run | Mutants | Killed | Survived | Hung | Unviable |
| --- | --- | --- | --- | --- | --- |
| Before (unmodified spec, this lane) | 94 | 77 | 12 | 5 | 0 |
| After (this change) | 94 | 81 | 8 | 5 | 0 |

By kind (both runs): flip-cmp 20, swap-logic 5, drop-step 9, swap-arith 48, ret-default 12.

The gate line: 13 not killed (8 survived, 5 hung), 13 accepted by the file, 0 accepted lines now killed.

## The 12 survivors before, and what became of each

| Region (lines) | Survivors | Verdict | Outcome |
| --- | --- | --- | --- |
| `rs_zero_ohm`, the row loop bound (168) | 1 | equivalent: the function returns at row 1 (the offset crosses the latch's own between the 1300 and 2592 ohm rows), so the extra row `i == NRS` is never read | 1 accepted |
| `rs_zero_ohm`, the bracket test `oa > lat && ob <= lat` (170; two mutants) | 2 | equivalent: the tie `oa == lat` or `ob == lat` is not met (9.4155 and 4.086 mV at row 0 and 1, then 4.086 and -0.8945 mV, against 0.153 mV) | 2 accepted |
| `rs_zero_ohm`, the "no row brackets" value (177) | 1 | equivalent: the line is never reached on the pinned data | 1 accepted |
| `late_errors`, the two error tests (255, 256) | 2 | test gap: no test put a threshold exactly on a pinned decision, so the rule "a 0 above, or a 1 at or below, the threshold is an error" was not pinned on its tie | 2 killed |
| `late_thr` and `pooled_min_errors`, the scan bound `j < SCAN_N` (275, 302) | 2 | equivalent: the extra point `j == SCAN_N` is -4.0 mV, where no index has fewer errors than its best (asserted) | 2 accepted |
| the same functions, the improvement test `e < best` (277, 305) | 2 | equivalent: at `e == best` the assignment writes the value `best` already holds | 2 accepted |
| `late_thr`, the bound of the run-finding loop `j < SCAN_N` (284) and the half width `SCAN_STEP * ((last - first) / 2)` (294; the mutant divides instead of multiplying) | 2 | test gap: every index a test read had records, so the fewest-errors run was never the whole grid, and neither the last grid point nor a wide half width was read | 2 killed |

Totals: 12 survivors, of which 4 killed and 8 accepted as equivalent (the rows add up: 1 + 2 + 1 + 2 + 2 + 2 + 2 = 12;
killed 2 + 2 = 4; accepted 1 + 2 + 1 + 2 + 2 = 8). The After run shows exactly the 8 accepted lines alive, which is
the check that the 4 others died. The per-mutant attribution to the test that killed it was not measured, only the
totals before and after.

The 5 hung mutants are drop-step mutants that make a loop endless (the row loop of `rs_zero_ohm`, the record loop of
`late_errors`, the two scan loops of `late_thr`, the scan loop of `pooled_min_errors`), so no test can go red and only
the clock ends the mutant. They are named in the accepted file with that reason. There are no unviable mutants.

## The 3 tests added (appended after the last original test, so the accepted line numbers did not move)

- a decision index without records has no errors and its threshold run is the whole scan grid (10 runtime asserts);
- a decision exactly on the threshold follows the rule: a 0 above, or a 1 at or below, is an error. Two pinned records
are used as the threshold, with neighbours at plus and minus 1 nV, so the delta is +1 or 0 by the rule and cannot be
met by accident (6 runtime asserts reported; the code has 10, the 4 that compare constant array elements are not in
the count, and why was not established);
- the scan grid end and the series resistor bracket leave no tie or extra point to decide anything: they back the
accepted notes of lines 168, 170, 177, 275 and 302 with strict assertions (11 runtime asserts).

`tri test specs/analog/sar_kick.t27`: 13 tests pass, 0 fail, 0 vacuous passes. The seal is re-saved with the
master-built `t27c` (`tri seals changed`: seal-save, seal-recheck, importers and importer-seal pass; its `seal` step
exits 1 on the stale seal before the save, as the tool expects).
`tri coverage 'specs/analog/*.t27' --strict`: 26 specs, `gaps 0 coverage 130/130`.

## Reading the accepted file

`docs/analog/mutation-accepted-sar_kick.txt` follows `specs/tri/mutate/survivors.t27`: one `path:line [kind]` per
mutant with a free-text note after it; one entry covers all mutants of that kind on the line. With `--accepted` the tool
exits 2 on an unaccepted survivor or hung mutant, and also on an accepted line whose mutant is now killed, so the file
cannot go stale in either direction. A moved line leaves its mutant unaccepted: edit the spec above line 306 and
the file must be regenerated. New tests go at the end of the spec for that reason.

## Not claimed

- The 8 accepted survivors are classified by reading the arithmetic (each line carries it in the accepted file), not
proved equivalent by a tool; the tie-free brackets that back them are asserted in the last test on the pinned data
only.
- The 5 hung mutants are classified by construction (a loop step dropped), not by a run that was allowed to finish.
- Which test killed which of the 4 killed mutants was not measured.
- The count of 6 runtime asserts for the second test against 10 written is unexplained; the test passes and is not
vacuous either way.
- The tests pin the model's own tie rules and scan bounds; they do not compare the model with ngspice.
- Only `sar_kick.t27` was gated here. The other analog specs have a census with survivors not yet pinned: #12098
(`sar_cdac`), #12099 (`sar_comparator_psp`).
66 changes: 66 additions & 0 deletions specs/analog/sar_kick.t27
Original file line number Diff line number Diff line change
Expand Up @@ -554,3 +554,69 @@ test the_dac_has_settled_where_it_matters_and_the_edge_kick_is_alike_from_the_si
assert fabs64(DEC_D[21] - LD_D[7]) < 0.00000001;
}

; --- the tests added by the mutation lane (issue 12097): appended here so no line above moved -------------------------------------------------

test a_decision_index_without_records_has_no_errors_and_its_threshold_run_is_the_whole_scan_grid {
// no pinned decision has index 0 or 11: nothing disagrees with any threshold, so the fewest errors are 0 and the first run of them is the whole grid, points 0 .. 999
assert_eq(late_count(0), 0);
assert_eq(late_count(11), 0);
assert_eq(late_errors(0, 0.0 - 0.009), 0);
assert_eq(late_errors(0, 0.0), 0);
// by hand: the middle is point 499.5, -9 mV + 499.5 * 5 uV = -6.5025 mV; the half width is 999 steps over two, 499.5 * 5 uV = 2.4975 mV
const t0 : KickThr = late_thr(0);
assert_eq(t0.errors, 0);
assert fabs64(t0.mid + 0.0065025) < 0.000000001;
assert fabs64(t0.hw - 0.0024975) < 0.000000001;
const t11 : KickThr = late_thr(11);
assert_eq(t11.errors, 0);
assert fabs64(t11.mid + 0.0065025) < 0.000000001;
assert fabs64(t11.hw - 0.0024975) < 0.000000001;
}

test a_decision_exactly_on_the_threshold_follows_the_rule_a_0_above_or_a_1_at_or_below_is_an_error {
const eps : f64 = 0.000000001;
// each of the two chosen D values is the only one of its kind among the 112 records, so a nanovolt step moves exactly one record across the threshold
var n26 : u32 = 0;
var n3 : u32 = 0;
var i : usize = 0;
while (i < NLD) {
if (fabs64(LD_D[i] - LD_D[26]) < eps) { n26 = n26 + 1; }
if (fabs64(LD_D[i] - LD_D[3]) < eps) { n3 = n3 + 1; }
i = i + 1;
}
assert_eq(n26, 1);
assert_eq(n3, 1);
// record 26 is a 0 of index 9 at -6.7000 mV: at t = D it is right (D is not above t), one nanovolt lower it is wrong
assert_eq(LD_IDX[26], 9);
assert_eq(LD_BIT[26], 0);
assert_eq(late_errors(9, LD_D[26]), late_errors(9, LD_D[26] + eps));
assert_eq(late_errors(9, LD_D[26] - eps), late_errors(9, LD_D[26]) + 1);
// record 3 is a 1 of index 10 at -6.8894 mV: at t = D it is wrong (D is at t), one nanovolt lower it is right
assert_eq(LD_IDX[3], 10);
assert_eq(LD_BIT[3], 1);
assert_eq(late_errors(10, LD_D[3]), late_errors(10, LD_D[3] + eps));
assert_eq(late_errors(10, LD_D[3] - eps) + 1, late_errors(10, LD_D[3]));
}

test the_scan_grid_end_and_the_series_resistor_bracket_leave_no_tie_or_extra_point_to_decide_anything {
// the point `j <= SCAN_N` would add is grid point 1000, -4.0 mV: above every pinned threshold, so no index with records has fewer errors there than its best, and the pooled count is no lower
const tend : f64 = SCAN_LO + SCAN_STEP * 1000.0;
assert fabs64(tend + 0.004) < 0.000000001;
var ix : u32 = 7;
while (ix <= 10) {
const th : KickThr = late_thr(ix);
assert(late_errors(ix, tend) >= th.errors);
ix = ix + 1;
}
assert(late_errors(9, tend) + late_errors(10, tend) >= pooled_min_errors());
// rs_zero_ohm reads no input: the offsets at 0, 1300 and 2592 ohm are 9.4155, 4.086 and -0.8945 mV against the latch's 0.153 mV, none equal to it
const lat : f64 = 0.0 - LATCH_FLIP_V;
assert(first_off(V_BASE) > lat);
assert(rs_off(0) > lat);
assert(rs_off(1) < lat);
// so the bracket is the second row (i == 1), the scan returns there, and neither the row past NRS nor the "none" value is reached
const r0 : f64 = rs_zero_ohm();
assert(r0 > RS_OHM[0]);
assert(r0 < RS_OHM[1]);
}

Loading