Skip to content

test(analog): raw_writer mutation test gaps A-D closed, gate passed (Closes #12273) - #12288

Merged
gHashTag merged 2 commits into
masterfrom
lane/analog-mut-raw-writer-12273
Oct 25, 2026
Merged

gHashTag merged 2 commits into
masterfrom
lane/analog-mut-raw-writer-12273

Conversation

@gHashTag

Copy link
Copy Markdown
Collaborator

Closes #12273. Part of epic #10486 (see it, not closing it). Groups E to J and the dead lines stay in #12274 (Refs #12274).

What

specs/analog/raw_writer.t27 gets 5 new tests (18 -> 23) for the test gaps in groups A to D found by the census of #12116, an accepted-mutants file for the mutants no test can or must kill, and a note with the measured numbers. The tracked generated file gen/rust/analog/raw_writer.rs is regenerated, because t27c seal --verify also checks that it equals what the generator emits today (it still said 18 tests).

Measured

tri mutate spec --accepted on this tree (T27 OS pane, jobs 3, timeout 30, master-built t27c, base d5650360c1, 12:32:08-13:01:29 +07, 29 min 21 s; the 1-minute load was 8.8 at the start and rose above 30 during the run): 526 mutants, 442 killed (442 by a failing test, 0 by an invariant), 53 survived, 31 hung, 0 unviable; 84 not killed, 84 accepted, 0 accepted lines now killed; "Gate passed". The forecast written before the run was 442 / 53 / 31 / 84. tri test on the spec: 23/23 pass, 0 vacuous.

The seal AnalogRawWriter was re-saved by tri seals changed with the master-built t27c; after the generated file was regenerated t27c seal specs/analog/raw_writer.t27 --verify exits 0.

Not claimed

  • The tool names no killing test; "killed by test N" is read from the asserts, and the gate shows only that the targeted mutants are no longer among the survivors.
  • The equivalent survivors and the two dead lines (683 and 954) are classified by reading the code, not proved by a tool; hung mutants are by construction (dropped loop step). 32 survivors wait for analog: raw_writer mutation test gaps, batch 2 of 2 - groups E to J and the two dead lines (after batch 1) #12274.
  • The census was not repeated with the master-built compiler on the unmodified spec (FR-001); before-counts are the census's.
  • The load rose during the run; a hung mutant under load can be a slow one, but the 31 hung entries are all dropped loop steps, which hang by construction.
  • tri coverage --strict was not run locally; CI runs it.

Files: the spec (+146 lines of tests at the end), docs/analog/mutation-accepted-raw_writer.txt, docs/analog/mutation-raw_writer-2026-10-25.md, the seal, gen/rust/analog/raw_writer.rs (generated, one line changed).

🤖 Generated with Claude Code

gHashTag and others added 2 commits October 25, 2026 13:07
Batch 1 of 2 for the census of specs/analog/raw_writer.t27 (526 mutants, 83
survivors, 31 hung): five new tests at the end of the spec target the 30
survivors of groups A to D.

- A (raw_nibble, raw_at_hex, 6 mutants): upper-case digits, an exact 2-byte
  buffer, and a changed last byte of the binary reference file as the
  negative control the byte-for-byte test lacked.
- B (raw_check_vars, 14 mutants): six malformed variable lines and a control.
- C (raw_put_f64 of zero, raw_from_bits of an infinity or NaN, 2 mutants).
- D (raw_uint, raw_count_line at their limits, 8 mutants).

docs/analog/mutation-accepted-raw_writer.txt has 84 entries: 19 equivalent,
2 dead line, 32 pending for #12274 (groups E to J) and 31 hung. The note
docs/analog/mutation-raw_writer-2026-10-25.md labels all 83 survivors and
names the assert that kills each of the 30.

tri test: 23 tests pass (18 before), 0 fail, 0 vacuous. The survivor gate was
not run and the seal was not saved; the after-numbers in the note are a
forecast. Every expected value is hand-derived.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…, gate passed (Refs #12273)

The gate on this tree: 526 mutants, 442 killed, 53 survived, 31 hung, 0 unviable;
84 not killed, 84 accepted, 0 accepted lines now killed. The tracked generated file
gen/rust/analog/raw_writer.rs said 18 tests; it now says 23, as the spec has.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-25 06:08:06 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 41
PRs with All Checks Green 9
READY 7
FAILING 41
PENDING 0
NO CHECKS YET 0

These columns do not partition: 7 + 41 + 0 + 0 = 48, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=55c0f1b1b32d != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@tri-claw

tri-claw Bot commented Oct 25, 2026

Copy link
Copy Markdown

Summary by t27-bees

Checked by the t27 compiler

t27c parse, parse-complete and typecheck on each changed .t27 file at fa70a1c. Nothing was generated, built or run. A pass means the file is well formed, not that it is right.

  • specs/analog/raw_writer.t27: parses and typechecks.

Read by the model

This PR adds 5 mutation tests (groups A–D) to specs/analog/raw_writer.t27, updates the seal, adds acceptance docs, and regenerates the Rust output. The gate forecast is 442 killed / 53 survived / 31 hung / 84 accepted.

Worth a look

  • specs/analog/raw_writer.t27:1420–1423 — raw_check_vars_refuses_malformed_lines labels b6 as “empty type” but the test string "\t0\ta\tb\tc\n" has four fields (index=0, name=a, type=b, extra=c), not an empty type. The mutants S71/S72 (e == t2 + 1 to e == t2 / e == t2 - 1 at line 723) target an empty type field; if no test exercises an actually empty type, those two mutants may survive and the gate will fail on the non-accepted lines.
  • specs/analog/raw_writer.t27:1347–1355 — raw_at_hex_checks_its_bytes asserts exact byte values [171, 205] from raw_put_hex(&o, 0, "abcd"), baking in big-endian byte order (0xAB, 0xCD). If the hex writer’s endianness ever changes, this test will fail even though the round-trip logic is correct.
  • docs/analog/mutation-accepted-raw_writer.txt — Contains 84 entries keyed by line number and mutant kind. Any edit to the spec above the new tests will shift line numbers and cause the gate to fail on stale accepted entries (the PR acknowledges this: “The line numbers hold only while the spec lines above are unchanged”).
  • gen/rust/analog/raw_writer.rs:3 — Only the test-count comment changed (18 → 23). Verify the generator didn’t drop or alter any other backend output; the seal hashes confirm a full regeneration.
  • .trinity/seals/analog_AnalogRawWriter.json — All four gen_hash_* values and the spec_hash changed. Ensure CI re-verifies the seal (t27c seal --verify) on the regenerated artifacts before merge.

Read: 5 files, +430 -11, the whole diff. Head fa70a1c. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

analog: raw_writer mutation test gaps, batch 1 of 2 - groups A to D (hex helpers, malformed vars lines, zero and non-finite bits, header counts)

1 participant