Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .claude/skills/ci-gates/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -1355,3 +1355,43 @@ Repository doctrine here is to fix a contradiction where it lives rather than
adding a second truth next to the first. That is easy to honour in code and easy
to forget in a document, where appending is always the cheaper edit — and where
the reader who most needs the correction is the one who stops before reaching it.

## 35. The third axis, and the value of an instrument that finds nothing

Two operators asked whether a gate can still reach its verdicts. `--invert`
asks whether it reaches the **right** one: `if C:` becomes `if not (C):` on
conditions whose body carries a verdict. A gate that fires FAIL on a healthy
tree and OK on a broken one satisfies both return operators and neither is
looking at that.

Thirteen gates, **one survivor** — the same branch both other operators leave,
declared uncovered with its reason.

**That is the first new instrument here to find nothing, and it is worth as much
as the ones that found plenty.** Controls written over a week against two
questions held against a third they were not written for. A tool that comes back
empty on a corpus that has been failing every new question is evidence; a tool
that comes back empty on its first ever run is untested.

### Scope a mutation operator by what its kills would MEAN

Inverting any condition is easy and useless. Inverting a loop guard or a
plumbing check makes the gate **crash**, and a control that reds on a traceback
scores that as a kill — for the wrong reason, which is the mistake the command
exists to catch. Only conditions whose body holds a verdict return, a
`SystemExit` or a FAIL print are sites.

**Then measure that the scoping worked**, rather than asserting it: 19 mutations
across four gates, 19 killed by message, **zero tracebacks**.

### The probe that checks a tool gets less care than the tool

Mine was broken twice in one run: it mutated inside the control functions, which
the real implementation excludes and the probe did not; and its verdict label
conflated *not killed* with *killed by a crash*, so it printed `killed by crash`
for a mutant that had survived.

Its first output read as a finding about the tool. **A throwaway script written
to verify a careful one inherits none of its care**, and it is trusted anyway
because it is short. Seven instruments of my own have now been broken in this
campaign; the throwaway ones account for most of them.
120 changes: 117 additions & 3 deletions cli/tri/src/gates.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,15 @@ pub enum GatesCmd {
/// requires SILENCE?
#[arg(long)]
loud: bool,

/// Invert the CONDITIONS that reach a verdict, rather than the returns.
///
/// The two return operators ask whether a gate can still reach its
/// verdicts. This asks whether it reaches the RIGHT one: a gate that
/// fires its FAIL branch on a healthy tree and its OK branch on a
/// broken one passes both of them.
#[arg(long)]
invert: bool,
},
/// List active workflows whose lifetime success count is zero.
Dead {
Expand Down Expand Up @@ -263,6 +272,16 @@ pub enum Direction {
Silent,
/// `return 0` -> `return 1`: does anything notice the gate getting LOUDER?
Loud,
/// `if <cond>:` -> `if not (<cond>):`, on conditions whose body carries a
/// verdict. The two return operators ask whether the gate can still reach
/// its verdicts; this asks whether it reaches the RIGHT one.
///
/// T104: scoped to conditions whose body holds a `return 1..4`, a
/// `raise SystemExit(N)` or a FAIL print. Inverting a loop guard or a
/// plumbing check makes the gate CRASH, and a control that reds on a
/// traceback would score that as a kill -- the wrong reason, which is the
/// mistake this command exists to catch.
Invert,
}

fn mutable_sites(src: &str) -> Vec<(usize, usize, String)> {
Expand Down Expand Up @@ -324,11 +343,78 @@ fn sites_in_direction(src: &str, dir: Direction) -> Vec<(usize, usize, String)>
sites
}

/// Conditions whose body carries a verdict: `(offset, len, replacement)`.
///
/// Line-based like everything else here, and the body is read to a dedent
/// rather than parsed. A condition whose body only prints or only accumulates
/// is not a decision this command has anything to say about.
fn invert_sites(src: &str) -> Vec<(usize, usize, String)> {
let lines: Vec<&str> = src.split_inclusive('\n').collect();
let mut sites = Vec::new();
let mut in_control = false;
let mut off = 0usize;
let offsets: Vec<usize> = {
let mut v = Vec::with_capacity(lines.len());
let mut o = 0usize;
for l in &lines {
v.push(o);
o += l.len();
}
v
};
for (i, line) in lines.iter().enumerate() {
off = offsets[i];
if let Some(rest) = line.strip_prefix("def ") {
let fname: String = rest.chars().take_while(|c| *c != '(').collect();
in_control = is_control_fn(&fname);
}
if in_control {
continue;
}
let t = line.trim_start();
let indent = line.len() - t.len();
if indent == 0 {
continue;
}
let body_line = t.trim_end();
let cond = match body_line
.strip_prefix("if ")
.and_then(|c| c.strip_suffix(':'))
{
Some(c) if !c.is_empty() => c,
_ => continue,
};
// The body, to the first line at or left of this indent.
let mut carries_verdict = false;
for l in lines.iter().skip(i + 1) {
let s = l.trim_end();
if !s.trim().is_empty() && (l.len() - l.trim_start().len()) <= indent {
break;
}
let b = s.trim();
if b.starts_with("return ")
&& verdict_literals(&b[7..])
.is_some_and(|v| v.iter().any(|&x| (1..=4).contains(&x)))
|| b.starts_with("raise SystemExit(")
|| b.contains("FAIL")
{
carries_verdict = true;
break;
}
}
if carries_verdict {
sites.push((off + indent, body_line.len(), format!("if not ({cond}):")));
}
}
let _ = off;
sites
}

fn line_of(src: &str, byte: usize) -> usize {
src[..byte].matches('\n').count() + 1
}

fn mutate(only: Option<&str>, loud: bool) -> Result<()> {
fn mutate(only: Option<&str>, loud: bool, invert: bool) -> Result<()> {
let root = repo_root()?;
let dirty = Command::new("git")
.args(["status", "--porcelain", "--", "tools/"])
Expand All @@ -353,7 +439,10 @@ fn mutate(only: Option<&str>, loud: bool) -> Result<()> {
files.sort();

println!("{:<38} {:>9} {}", "gate", "mutants", "verdict");
if loud {
if invert {
println!("(invert: `if C:` -> `if not (C):` where the body carries a verdict.");
println!(" A survivor means the gate can reach the WRONG verdict unnoticed.)");
} else if loud {
println!("(loud: `return 0` -> `return 1`. A survivor means NOTHING requires this gate to be silent.)");
}
let mut total_survived: Vec<String> = Vec::new();
Expand Down Expand Up @@ -581,7 +670,7 @@ fn repo_root() -> Result<std::path::PathBuf> {
pub fn run(cmd: &GatesCmd) -> Result<()> {
match cmd {
GatesCmd::Sweep { controls_only } => sweep(*controls_only),
GatesCmd::Mutate { only, loud } => mutate(only.as_deref(), *loud),
GatesCmd::Mutate { only, loud, invert } => mutate(only.as_deref(), *loud, *invert),
GatesCmd::Dead { repos, min_runs } => {
let list: Vec<String> = if repos.is_empty() {
["gHashTag/trinity", "gHashTag/trinity-fpga", "gHashTag/t27"]
Expand Down Expand Up @@ -877,6 +966,31 @@ def main():\n if problems:\n return 2\n return 0\n";
assert!(mutable_sites(src).is_empty());
}

#[test]
fn invert_takes_only_verdict_bearing_conditions() {
// T104: inverting a loop guard or a plumbing check makes the gate
// CRASH, and a control that reds on a traceback scores that as a kill
// for the wrong reason. Measured over 19 mutations on four gates with
// this scoping: 19 killed by MESSAGE, zero tracebacks.
let s = invert_sites("def main():\n if bad:\n return 1\n");
assert_eq!(s.len(), 1);
assert_eq!(s[0].2, "if not (bad):");

// A body that only prints decides nothing this command can speak to.
assert!(invert_sites("def main():\n if x:\n print(x)\n").is_empty());
// A body that returns success is not a verdict branch either.
assert!(invert_sites("def main():\n if x:\n return 0\n").is_empty());
// A FAIL print counts even without a return: the branch still decides.
assert_eq!(
invert_sites("def main():\n if x:\n print(\"FAIL: no\")\n").len(),
1
);
// Inside the control, never.
assert!(invert_sites("def self_check():\n if bad:\n return 1\n").is_empty());
// Top-level `if` is module plumbing, not a gate decision.
assert!(invert_sites("if bad:\n return 1\n").is_empty());
}

#[test]
fn control_files_are_not_counted_as_gates() {
// T85: wp18_selftest_gate.py IS the control for wp18_conformance_gate.py.
Expand Down
21 changes: 21 additions & 0 deletions docs/now/2026-08-23-the-third-axis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# NOW — the third axis (2026-08-23)

Two operators asked whether a gate can still reach its verdicts. The third asks whether it reaches the **right** one: a gate that fires its FAIL branch on a healthy tree and its OK branch on a broken one satisfies both of the others.

- **`--invert`.** `if C:` becomes `if not (C):`, on conditions whose body carries a verdict. All thirteen gates: **one survivor**, and it is `check_elab_ratchet`'s no-baseline branch — the same one both other operators leave, declared `UNCOVERED` with its reason.

- **Three axes now converge on one line.** That is the first result in this campaign where a new instrument found nothing new, and it is worth as much as the ones that did: the controls written over the last week hold up against a question they were not written for.

- **The scoping is the whole design.** Inverting a loop guard or a plumbing check makes the gate **crash**, and a control that reds on a traceback would score that as a kill — for the wrong reason, which is the mistake this command exists to catch. Only conditions whose body holds a `return 1..4`, a `raise SystemExit(N)` or a FAIL print are sites.

Measured over 19 mutations on four gates: **19 killed by message, zero tracebacks.** The scoping does what it claims.

- **My verification probe was broken twice before it worked.** It mutated inside the control functions, which the Rust implementation excludes and my Python probe did not; and its verdict label conflated "not killed" with "killed by a crash", so it printed `killed by crash` for a mutant that had survived. Both wrong in the same run, and the first output looked like a finding about the tool.

That is the seventh instrument of my own to be broken in this campaign, and it is the same lesson each time: **the probe written to check a tool needs the care the tool got, and usually gets less.**

Refs #2492

- **This note was first dated 2026-08-25 and the gate rejected it.** The real UTC date is 2026-08-23. I had been inferring the passage of days from the number of iterations rather than reading a clock, and two earlier notes carry the same invented drift.

That is mechanism 8.2 — *the claim was true and the world moved* — turned inside out: the world did not move and I recorded that it had. The gate caught it, which is the gate doing precisely its job, and it is the first time in this campaign that an existing gate caught **me** rather than the other way round.
Loading