Spun out of Lane C (#2936) during adversarial validation. Refs #2933.
What we found
The tautological-test anti-pattern is well covered in prose — plugins/tdd/skills/principles/reference/anti-patterns-khorikov.md:96,270 and, as a checklist item, plugins/testing/skills/write/context/write.md:78 ("expected values are independently sourced … never recomputed the same way the code under test computes them").
While adjudicating Lane C, coverage was initially claimed to be executable as well, via testing/audit's cant-fail-scan.sh rule testing/audit/rule-recomputed-expectation. That claim is false, and the script says so in its own header:
v1 detects the decidable core — textually identical sides — not every recomputation shape.
A validator ran the scanner over three canonical tautological tests and got blocks that fired a block rule: 0. The canonical Khorikov shape — compute expected with the production algorithm in the arrange section, then assert against it — has non-identical sides and does not fire.
The gap
Nothing judges the semantic shape. A reviewer reading the diff can see "this expected value was derived the same way the code derives it"; no criterion asks them to.
What to build
A criterion in the review code lens (plugins/review/skills/quality-gate/context/criteria.md and/or plugins/review/agents/code-reviewer.md — placement is part of the work) that asks, of a changed or added test: is the expected value independently sourced? A known-good literal, a hand-computed value, a worked example, a fixture, or the spec — as against re-deriving it through the same steps the code under test takes.
Scope notes:
- This is a review-lens criterion, not a detector upgrade. Widening
rule-recomputed-expectation past textually-identical sides is a separate, harder question (the general shape is undecidable) and is explicitly not in scope here.
- The criterion must respect the finding-suppression doctrine already in the lens: skip what tooling already enforces. Where
cant-fail-scan.sh does fire, the scanner owns the finding.
- Round-trip / identity assertions (output compared against its own input) are the adjacent case named at
write.md:78 and belong in the same criterion.
Acceptance criteria
Spun out of Lane C (#2936) during adversarial validation. Refs #2933.
What we found
The tautological-test anti-pattern is well covered in prose —
plugins/tdd/skills/principles/reference/anti-patterns-khorikov.md:96,270and, as a checklist item,plugins/testing/skills/write/context/write.md:78("expected values are independently sourced … never recomputed the same way the code under test computes them").While adjudicating Lane C, coverage was initially claimed to be executable as well, via
testing/audit'scant-fail-scan.shruletesting/audit/rule-recomputed-expectation. That claim is false, and the script says so in its own header:A validator ran the scanner over three canonical tautological tests and got
blocks that fired a block rule: 0. The canonical Khorikov shape — computeexpectedwith the production algorithm in the arrange section, then assert against it — has non-identical sides and does not fire.The gap
Nothing judges the semantic shape. A reviewer reading the diff can see "this expected value was derived the same way the code derives it"; no criterion asks them to.
What to build
A criterion in the review code lens (
plugins/review/skills/quality-gate/context/criteria.mdand/orplugins/review/agents/code-reviewer.md— placement is part of the work) that asks, of a changed or added test: is the expected value independently sourced? A known-good literal, a hand-computed value, a worked example, a fixture, or the spec — as against re-deriving it through the same steps the code under test takes.Scope notes:
rule-recomputed-expectationpast textually-identical sides is a separate, harder question (the general shape is undecidable) and is explicitly not in scope here.cant-fail-scan.shdoes fire, the scanner owns the finding.write.md:78and belong in the same criterion.Acceptance criteria
cant-fail-scan.shso the two do not double-reportdocs/upstream/aihero-shipping-course.md, Lane C) records the criterion as landed