Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,12 @@ Run `npm run test:projects:check` after adding or moving a test.
## Regression evidence

Reproduce a defect before fixing it when feasible. If reproduction is not feasible, record why and
preserve the strongest pre-fix evidence. Add regression coverage at the earliest stable behavior
boundary that could detect the defect. Add higher-level coverage only for a distinct integration
boundary. Include negative and state-safety evidence when the acceptance criteria or risk require it.
preserve the strongest pre-fix evidence. Find the nearest semantic test owner before adding coverage.
Prefer improving its input or oracle and remove overlapping coverage when that preserves the
contract. Add a new case or file only for a distinct gap that existing tests cannot express. Choose
the smallest coverage change that detects the defect. Use higher-level coverage only for a distinct
integration boundary. Include negative and state-safety evidence when the acceptance criteria or
risk require it.

Rerun affected tests after an edit or hook autofix changes tested behavior.

Expand Down
14 changes: 12 additions & 2 deletions test/automation/pull-requests/pr-review-advisor-context.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -81,15 +81,25 @@ describe("PR review advisor", () => {
expect(
[
"testDepth.suggestedTests and staticTestInventory are internal starting points for selecting existing validation, not proof that coverage is absent or authorization to add or modify tests.",
"Prefer, in order: cite existing coverage unchanged; extend an existing owner with one missing case; add a new test only when no existing owner can express the behavior; or state why automated coverage does not apply.",
"Prefer improving or replacing existing coverage when its input or oracle can detect the defect.",
"Recommend new coverage only for a distinct behavior gap that no existing owner can express without losing another contract, and explain why.",
"Choose the smallest coverage change that detects the defect and remove overlap it makes redundant.",
"A changed source file without a changed test file does not establish a gap.",
"Review every invariant listed in riskPlan against the diff and checked-in test evidence under the general regression-evidence rule above. After applying that rule, report a finding when a changed invariant lacks applicable checked-in regression evidence, unless a more specific finding already covers the same gap.",
"Selecting an existing E2E selector identifies applicable validation; only its revision-bound result can validate the PR. It does not authorize adding or modifying E2E tests, assertions, fixtures, selectors, matrix entries, jobs, or workflow fan-out.",
"Improve existing live proof when it already reaches the changed boundary; remove overlapping proof when that preserves the contract.",
"Propose a new live E2E test only when the changed behavior crosses a real external boundary that no existing live proof reaches.",
"If a real boundary gap is outside the accepted scope of the current PR, record it as a limitation instead of asking this PR to add coverage.",
"missingRegressionTest with exactly one decision",
"missingRegressionTest that identifies existing evidence or the smallest coverage change needed.",
].filter((clause) => !prompt.includes(clause)),
).toEqual([]);

expect(
[
"classify each finding's coverage",
"expected net change in test cases, assertions, and test files",
].filter((clause) => prompt.includes(clause)),
).toEqual([]);
});

it("keeps heuristic test-depth outputs factual while the prompt owns coverage decisions", () => {
Expand Down
4 changes: 4 additions & 0 deletions tools/pr-review-advisor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,10 @@ Each specialist returns a Markdown review grounded in repository evidence and sh
guidance. No component combines findings or makes merge decisions. Specialist reviews are advisory.
They do not replace required human review or change repository merge gates.

The canonical coverage-decision contract lives in `trusted-guidance.mts`. It asks specialists to
find the nearest test owner, prefer improving or replacing existing coverage, and justify any new
coverage as the smallest way to detect a distinct behavior gap.

Each specialist also records all additional E2E recommendations through a validated tool.
The receipt preserves the deterministic floor, optional coverage, explicit empty decisions, and unresolved coverage.
The [review queue contract](REVIEW-QUEUE.md) defines discovery, identity, dispatch, and result rules for read-only consumers.
Loading
Loading