fix(testing): test judge marks a linked-worktree test UNKNOWN because its quotes are grounded against the wrong repository #6206

Activity

1. kyle-sexton

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 4, 2026
2. github-actions commented on Oct 4, 2026

   

   github-actionsbot

   on Oct 4, 2026 – with GitHub Actions

   Contributor

   More actions

   Thanks for the report. Automated triage handed this issue to a maintainer.

   _{Automated triage. A maintainer reviews it.}
3. github-actions

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 4, 2026
4. kyle-sexton commented on Oct 4, 2026

   

   kyle-sexton

   on Oct 4, 2026

   ContributorAuthor

   More actions

   This was generated by AI during work-loop.

   Second occurrence, 2026-10-04T08:07:50Z: the same session (rooted at the main checkout) got the same UNKNOWN for the same linked-worktree file, now lines 1-1743, after the #6187 ready flow had pushed further commits there. Review file: .work/reviews/main/20261004T080750Z-test-judge.md. It recurs at every Stop while the worktree's test differs from main, so each recurrence blocks Stop again.
5. kyle-sexton commented on Oct 4, 2026

   

   kyle-sexton

   on Oct 4, 2026

   ContributorAuthor

   More actions

   This was generated by AI during triage.

   Triage verdict: cause 1 confirmed for the code that ran, and fixed by #6168 (testing 0.22.6, on origin/main). Cause 2 and acceptance criterion 3 remain open. Needs an operator decision on scope.

   Evidence

   • The stored verdict behind the 07:10:11Z review (judged 2026-10-04T07:10:10Z) records repo as the main checkout, not the -6187 worktree. The judge itself returned PASS; validation turned it UNKNOWN because a quoted line was not found in the main checkout. Basis: verified, the testing plugin's data directory verdict record for that file, repo: C:/Projects/melodic/claude-code-plugins, verdict: PASS (before validation).
   • The code that ran predates fix(testing): judge each test file in its own repository, with an absolute Read rule #6168: neither installed testing version (0.21.3 project scope, 0.22.5 user scope) has judge::file_repo, and the local main checkout the directory marketplace points at is 17 commits behind origin/main and does not contain fix(testing): judge each test file in its own repository, with an absolute Read rule #6168. The recurrence at 08:07:50Z is the same stale code.
   • On origin/main 79abd80, judge::run resolves the repository from the test file's own git rev-parse --show-toplevel and records it on the verdict, and judge::validate grounds quotes against that recorded repository. Basis: verified, origin/main plugins/testing/hooks/judge-lib.sh:567-571 and :662-667; plugins/testing/CHANGELOG.md 0.22.6 entry.
   • Acceptance criterion 1 is covered only indirectly: origin/main's test-judge.test.sh asserts a linked-worktree file is judged from the worktree and its verdict names the worktree as its repository (around :780-785), but no test grounds a quote that exists only on the worktree's branch. The stay-quiet half (a quote found nowhere still yields the UNKNOWN) is tested (around :323).
   • Cause 2 is still live: judge::validate filters quotes against the test file as it is at validation time, so if the file is edited after the judge read it, a quote can fail grounding even in the correct repository and produce this same UNKNOWN.
   • Acceptance criterion 3 is untouched: 0.22.5 exempts from blocking Stop only UNKNOWNs for no repository or no judge class; a grounding-failure UNKNOWN still blocks with a request for user action.

   Open question for the operator

   1. Narrow fix(testing): test judge marks a linked-worktree test UNKNOWN because its quotes are grounded against the wrong repository #6206 to cause 2 (a quote made stale by a later edit of the test file) plus acceptance criterion 3 (whether a grounding-failure UNKNOWN carrying no finding should block Stop), and add the explicit worktree-only-quote regression test. Recommended.
   2. Close fix(testing): test judge marks a linked-worktree test UNKNOWN because its quotes are grounded against the wrong repository #6206 as fixed by fix(testing): judge each test file in its own repository, with an absolute Read rule #6168, accepting that a later-edit UNKNOWN can still block Stop.

   Either way, the immediate recurrence stops only once the session runs testing 0.22.6 or later (update the local checkout and installed plugin).

   Routing: human-gated, needs a decision. Priority medium. Work class scoped (bug fix) for whatever remains after the decision.
6. kyle-sexton

   added

   priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

   status: needs-decisionAwaiting a human or maintainer judgment call.Awaiting a human or maintainer judgment call.

   work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

   and removed

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 4, 2026
7. kyle-sexton commented on Oct 4, 2026

   

   kyle-sexton

   on Oct 4, 2026

   ContributorAuthor

   More actions

   This was generated by AI during work-loop.

   Operator decision (2026-10-04): narrow this issue (option a, recommended in the triage comment).

   Cause 1 (grounding against the wrong repository) is fixed by #6168, shipped in testing 0.22.6; the occurrences reported here came from an older installed testing version. What stays in scope:

   1. A quote can fail grounding when the test file changes between the judge's read and validation.
   2. Whether an UNKNOWN that carries no finding should block Stop and ask the user to act.
   3. A regression test: a quote that exists only on a linked worktree's branch is grounded successfully.

   Acceptance: a test for each of 1 and 3, red before the fix, and a recorded decision for 2, implemented if the answer is no.
8. kyle-sexton

   removed

   status: needs-decisionAwaiting a human or maintainer judgment call.Awaiting a human or maintainer judgment call.

   on Oct 4, 2026
9. kyle-sexton

   closed this as completedin #6305on Oct 4, 2026