fix(source-control): narrow review-trigger gate contract to pending - #1392
Conversation
…324) Closes #324 The AI-review-trigger contract and its implementation disagreed on whether a FAILING `<review-gate-context>` — not only a PENDING one — makes a PR a review-trigger candidate. `reference/review-trigger.md` documented the trigger signal as the gate being "pending or failing"; every `gate_state` comparison in the engine accepts only `pending`, and `request_signal_pending` is derived solely from a StatusContext whose `effective_state == "PENDING"` with no target URL. ## Decision: honor the code, narrow the doc The issue asked for an empirical tiebreaker — what the wired gate context actually reports when no reviewer has engaged. That configuration lives outside this repository and was not reachable from this worker, so it did not settle the fork. Three pieces of repo-internal evidence did, and they point the same way: 1. The doc's own Engagement Gate Semantics section is the mechanism-level definition of the gate's vocabulary, and it defines exactly two states: `PENDING` = no qualifying reviewer activity after the polling window, and `SUCCESS` = may reflect activity from an earlier head. `FAILING` is given no engagement meaning anywhere in the file. The "pending or failing" phrasing appears only in the downstream restatement of the helper's preconditions, one paragraph below a separate and correct rule about never discarding "a pending or failed StatusContext" — the likelier provenance of the phrase than an authored intent to include failing gates. 2. `request_signal_pending` is `effective_state == "PENDING" and not target_url`. A failing analogue would require knowing what `target_url` a failing gate carries. Nothing in this repository records that, so honoring the doc would have meant inventing a signal shape inside logic that posts comments to GitHub. 3. The issue's motivating harm does not exist. `classify_checks` buckets the gate StatusContext like any other check, so a failing gate already raises `babysit_delta`'s "N failing check(s)" blocker. A failing gate is escalated to a human, not silently dropped — which is the coherent reading of the split: an unengaged reviewer is nudged automatically, a broken gate is not. ## Changes - `reference/review-trigger.md`: the trigger precondition now reads `pending`, and Engagement Gate Semantics states the failing case once, so the contract is complete rather than merely non-contradictory. - `test_review_trigger_race.py`: a regression class pinning the chosen behavior. The rollup makes the gate StatusContext the only non-success check and asserts `ci_gateway_green` and `non_review_checks_green` are both true, so the failing gate is the isolated variable rather than one of several reasons `request_eligible` could be false. It also pins the `else`-branch pass-through (`state == "failing"`) and the failing-check bucketing from point 3. - No code change. All three sites named in the issue, plus `pr_queue_snapshot.py`, already read `pending` and are correct under this decision; a grep of every `gate_state` comparison confirms the enumeration is complete. Reviewer: this is the reversible, no-behavior-change branch of a genuine design fork, taken on repo-internal evidence rather than the direct gate observation the issue asked for. If the deployed `<review-gate-context>` does report FAILING (not PENDING) for "no reviewer engaged yet", override this — the fix is then to widen the candidate predicate and define a failing analogue of `request_signal_pending`, with the observed `target_url` shape in hand. Verification, all run locally in the worktree: - `python -m unittest discover` over `plugins/source-control/skills/babysit-prs/scripts/tests` — 351 tests, OK. - `scripts/validate-plugins.sh` — all plugin manifests and the catalog validated. - `scripts/check-changelog-parity.sh --check` — pass. - `markdownlint-cli2` on both changed markdown files — 0 errors. Local disclosure: `scripts/check-changed-skills.sh` and `scripts/check-skill-portability.sh` were started against `origin/main` but did not complete inside this worker's timebox — bash startup in this environment runs to minutes per invocation, and the base-ref sweep covers every skill that has moved on `origin/main`, not only this change. Neither is likely to be implicated: this change touches a `reference/` spoke and a test file, with no SKILL.md, frontmatter, or portability-token surface. Both are left to required CI. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PiTWQedepGkVz6vxFexpUm
…r-gate-contract # Conflicts: # plugins/source-control/CHANGELOG.md
Resolve the source-control CHANGELOG collision: origin/main released 0.26.4 for #601, so this branch's entry moves to a new 0.26.5 heading and the plugin manifest bumps to match. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H8CQByXzBS3JVJboVJ9eBN
|
Claude finished @kyle-sexton's task in 3m 30s —— View job PR Review —
|
|
Claude finished @kyle-sexton's task in 1m 51s —— View job Security Review — PR #1392
No security issues found. This PR is documentation + regression tests only; no production logic is changed. Here is the full rationale by surface:
No GitHub Actions files changed. No dependency pins touched. No secret or token handling introduced or modified. |
…r-gate-contract # Conflicts: # plugins/source-control/.claude-plugin/plugin.json # plugins/source-control/CHANGELOG.md
|
Claude finished @kyle-sexton's task in 2m 15s —— View job PR Review —
|
|
Claude finished @kyle-sexton's task in 57s —— View job Security Review — PR #1392
No security issues found. This PR is documentation + regression tests only; no production logic is changed. Full rationale by surface:
All test data is fully hardcoded ( The
No GitHub Actions files changed. No dependency pins touched. No secret or token handling introduced or modified. No |
|
This was generated by AI during autonomous /work-items:work-loop execution. Review-comment triage — all four bot commentsFour issue-level comments across two review rounds (pre-merge SHA
On the design decisionNeither reviewer challenged the honor-code-over-honor-doc resolution; both independently re-derived it and confirmed the Fix appliedFinding 1 was a PR-description defect, not a branch defect — no commit was needed. The body has been read-modify-written in place: the version line now reads All 26 check runs on No follow-up issue was filed — there was no VALID-but-deferred finding. |
This was generated by AI during autonomous /work-items:work-loop execution.
Closes #324
Summary
reference/review-trigger.mddocumented the review-trigger signal as<review-gate-context>"pending or failing", while every
gate_statecomparison in the engine accepts onlypending— the candidate predicate and
request_signal_pendinginbabysit_review_trigger.py, and the"awaiting requested review" material signal in
babysit_delta.py. #324 called this a genuinedesign fork and explicitly forbade resolving it by inference.
Decision: honor the code, narrow the doc. Documentation and test only; no behavior change.
The empirical tiebreaker #324 asked for
#324 named the deciding fact: does the
<review-gate-context>the cutover config wires reportFAILING or PENDING when no reviewer has engaged? That fact was located, and it settles the fork
in favor of the code:
codex-review—babysit_review_gate_contextunderpluginConfigs["source-control@melodic-software"]in the dotfiles chezmoi data(
melodic-software/dotfiles,.chezmoidata/claude.json).melodic-software/medley,.github/workflows/comment-review-gate.yml. Every status it writes is eitherstate: "success"(reviewer engaged) or
state: "pending"(not engaged). It never postsfailureorerror:the file's only four
state:assignments are threesuccessand onepending.push) … If not → 'pending'", and "its status stays 'pending' until the next push re-checks."
So "no reviewer engaged yet" is PENDING, never FAILING. The honor-doc branch — widening the
code so a failing gate becomes a trigger candidate — would have been correct only if the gate
reported FAILING for that state, and it does not.
review-trigger.md's "or failing" was theerroneous restatement; the code was right.
Scope note for the reviewer: the gate is deployed in
medley, not in this repository — recentclaude-code-pluginsPRs carry noStatusContextat all, so the evidence above comes from theconfig that names the context plus the workflow that posts it, not from a live status on this PR.
Corroborating repo-internal evidence
Three signals inside this repository point the same way, independently of the gate observation:
review-trigger.md's own Engagement Gate Semantics section — the mechanism-level definition ofthe gate's vocabulary — defines exactly two states (
PENDING= no qualifying reviewer activityafter the polling window,
SUCCESS= may reflect activity from an earlier head) and givesFAILINGno engagement meaning anywhere in the file. The "pending or failing" phrasing appearsonly in the downstream restatement of the helper's preconditions, one paragraph below a
separate and correct rule about never discarding "a pending or failed StatusContext" — the
likelier provenance of the phrase than authored intent to include failing gates.
request_signal_pending(effective_state == "PENDING" and not target_url) would require knowing whattarget_urla failing gate carries. Nothing recordsthat, so honoring the doc would have meant inventing a signal shape inside logic that posts
comments to GitHub.
classify_checksbuckets the gateStatusContextlike anyother check, so a failing gate already raises
babysit_delta's "N failing check(s)" blocker. Afailing gate is escalated to a human, not silently dropped — the coherent reading of the split:
an unengaged reviewer is nudged automatically, a broken gate is not.
Changes
reference/review-trigger.md— the trigger precondition now readspending, and EngagementGate Semantics states the failing case once (a failing gate is not an engagement signal, is never
a trigger candidate, and reaches the operator through the ordinary failing-check blocker), so the
contract is complete rather than merely non-contradictory.
test_review_trigger_race.py— a regression class pinning the chosen behavior. The rollupmakes the gate
StatusContextthe only non-success check and assertsci_gateway_greenandnon_review_checks_greenare both true, so the failing gate is the isolated variable rather thanone of several reasons
request_eligiblecould be false. It also pins theelse-branchpass-through (
state == "failing") and the failing-check bucketing from point 3 above.pr_queue_snapshot.py, already readpendingand are correct under this decision; a grep of everygate_statecomparison confirmsthe enumeration is complete.
0.26.9with the matching CHANGELOG entry.If you want to override
This is the reversible branch. Should the deployed gate ever be rewired to report FAILING (not
PENDING) for "no reviewer engaged yet", the fix is to widen the candidate predicate and define a
failing analogue of
request_signal_pendingwith the observedtarget_urlshape in hand — and theregression class added here is what would need inverting.
Test plan
Run locally in the worktree:
python -m unittest discoveroverplugins/source-control/skills/babysit-prs/scripts/tests—375 tests, OK after integrating
main(includes the 3 new regression tests).scripts/validate-plugins.sh— all plugin manifests and the catalog validated.scripts/check-changelog-parity.sh --checkand--check-bump origin/main— both pass.markdownlint-cli2on both changed markdown files — 0 errors.Left to required CI:
scripts/check-changed-skills.shandscripts/check-skill-portability.sh,which run in the
skill-quality-gateandportability-lintjobs respectively.Related
Refs #322 — PR-B (capability convergence); Codex flagged this doc/code fork inline there.
🤖 Generated with Claude Code