π€ Agent-authored β filed by the adversarial adjudication of #1614, from a code finding that neither the issue nor its synthesis had reached.
The merge gate can pass while the configured review bot is mid-review of the current head. Nothing in the gate asks whether the reviewer has reviewed this head, and the module that would answer cannot be configured for a reviewer that posts a review instead of a commit status.
The gap, in three parts
1. The merge gate never receives the reviewer configuration. babysit_review_bot_logins reaches exactly two consumers:
plugins/source-control/skills/babysit-prs/scripts/pr_queue_snapshot.py:82
plugins/source-control/skills/babysit-prs/scripts/request_review.py:55
babysit_merge.py is not one of them. It fetches reviews, review threads, and the branch rules β including required_review_thread_resolution β but it has no notion of a configured reviewer, so it cannot ask the one question that matters here: has that reviewer reviewed the head I am about to merge?
2. The reviewer-awareness module is all-or-nothing and requires a commit-status context. babysit_review_trigger.py:51-53:
@property
def configured(self) -> bool:
return bool(self.trigger_phrase and self.reviewer_logins and self.gate_context)
and review_gate_state() (same file, line 243) derives every signal β gate_state, workflow_state, request_signal_pending β by matching gate_context against StatusContext / CheckRun entries in the check rollup. Its own docstring: "With no configured contexts nothing matches, so every summarized state degrades to absent β¦ the dormant default."
3. This reviewer publishes no such context. chatgpt-codex-connector posts a pull-request review. It creates no commit status and no check run, so there is no gate_context value that could be configured. The module is therefore not merely unconfigured here β it is unconfigurable for this reviewer, and reviewer_logins alone cannot activate it because configured demands all three slots.
Net effect: for a reviewer of this shape, no pending-engagement blocker can ever fire, and the merge gate is independently blind regardless.
Observed consequence β PR #1594
Codex reviewed five successive heads. Latency from commit to posted review, every round:
| Head |
Committed |
Reviewed |
Latency |
dddce901 |
18:43:30Z |
18:48:44Z |
5m14s |
81e5369d |
19:56:53Z |
20:01:47Z |
4m54s |
372b9872 |
21:24:27Z |
21:28:29Z |
4m02s |
3ec45192 |
21:34:46Z |
21:39:04Z |
4m18s |
edd9cbfb |
21:44:00Z |
21:49:06Z |
5m06s |
Mean 4m43s, never below 4m02s. The PR merged at 21:48:40Z β 4m40s after its final commit, and 26 seconds before round 5 landed. Round 5 carried two valid findings, one of them a regression the PR itself introduced (both filed as #1613). The gate had no way to know a review was in flight, so it reported ready and the merge proceeded.
This was not a near-miss at the edge of a wide distribution. 4m40s sits below the mean of every prior observation on the same PR.
Note also that round 5 posted after head_ref_deleted. No push-side discipline can prevent a trailing round; only declining to merge inside the reviewer's latency window can.
Why configuration cannot close this
Setting babysit_review_bot_logins alone leaves ReviewTriggerConfig.configured false, so the module stays dormant. There is no gate_context to set. And even a fully configured module would only raise a snapshot blocker β babysit_merge.py would still merge on its own verdict, because it never sees the keys.
Proposed shape
A hold in the merge gate that does not depend on a status context: refuse ready when both hold β
- no submitted review or inline review comment whose own commit id equals the current head exists from any login in the configured reviewer set (the same current-head test
review-trigger.md already specifies for completion: "a submitted review or inline review comment whose own commit ID equals the current head SHA and whose author has the authoritative GitHub Bot type plus an exact login from <review-bot-logins>"); and
- the current head is younger than a configured settle window.
Two conventions the design should follow rather than re-derive:
- Fail open when unconfigured. No reviewer set configured means no hold and no behavior change β mirroring how
gate_context already degrades to absent. This keeps every repo that has not wired a reviewer exactly as it is today.
- Bounded, not indefinite. The hold expires with the settle window; a reviewer that never engages must not wedge the PR.
review-trigger.md already takes this position for the trigger path: "If the reviewer does not engage after that one request, report it rather than retrying."
The change is strictly in the safe direction β it can only withhold a merge, never permit one the gate would otherwise refuse.
Whoever picks this up should decide the settle-window default against fresh measurements rather than inheriting the five samples above, and should confirm the fail-open-when-unconfigured reading before implementing. If either of those turns out to be a real policy call rather than a convention lookup, bounce this to status: needs-decision instead of guessing.
Acceptance criteria
babysit_merge.py receives the configured reviewer set and holds ready per the rule above, with the settle window configurable through the existing effective-configuration seam.
- With no reviewer configured, gate behavior is byte-for-byte what it is today β covered by a test.
- A head with a current-head review from a configured reviewer merges immediately, with no settle wait.
- The hold expires with the window rather than persisting until a review arrives.
reference/safety.md's "Two Gates, One Merge-Ready Authority" reflects the new policy blocker, since that section enumerates what the merge gate adds beyond GitHub's own mergeability.
Context
Surfaced while adjudicating #1614, which attributed the #1594 merge to an unbounded review-round loop. The round count was not the defect β every one of the 15 findings across five rounds was valid and distinct. The defect is that the merge fired inside a well-characterized reviewer latency window with no mechanism able to see it. Related: #1594 (the run), #1613 (the findings that arrived 26 seconds late).
π€ Agent-authored β filed by the adversarial adjudication of #1614, from a code finding that neither the issue nor its synthesis had reached.
The merge gate can pass while the configured review bot is mid-review of the current head. Nothing in the gate asks whether the reviewer has reviewed this head, and the module that would answer cannot be configured for a reviewer that posts a review instead of a commit status.
The gap, in three parts
1. The merge gate never receives the reviewer configuration.
babysit_review_bot_loginsreaches exactly two consumers:plugins/source-control/skills/babysit-prs/scripts/pr_queue_snapshot.py:82plugins/source-control/skills/babysit-prs/scripts/request_review.py:55babysit_merge.pyis not one of them. It fetches reviews, review threads, and the branch rules β includingrequired_review_thread_resolutionβ but it has no notion of a configured reviewer, so it cannot ask the one question that matters here: has that reviewer reviewed the head I am about to merge?2. The reviewer-awareness module is all-or-nothing and requires a commit-status context.
babysit_review_trigger.py:51-53:and
review_gate_state()(same file, line 243) derives every signal βgate_state,workflow_state,request_signal_pendingβ by matchinggate_contextagainstStatusContext/CheckRunentries in the check rollup. Its own docstring: "With no configured contexts nothing matches, so every summarized state degrades toabsentβ¦ the dormant default."3. This reviewer publishes no such context.
chatgpt-codex-connectorposts a pull-request review. It creates no commit status and no check run, so there is nogate_contextvalue that could be configured. The module is therefore not merely unconfigured here β it is unconfigurable for this reviewer, andreviewer_loginsalone cannot activate it becauseconfigureddemands all three slots.Net effect: for a reviewer of this shape, no pending-engagement blocker can ever fire, and the merge gate is independently blind regardless.
Observed consequence β PR #1594
Codex reviewed five successive heads. Latency from commit to posted review, every round:
dddce90181e5369d372b98723ec45192edd9cbfbMean 4m43s, never below 4m02s. The PR merged at 21:48:40Z β 4m40s after its final commit, and 26 seconds before round 5 landed. Round 5 carried two valid findings, one of them a regression the PR itself introduced (both filed as #1613). The gate had no way to know a review was in flight, so it reported ready and the merge proceeded.
This was not a near-miss at the edge of a wide distribution. 4m40s sits below the mean of every prior observation on the same PR.
Note also that round 5 posted after
head_ref_deleted. No push-side discipline can prevent a trailing round; only declining to merge inside the reviewer's latency window can.Why configuration cannot close this
Setting
babysit_review_bot_loginsalone leavesReviewTriggerConfig.configuredfalse, so the module stays dormant. There is nogate_contextto set. And even a fully configured module would only raise a snapshot blocker βbabysit_merge.pywould still merge on its own verdict, because it never sees the keys.Proposed shape
A hold in the merge gate that does not depend on a status context: refuse
readywhen both hold βreview-trigger.mdalready specifies for completion: "a submitted review or inline review comment whose own commit ID equals the current head SHA and whose author has the authoritative GitHubBottype plus an exact login from<review-bot-logins>"); andTwo conventions the design should follow rather than re-derive:
gate_contextalready degrades toabsent. This keeps every repo that has not wired a reviewer exactly as it is today.review-trigger.mdalready takes this position for the trigger path: "If the reviewer does not engage after that one request, report it rather than retrying."The change is strictly in the safe direction β it can only withhold a merge, never permit one the gate would otherwise refuse.
Whoever picks this up should decide the settle-window default against fresh measurements rather than inheriting the five samples above, and should confirm the fail-open-when-unconfigured reading before implementing. If either of those turns out to be a real policy call rather than a convention lookup, bounce this to
status: needs-decisioninstead of guessing.Acceptance criteria
babysit_merge.pyreceives the configured reviewer set and holdsreadyper the rule above, with the settle window configurable through the existing effective-configuration seam.reference/safety.md's "Two Gates, One Merge-Ready Authority" reflects the new policy blocker, since that section enumerates what the merge gate adds beyond GitHub's own mergeability.Context
Surfaced while adjudicating #1614, which attributed the #1594 merge to an unbounded review-round loop. The round count was not the defect β every one of the 15 findings across five rounds was valid and distinct. The defect is that the merge fired inside a well-characterized reviewer latency window with no mechanism able to see it. Related: #1594 (the run), #1613 (the findings that arrived 26 seconds late).