watch_pull_request flaps between "Checks failed (action-required)" and "All N checks passed" on a fork PR whose workflows await maintainer approval #24575

What happened

A thread called watch_pull_request on a first-contributor PR (#15889, head 3b4d479) whose pull_request workflows sat in action_required, waiting for a maintainer to approve them. The watch woke the thread over and over on that one unchanged commit, alternating between two contradictory messages:

• Checks failed on 3b4d479: listing the gated workflows as (action-required) — notification outcome failed
• All 7 checks passed on 3b4d479. — notification outcome completed

Nothing had run. The 7 "passed" checks are the bot and pull_request_target jobs that do not need approval; the four workflows that carry the real CI (CI, Web Preview, Mobile EAS Preview, Mobile Fingerprint Check) never started. The thread was woken on every flip, and the "all checks passed" wake tells the agent the PR is green when it is not.

Diagnosis

Read at main 3cc31ed (2026-10-24). The classification has not changed since the watch landed in #15057.

For a fork PR, the watch's check list is assembled from two independent reads (packages/source-control-github/src/server/GitHubPullRequestProvider.ts:294-340, readChecks):

1. the GraphQL statusCheckRollup → the real check-runs and commit statuses;
2. a separate REST read, GET /actions/runs?head_sha=…&event=pull_request&status=action_required (GitHubPullRequestApi.ts:1540-1575, listActionRequiredRuns), whose runs are appended as synthetic checks with status action-required (GitHubPullRequestProvider.ts:150-183, withWorkflowApprovals).

A workflow run awaiting approval has no check-run at all, so those four workflows exist only through read 2. On this commit, read 1 yields exactly 7 checks (6 check-runs: 4 success, 2 skipped; plus the CodeRabbit commit status success), and read 2 yields 4 runs with conclusion: action_required (GitHub API, queried 2026-10-24, same answer as on 2026-10-05).

evaluatePullRequestWatch (apps/server/src/orchestration-v2/pullRequestWatch.ts) then:

• treats action-required as a failed check (:33-35, isFailedCheck), so the four synthetic entries produce a checks-failed wake;
• recomputes failedChecks from the current list every pass (:56-59), and reports every name not already in it as newly failed (:57);
• passes the gate when nothing in the list is pending or failed (:63, :80-81); with no required checks the gate is the whole list, so "All 7 checks passed" is the rollup alone.

So whenever read 2 answers with zero runs on one pass (a successful empty answer, not an error — an error produces a placeholder Workflow approval status entry that is still action-required), the four synthetic entries vanish, failedChecks resets to [], the 7 rollup checks satisfy the gate, and the watch reports "All 7 checks passed". On the next pass that returns the runs, all four names are "newly" failed again and the checks-failed wake repeats. The guard that "an empty list keeps the last state" (:50-52) covers only a wholly empty list; it does not notice the approval slice going empty while the rollup still has checks.

What I cannot show from the code is why the run list answered empty on alternate passes on 2026-10-05 (then read through gh run list --status action_required, same filter). Either way the watch should not turn a transient empty answer into a green verdict, and an action_required run is not a terminal state: GitHub's own rollup mapping in the same package already treats it as pending (gitHubPullRequestJson.ts:685-694, rollupChecksState → "pending"), while the watch treats it as failed — the two disagree.

Expected: action_required is "waiting for a maintainer", non-terminal — not a failure wake, and never counted as passed. At most one wake saying CI is awaiting maintainer approval, then silence until the runs are approved and finish (or the head moves). A pass on which the approval read returns no runs while the previous pass had some should keep the last approval state (as the whole-list guard already does), not flip the gate.

Steps to reproduce

1. Open a PR from a fork of a repository where first-contributor workflows need maintainer approval (pull_request runs sit in action_required; some pull_request_target/bot checks still complete).
2. In a T3 Code thread, link the PR and call watch_pull_request.
3. Leave the PR untouched, with the workflows unapproved.
4. Observe the wakes over the next hours: Checks failed … (action-required) and All N checks passed alternate on the same head SHA, N being the count of checks that did not need approval.

Version

Observed on our own build of upstream main a1d9d72 (2026-10-05, watch from #15057). Code read at main 3cc31ed (2026-10-24): the mapping and the two-read assembly are unchanged.

Environment

macOS 26 (Apple Silicon); GitHub.com; fork PR from a first-time contributor (vouch:unvouched); PR linked to the thread with link_pull_request, then watch_pull_request.

Evidence

• PR: feat(usage): count Pi sessions and group usage by model family #15889, commit 3b4d479b0f7c1f800d2557c02da8270769a88b20.
• Gated run: https://github.com/pingdotgg/t3code/actions/runs/37264286076 (CI, status: completed, conclusion: action_required, created 2026-10-05T04:36:50Z). The three sibling runs on the same head: 37264285966, 37264286046, 37264286061.
• GET /repos/pingdotgg/t3code/commits/3b4d479…/check-runs?filter=all → total_count: 6 ([code]smith skipped, Sync PR size label definitions skipped, Label PR size, Label PR 15889, Collect PR targets, Prepare PR size config success). GET …/commits/3b4d479…/status → one status, CodeRabbit success. That is the 7.
• GET /repos/pingdotgg/t3code/actions/runs?head_sha=3b4d479…&branch=usage-group-by&event=pull_request&status=action_required → total_count: 4, all completed / action_required.
• Wake texts seen in the thread on 2026-10-05: Checks failed on 3b4d479: with (action-required) entries, then All 7 checks passed on 3b4d479., repeating on the unchanged head.

Related issues

• Contributor fork PRs can appear red due to Actions approval and Vercel authorization #249 (closed): fork PRs look red because of Actions approval — the same gating, seen in the PR UI rather than the watch.
• [Bug]: PR watch reports "All checks passed" while a second run of the same check is still in progress #17207 (closed): the watch reported "All checks passed" while a second run was still going — a different source of a false green.
• Ideas Watch a GitHub Actions workflow run (e.g. a post-merge deploy) like watch_pull_request #16793 (watch a workflow run) touches the approval state but not this.

Fix applied or workaround

None applied in our build. Workaround: unwatch_pull_request and poll from a scheduled task until a maintainer approves the workflows.

Suggested shape: map action_required to a non-terminal state in the watch (treat it like pending for the gate, with one dedicated wake), and when the approval read returns no runs on a pass where the previous pass had some, keep the previous approval entries, as the whole-list guard at pullRequestWatch.ts:50-52 does for a fully empty answer.

Filed by

astarktc, from a T3 Code thread (Claude), after reading main 3cc31ed and querying the GitHub API for the commit above.