dev-lead: lone cancelled check cycles 30-minute retries forever — nothing re-triggers the cancelled workflow #466

Problem

Follow-up to #461 / PR #462 (documented there as a known limitation).

has_hard_blockers in scripts/dev-lead-fix-reviews.sh counts conclusion == "cancelled" (and timed_out) check runs as Tier-1 blockers. Since #462, a cancelled run that was superseded by a newer same-named non-cancelled run is correctly dropped by the Stage-2 dedup in fetch_pr_context. But a lone cancelled run — genuinely the latest of its (name, app, suite) with no replacement — still blocks, and the retry loop can never resolve it:

1. dev-lead-retry.sh cron sees the status=rate-limited reason=blocked marker, waits out the 30-minute reset, re-dispatches the intent.
2. dev-lead-fix-reviews.sh re-runs the engine, which makes no changes (there is nothing to fix in the code).
3. fetch_pr_context re-fetches the same lone cancelled check run; has_hard_blockers is still true.
4. A fresh reason=blocked marker with a new +30min reset is posted (the old one is deleted). GOTO 1.

The cycle burns an engine invocation plus a few dozen API calls every 30 minutes, indefinitely, until a human notices and clicks "Re-run" on the cancelled workflow. The retry machinery retries the agent, but the blocker is the check — nothing in the loop ever re-triggers it.

Observed in practice on PR #462 itself: a concurrency-cancelled PR Review Agent run sat as the only red check until it was manually re-run (gh run rerun <run-id> → success).

Why cancellations end up lone

• concurrency.cancel-in-progress races where the superseding run lands on a different head SHA (the cancelled run stays the latest for its SHA, which the PR may return to via revert/rebase).
• Manual cancellation from the Actions UI.
• Runner eviction / infra cancellations with no automatic re-queue.

Proposed solutions

Option A — dev-lead re-runs the cancelled workflow (preferred)

When the hard-blocker set consists only of cancelled/timed_out check runs (no failures, no CHANGES_REQUESTED), have dev-lead-fix-reviews.sh re-trigger them before falling back to the blocked-retry marker:

• Map check run → workflow run (GET /repos/{repo}/actions/runs?head_sha={sha}, match check_suite_id), then gh run rerun <run_id> (or POST /actions/runs/{id}/rerun).
• Post the reason=blocked marker as today so the cron re-checks after the re-run completes.
• Guard against re-run loops: only re-run a given workflow run once per SHA (record the attempt in the marker, e.g. rerun-attempted=<run_id>), so a workflow that is cancelled again (e.g. by a still-racing concurrency group) degrades to today's behavior instead of ping-ponging.

Engine-free: this path needs no AI invocation at all, so it can run before the engine and skip the wasted invocation in step 2 above.

Option B — retry-budget cap with escalation

Cap consecutive reason=blocked cycles per (SHA, intent) — e.g. 4 cycles (~2h) — by counting prior markers, then stop retrying and post a visible escalation comment mentioning the PR author: "blocked on a cancelled required check; please re-run it". Cheap to implement, but still requires a human and leaves the PR stalled in the meantime.

Option C — stop counting cancelled as a hard blocker

Treat cancelled like skipped/neutral. Rejected as the sole fix: a cancelled required check genuinely blocks GitHub's merge gate, and silently posting status=no-changes would mask a real blocker (this is why #462 deliberately kept lone-cancelled blocking — see the regression-guard test review-changes: lone cancelled check run still counts as a hard blocker).

Recommendation: A, with B's cap as a backstop for workflows that keep getting cancelled on re-run.

References

• dev-lead falsely reports "all AI engines are currently rate-limited" when the real cause is stale cancelled check runs #461 (original false "rate-limited" incident), PR fix(dev-lead): stop false "rate-limited" claims; dedupe superseded check runs #462 (fix + known-limitation note in description)
• scripts/dev-lead-fix-reviews.sh: has_hard_blockers, fetch_pr_context Stage-2 dedup comment, blocked-path in the fix-reviews/review-changes intent arms
• scripts/dev-lead-retry.sh: scan_pr_for_rate_limits re-dispatch loop