Skip to content

Make the visual-preview retry budget durable per-PR-head, not per-job-chain (fixes repeated re-review storms) #6323

Description

@JSONbored

Problem

MAX_PREVIEW_POLLS = 5 (src/queue/processors.ts) is meant to bound the visual-preview self-poll to 5 attempts × PREVIEW_POLL_SECONDS (90s) ≈ 7.5 minutes before giving up on a preview deploy that never becomes discoverable. In practice this budget does nothing, because it only lives inside the dedicated recapture-preview job chain's own attempt field, and at least three OTHER re-review triggers call reReviewStoredPullRequest without threading previewPollAttempt through at all (it silently defaults to 0):

  • the CI-completion (check_run/check_suite) webhook handler
  • the deployment_status webhook handler
  • the sweep/reconciliation pass

Each of these independently runs a fresh buildCapture. If previewPending comes back true — which it does on EVERY call for a repo whose CI never produces a deployment buildCapture's discovery chain can find (no GitHub Deployment, no *.workers.dev/*.pages.dev check, no Cloudflare-bot PR comment) — the code re-arms a BRAND-NEW 5-retry countdown, regardless of how many times any prior chain already exhausted its own budget.

Observed live on JSONbored/metagraphed#6036: 7 check-runs completed over ~8 minutes, each independently capable of re-arming the countdown (subject only to an unrelated 60s CI-completion coalesce window, not the visual-poll cap) — the review comment was edited 12+ times over 52+ minutes and was still actively re-triggering when observed. This is pure wasted Browser Rendering + queue/webhook cost on a PR whose "after" side can never resolve anyway.

Area

src/queue/processors.tsMAX_PREVIEW_POLLS, PREVIEW_POLL_SECONDS, the recapture-preview scheduling block, and every reReviewStoredPullRequest call site that currently omits previewPollAttempt.

Proposal

Make the preview-poll budget durable and scoped to the PR's head SHA, not to a single job chain's own payload — e.g. a persisted counter (KV/D1/R2, whatever this pipeline's existing best-effort-state store convention is) keyed by repoFullName:prNumber:headSha, incremented by buildCapture's caller on every attempt regardless of WHICH trigger caused it, and consulted before scheduling (or even attempting) another capture once past MAX_PREVIEW_POLLS. A new head SHA (new push) should reset the counter — it's a genuinely new capture target.

Alternative shape (simpler, if sufficient): fold the visual-preview capture attempt into whatever existing per-(repo, PR, headSha) re-review cache/cooldown key reReviewStoredPullRequest already threads through unrelated caching), so it naturally caps regardless of trigger.

Either way, once the durable cap is hit, buildCapture should stop attempting the "after" render entirely for that head SHA (still may serve the last-known placeholder/dash) rather than silently retrying forever.

Deliverables

  • A capture attempt via ANY trigger (self-poll, CI-completion, deployment_status, sweep) counts against the SAME budget for a given (repo, PR, headSha)
  • Once the budget is exhausted, no further capture attempt fires for that head SHA — no more re-edits driven by the visual-preview mechanism specifically
  • A new push (new headSha) resets the budget
  • Tests: simulate the exact fix(ui): apply the pre-2000 placeholder rule in normalizeFreshnessSources metagraphed#6036 shape — repeated CI-completion + deployment_status triggers on a head with no discoverable preview — assert capture attempts stop at the cap, not per-trigger-type
  • Structured log line when the budget is exhausted (so this is observable going forward, not silently discovered again)

Resources

Boundaries

This is about the RETRY budget, not the decision of whether to capture at all (that's the companion isVisualPath issue in this milestone) or the image pipeline itself. Maintainer-only — assigned JSONbored.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions