Skip to content

fix(review): gate every AI-spend call site on the repo's paused/frozen state - #4420

Merged
JSONbored merged 1 commit into
mainfrom
claude/fix-ai-spend-freeze-gate
Jul 9, 2026
Merged

fix(review): gate every AI-spend call site on the repo's paused/frozen state#4420
JSONbored merged 1 commit into
mainfrom
claude/fix-ai-spend-freeze-gate

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Root cause of a real, confirmed financial incident: resolveRepoActionMode() is already computed once per pass in maybePublishPrPublicSurface and correctly threaded into every GitHub-write call (labels, comments, check-runs) — but was never threaded into the functions that actually spend real tokens: runAiReviewForAdvisory, runAiSlopForAdvisory, runLinkedIssueSatisfactionForAdvisory, runVisualVisionForAdvisory, plus the maintainer-invoked agent-run summary (attachPrivateAiSummary). Each was gated only by its own independent feature flag (aiReviewMode, slop.aiAdvisory, linkedIssueSatisfactionGateMode), completely orthogonal to the fleet-wide env brake, the DB freeze, and per-repo pause.
  • Confirmed live: reactivating a repo with a large stale PR backlog let already-queued and newly-triggered review jobs keep spending for as long as the backlog took to drain — even after immediately re-engaging the freeze override. Two repos burned through 500 fresh AI-review/slop calls across 98 distinct PRs over several hours before the queue emptied, because the freeze stopped new sweep dispatches but never stopped an already-dequeued job from reaching the actual LLM call.
  • Fix: thread the already-resolved mode into all five spend paths, with mode === "paused" as the very first check in each — gating at the point of spend, not just the point of dispatch, so a freeze that engages between enqueue and execution is still honored. "dry_run" still computes (so a maintainer can validate review decisions locally without publishing); only "paused" stops spend outright.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused on one concern (AI-spend freeze gating), though it touches many call sites + their tests since the same required field was added consistently everywhere the audit found a gap.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves — owner-authored PR fixing a live-confirmed financial incident; no separate issue was filed for this emergency follow-up fix.

Validation

  • npm run typecheck
  • npx vitest run test/unit/ai-review-advisory.test.ts test/unit/ai-slop.test.ts test/unit/linked-issue-satisfaction-run.test.ts test/unit/visual-vision-wiring.test.ts test/unit/rag-wiring.test.ts test/unit/repo-culture-profile-wiring.test.ts test/unit/reputation-wiring.test.ts test/unit/enrichment-wiring.test.ts test/unit/grounding-wiring.test.ts test/unit/impact-map-processor-wiring.test.ts test/unit/advisory-ai-routing-call-sites.test.ts test/unit/repository-settings-enforcement.test.ts test/unit/agent-orchestrator.test.ts — 279/279 pass, including 5 new dedicated regression tests (one per spend path) proving a "paused" mode never reaches the LLM/vision provider call even when every feature-specific flag would otherwise allow it.
  • npx vitest run test/unit/queue.test.ts — full suite, 754/754 pass (this is the primary integration suite exercising the changed processors.ts call sites end-to-end).
  • npm run actionlint / npm run test:workers / npm run build:mcp / npm run test:mcp-pack / npm run ui:openapi:check / npm run ui:lint / npm run ui:typecheck / npm run ui:build / npm audit — not run locally for this focused change; relying on CI (validate) for the full gate.

If any required check was skipped, explain why:

  • This is a backend-only logic fix (no schema/API/UI surface); the full npm run test:ci gate is left to CI per the repo's own established practice for this size of change.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. — N/A, no auth/session surface touched.
  • API/OpenAPI/MCP behavior is updated and tested where needed. — N/A, internal spend-gating logic only, no external surface.
  • UI changes use live API data or real empty/error/loading states. — N/A, no UI changes.
  • Visible UI changes include a UI Evidence section. — N/A, no UI changes.
  • Public docs/changelogs are updated where needed. — N/A, internal automation behavior only.

Notes

  • Root-caused live during an active incident: two repos (recently reactivated from an extended fleet-wide freeze) burned through real AI provider spend for hours despite the operator repeatedly re-engaging the freeze override, because the freeze mechanism only ever gated autonomous GitHub actions, never the AI compute itself.
  • A few complementary hardening items surfaced during the investigation but are intentionally left for a follow-up (to keep this PR focused on the confirmed root cause): decoupling the auto_pause_after_reviewed_commits cost counter from GitHub-publish success (it currently undercounts when a publish step is slow/rate-limited), and a coarser per-(repo, PR) minimum-reinterval throttle spanning all three independent trigger sources (scheduled sweep, CI-completion webhook, live push webhook).

…n state

resolveRepoActionMode() is already computed once per pass in
maybePublishPrPublicSurface and correctly threaded into every GitHub-write call
(labels, comments, check-runs) -- but never into the four functions that actually
spend real tokens (runAiReviewForAdvisory, runAiSlopForAdvisory,
runLinkedIssueSatisfactionForAdvisory, runVisualVisionForAdvisory) or the
maintainer-invoked agent-run summary (attachPrivateAiSummary). Each of those was
gated only by its own independent feature flag (aiReviewMode, slop.aiAdvisory,
linkedIssueSatisfactionGateMode), completely orthogonal to the fleet-wide env
brake, the DB freeze, and per-repo pause.

This meant reactivating a repo with a large stale PR backlog (or simply forgetting
one of several independent settings while trying to pause) let already-queued and
newly-triggered review jobs keep spending for as long as the backlog took to drain
-- confirmed live: two repos burned through 500 fresh AI-review/slop calls across
98 distinct PRs over several hours before the queue emptied, because re-engaging
the freeze override alone did not stop already-dispatched work from reaching the
LLM call once dequeued.

Threads the mode already resolved by the caller into all five spend paths, with
"paused" as the first check in each -- gating at the point of spend rather than
only at the point of dispatch, so a freeze that engages between enqueue and
execution is still honored. "dry_run" still computes (so a maintainer can validate
review decisions locally); only "paused" stops spend outright.
@JSONbored
JSONbored merged commit e16a625 into main Jul 9, 2026
7 of 8 checks passed
@JSONbored
JSONbored deleted the claude/fix-ai-spend-freeze-gate branch July 9, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant