Skip to content

orb(cost): a failed AI review caches nothing, so the entire expensive prologue re-runs every 2 minutes forever #9060

Description

@JSONbored

The #regate-churn fix bounded re-spend for disputed verdicts, but it only writes a cache row when runAiReviewForAdvisory returns something defined:

if (aiReview && aiReview.persistable !== false) { ... putCachedAiReview(...) }   // processors.ts ~10422

Both failure exits return undefined: if (result.status !== "ok") return undefined; (ai-review-orchestration.ts ~729) and the catch block (~935).

So any pass that throws inside the try (a DB hiccup in getRepository, a GitHub 5xx during grounding, an enrichment/REES failure) or returns non-ok writes no row at all. The next tick — 2 minutes later (wrangler.jsonc crons */2 * * * *; self-host setInterval default 120s) — misses the cache and re-executes the whole prologue: listPullRequestFiles → grounding (up to FILE_CONTENT_BUDGET = 96_000 chars fetched from GitHub) → RAG embeddings → impact-map embeddings → culture profile → the external REES POST → the LLM attempt.

This is the exact shape and cadence of the known 259-calls-in-24h incident, and the existing fix does not cover it.

Compounding: the budget ceiling sits AFTER the spend it bounds

AI_DAILY_NEURON_BUDGET is checked inside runLoopOverAiReview (src/services/ai-review.ts ~2504-2522), which is called at ai-review-orchestration.ts ~670 — after grounding, RAG, impact map, culture profile, and enrichment have all run (~549-662).

So on budget exhaustion: quota_exceeded → non-ok → no cache row → next tick re-runs all the context building again, then hits the quota again. And because every embedding is booked with estimatedNeurons: 0 (src/review/adapters.ts ~80, ~93), none of that spend increments the governor's counter — the ceiling can never converge and the loop never self-limits.

Fix

  1. Persist a non-durable "attempt failed" row (cacheable: false, persistable: true) on the undefined paths, keyed on the same head+fingerprint, so the 30-minute cooldown applies to failures too.
  2. Hoist the budget/quota check to the top of runAiReviewForAdvisory, before any context building.
  3. Charge embeddings a real non-zero estimate.

Acceptance

  • A PR whose AI review fails repeatedly costs one prologue per cooldown window, not one per 2 minutes.
  • With the budget exhausted, no grounding/RAG/enrichment work is performed at all.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions