Skip to content

perf(review): cache the AI review by (repo, pr, head SHA) to skip redundant LLM calls - #1462

Merged
JSONbored merged 1 commit into
mainfrom
claude/ai-review-cache
Jun 26, 2026
Merged

perf(review): cache the AI review by (repo, pr, head SHA) to skip redundant LLM calls#1462
JSONbored merged 1 commit into
mainfrom
claude/ai-review-cache

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

The single biggest self-host compute win: cache the AI review by (repo, pull, head SHA). On self-host
there's no Cloudflare AI Gateway, so every re-trigger re-spends the full LLM review. In particular, in
block mode the ~2-minute re-gate sweep re-runs the AI for every open PR (reReviewStoredPullRequest passes
skipAiReview: false in block mode) — re-reviewing identical code on a loop.

The expensive runAiReviewForAdvisory is now wrapped at its single call site with a commit-keyed cache:

  • Hit (same repo+pr+head SHA, same review mode) → reuse the stored {notes, reviewerCount}, no LLM call.
  • Miss → run the review, then store it.
  • Invalidation: a new head SHA (new code) is a new key; a changed review mode (block↔advisory materially
    changes the prompt) is treated as a miss. The deterministic gate still re-evaluates every time — only the
    AI leg is reused, so CI/mergeability changes are never served stale.

getCachedAiReview / putCachedAiReview (repositories.ts) handle the nullish-head-SHA
case internally (so the call site has no dead branches), backed by a new ai_review_cache table (migration 0074).

Scope

  • src/** + migration + tests; the deterministic gate path is unchanged.
  • No secrets; parameterized SQL only.

Validation

  • npm run typecheck, npm run db:migrations:check (contiguous 0001..0074)
  • npm run test:coverage — full suite green; 100% changed lines + branches. Unit tests cover the SHA/mode/nullish matrix; an integration test proves the block-mode sweep reuses the cache with zero AI calls.

Roadmap item #1 (highest-value perf win).

…undant LLM calls

On self-host there is no AI gateway, so every re-trigger re-spends the LLM. The
expensive runAiReviewForAdvisory is now cached by (repo, pull, head SHA) and reused
when the review mode matches — a re-delivered webhook or the block-mode ~2-minute
re-gate sweep (which re-runs the AI for every open PR) reuses the prior review
instead of re-calling the model. A new head SHA (new code) or a changed review mode
invalidates the entry; the deterministic gate still re-evaluates every time.
@dosubot dosubot Bot added the size:L label Jun 26, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jun 26, 2026
@JSONbored JSONbored added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jun 26, 2026
@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.44%. Comparing base (8309b23) to head (c7ea780).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1462   +/-   ##
=======================================
  Coverage   95.44%   95.44%           
=======================================
  Files         194      194           
  Lines       21054    21064   +10     
  Branches     7619     7623    +4     
=======================================
+ Hits        20094    20104   +10     
  Misses        383      383           
  Partials      577      577           
Files with missing lines Coverage Δ
src/db/repositories.ts 96.12% <100.00%> (+0.01%) ⬆️
src/queue/processors.ts 88.42% <100.00%> (+0.03%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit 2ffd657 into main Jun 26, 2026
19 checks passed
@JSONbored
JSONbored deleted the claude/ai-review-cache branch June 26, 2026 10:49
JSONbored added a commit that referenced this pull request Jun 26, 2026
)

Both 0074_ai_review_cache (#1462) and 0074_orb_self_enrollment_disabled (#1465) merged
and deployed before the duplicate number surfaced. The latter is a bare ALTER TABLE ADD
COLUMN, which SQLite cannot guard with IF NOT EXISTS, and D1/self-host track applied
migrations by filename — so renaming it now would re-run the ALTER and fail the deploy.
Grandfather the pair (same rationale as the pre-existing 0015/0017 entries) so
db:migrations:check passes on main again; this unblocks validate on every open PR.
JSONbored added a commit that referenced this pull request Jun 26, 2026
… embed stack (#1555)

Reconciles bb90b64e onto current main. Per-repo `features.{rag,reputation,unifiedComment,safety}`
overrides (manifest → GITTENSORY_REVIEW_REPOS allowlist default, byte-identical when unset), the
`aiReviewAllAuthors` opt-in (review every author, not only confirmed contributors), and a dedicated
RAG embed provider (AI_EMBED_BASE_URL) so the review chain stays frontier-only.

Resolution notes: KEPT main's #1462 AI-review cache, #1425 contributor blacklist, and inline-comments
on the review path; DROPPED the batch's head-SHA dedup (ai-review-dedup) as superseded by #1462.
Migration renumbered 0072→0075 (0072 was taken by contributor_blacklist on main).
JSONbored added a commit that referenced this pull request Jun 27, 2026
…oft-fail + idempotent migrations (#1562)

Three self-host reliability fixes:
- SWEEP REGARDLESS OF MODE (#sweep-all-modes): the re-gate sweep now enumerates the GITTENSORY_REVIEW_REPOS
  allowlist UNION the registered repos and sweeps any review-active (allowlisted) repo, not only repos with
  acting autonomy — so ADVISORY/observe repos are re-reviewed and get fresh reviews on existing open PRs
  without a per-PR webhook. The AI review runs for advisory + block (only skipped when AI is off); the #1462
  cache bounds the token cost. The action layer stays autonomy-gated, so observe repos never auto-merge/close.
- REGISTRY SOFT-FAIL (#registry-soft-fail): the self-host consume wrapper swallows a failed periodic
  refresh-registry job (a degraded/unconfigured GITTENSOR_REGISTRY_URL otherwise retried→dead-lettered every
  cron tick, flooding the dead-letter alert). The next scheduled tick is the retry; the Worker path is untouched.
- IDEMPOTENT MIGRATIONS (#migrate-drift): the runner tolerates a duplicate-column/already-exists migration
  (a renumbered file whose schema change a prior deploy already applied) instead of crash-looping the boot.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant