fix(review): bound AI review re-spend and public-surface republish on unchanged heads - #2667
Conversation
… unchanged heads Scheduled re-gate sweeps were re-spending a full AI review on every pass for a PR whose outcome landed in a non-cacheable state (consensus defect / inconclusive / a dynamic-context repo with grounding or RAG enabled) — the durable ai_review_cache correctly never stores those outcomes, so nothing throttled the retry. Root-caused in production: one PR generated 281 AI review calls in 24h at an unchanged head, ~92% of it from the RAG-active unconditional-bypass path. - ai_review_cache gains a `cacheable` column; a non-cacheable outcome (and a dynamic-context result, now bounded rather than unconditionally bypassing the cache) is still persisted for a 30-minute cooldown reuse, never as a durable hit. A lock-contention placeholder is still never persisted at all. - New audit events + counters: ai_review_cache_hit/miss/write_error, ai_review_non_cacheable, agent.sweep.regate_ai_skipped_current, github_app.public_surface_publish_skipped_current. A cache write failure is now observable instead of a silent catch. - A narrow public-surface no-op guard skips republishing a check-run-only repo's completed check when nothing provably changed since the last pass (head match + a live-verified completed check run + no pending refresh signal), falling through to a full republish on any doubt. - `agent-regate-pr` jobs carry an optional `force` flag that bypasses both the cache and the cooldown for an explicit manual re-gate. Validated against production Postgres audit_events/ai_usage_events data for the incident repo/PR before and during the fix.
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review result - manual review recommendedReview updated: 2026-07-03 02:39:20 UTC
⏸️ Suggested Action - Manual Review
Review summary Nits — 5 non-blocking
Concerns raised — review before merging
Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2667 +/- ##
=======================================
Coverage 96.10% 96.11%
=======================================
Files 237 237
Lines 26540 26582 +42
Branches 9625 9640 +15
=======================================
+ Hits 25507 25549 +42
Misses 424 424
Partials 609 609
🚀 New features to boost your workflow:
|
…ypass telemetry The new test fixtures reused the pre-existing "installation-token" literal verbatim; since this is the first time those specific lines appear as new diff content, the scanner flags it the same way #2639 already worked around this exact false positive — rename the new occurrences to the established "fake-installation-token" convention. Also: a forced re-gate bypass was being counted under the cache-miss metric/audit, conflating "the cache had nothing to serve" with "a caller explicitly opted out" — split it into its own gittensory_ai_review_force_bypass_total counter and github_app.ai_review_force_bypass audit event, and tighten the stale forceAiReview comment.
…boot (#2686) * fix(review): add the missing securityFocus field to 5 AI-review-cache test fixtures #2675 (feat(review): add a security-focused review profile toggle) added securityFocus as a required AiReviewCacheInput field after these fixtures (from #2667) were written, breaking npm run typecheck on main for anyone branching fresh off it. * fix(selfhost): reject known-placeholder and weak critical secrets at boot .env.selfhost.example shipped ENABLED (not commented-out) placeholder values for GITHUB_WEBHOOK_SECRET, GITTENSORY_API_TOKEN, GITTENSORY_MCP_TOKEN, INTERNAL_JOB_TOKEN, and SELFHOST_SETUP_TOKEN. An operator who copies the starter to .env per the quickstart docs and misses "fill in the placeholders" runs an instance with a PUBLICLY KNOWN webhook HMAC secret (forgeable signatures) and PUBLICLY KNOWN static bearer tokens -- GITTENSORY_API_TOKEN authenticates as the server-to-server actor and bypasses per-repo write checks, INTERNAL_JOB_TOKEN gates internal routes -- silently, with no error at boot or runtime. - The boot-time preflight check (already gates server.ts's main(), throwing before the process starts serving) now rejects any of the five critical secrets that is set to the exact known-placeholder string, or that is merely too short to be a real generated secret, or that duplicates another critical secret's value. Presence is still each secret's own concern (most are feature-gating, not universally required) -- this only judges STRENGTH whenever one is actually set, so it can never be silently bypassed by leaving the file's placeholder in place. - .env.selfhost.example now ships these five lines commented out, with explicit per-secret generation guidance, instead of enabled placeholders. - The quickstart doc callout it directed users through now explicitly warns about generating distinct random values for each secret. Defense in depth: the docs + example file guide an operator toward doing the right thing, and the preflight check makes doing the wrong thing impossible rather than merely discouraged.
Summary
Scheduled re-gate sweeps were repeatedly re-spending a full AI review (and republishing the public surface) on the same PR/head even though nothing had changed, on low-activity repos. Production evidence for one 6h window on one repo: 97
agent.sweep.regateevents, 225github_app.pr_public_surface_publishedevents, 184ai_review_prusage events — for "the same few open PRs ... despite virtually no repo activity."Root cause (confirmed against production Postgres data, not just code inspection): the durable
ai_review_cache(keyed on repo+PR+head SHA+mode) is correct and was being consulted, but two things bypassed it entirely with no throttle:The prior fix in this repo (#2639,
isReviewsCacheUpToDate) addressed a different cache — the GitHub-reviews-data cache, not this AI-review-result cache — so it did not touch this behavior.Fix
ai_review_cachegains acacheablecolumn (migration0098). A non-cacheable outcome is now persisted but marked non-durable, reusable only within a bounded 30-minute cooldown (AI_REVIEW_NON_CACHEABLE_RETRY_COOLDOWN_MS) — collapsing a sweep tick's worth of redundant LLM calls into one, while still periodically retrying (an LLM's own non-determinism, or a maintainer fix, may resolve the dispute) and never treating the result as durably trustworthy.persistable: false) — it's a transient scheduling artifact, not a real verdict, and replaying it after the concurrent pass finished would be actively wrong..catch(() => undefined)is replaced with an audit event (github_app.ai_review_cache_write_error) + counter, instead of a silent no-op.ai_review_cache_hit/ai_review_cache_miss/ai_review_cache_write_error/ai_review_non_cacheable,agent.sweep.regate_ai_skipped_current,github_app.public_surface_publish_skipped_current.publicSurface: "off"), skip republishing when the head matches the last published surface and an independently-verified completed check run already exists at that exact head and the AI dimension was reused, not fresh.markPullRequestSurfacePublished's own doc comment warns the stored marker alone is "reporting/diagnostic state, not a hard scheduled-sweep skip" (a comment can be stale/partial even when it matches) — so this guard is scoped to the one surface (the check run) GitHub itself gives an authoritative signal for, and any doubt falls through to a full republish. Comment/label republishing is intentionally left untouched.agent-regate-prjobs carry an optionalforceflag (unused by any current caller) that bypasses both the cache and the cooldown, so a future manual re-gate trigger has a supported way to force a fresh opinion.Scope
wantedPaths(src/,test/,migrations/)site/,CNAME,**/lovable/**, orCHANGELOG.mdgetCachedAiReview/putCachedAiReview(src/db/repositories.ts), thegithub_app.miner_detection_cache_hit/missaudit-event pair, andsurfaceRepairPriorityPullNumbers's existing completed-check-run verification patternValidation
npm run typechecknpm run db:migrations:checknpm run test:coverage(unsharded) — 100% line+branch coverage on every changed line insrc/db/repositories.tsandsrc/queue/processors.ts, verified by diffing changed lines against the v8 coverage map directlynpm run test:workersnpm run ui:lint/ui:test(unaffected, no UI changes)npm audit --audit-level=moderate— 0 vulnerabilitiesnpm run actionlint,cf-typegen:check,selfhost:env-reference:check,build:mcp,test:mcp-pack,ui:openapi:check,ui:openapi:settings-parity,ui:version-auditaudit_events,ai_usage_events,ai_review_cache) on the self-host VPS before writing the fix, to confirm the actual root cause rather than guessing from code aloneNew/updated tests (
test/unit/ai-review-cache.test.ts,test/unit/queue.test.ts, tagged#regate-churn):audit_eventsand counters, not silently swallowedforcere-gate bypasses the cache and cooldownSafety