fix(review): cache the AI slop advisory result per head SHA by JSONbored · Pull Request #4025 · JSONbored/loopover

JSONbored · 2026-07-07T10:48:21Z

Summary

runGittensoryAiSlopAdvisory made a real LLM call (up to 6 free-tier attempts, or a BYOK call) on every gate pass with zero caching, so the scheduled re-gate sweep re-spent it on every tick even at an unchanged head. Confirmed with a direct production query: 1,469 ai_slop_pr calls in the last 24 hours across our 3 registered repos, with 110 calls on a single PR alone.

This adds ai_slop_cache, keyed on (repo_full_name, pull_number, head_sha), following the shape of the existing ai_review_cache (see getCachedAiReview/putCachedAiReview in src/db/repositories.ts) but deliberately simpler: the slop advisory has no dynamic-context dimension the way AI review does (no RAG/grounding/enrichment feeds into it — its only inputs are the PR title/body/diff/deterministic band, all of which are already pinned to the head SHA) and nothing analogous to a "published" GitHub artifact to protect against replaying (its output folds into the same advisory pass that computes it, never stamped separately). So a cache hit here is unconditionally durable for a given head — no cacheable/published_at cooldown columns, unlike ai_review_cache.

The cache is fingerprinted on the one input that CAN change independently of the head SHA: which provider would answer (free/default reviewer vs. a repo's BYOK key/model), so toggling BYOK or changing its provider/model correctly misses a stale cache row instead of replaying an opinion from a different reviewer.

Observability mirrors the ai_review_cache house style: gittensory_ai_slop_cache_hit_total / _miss_total / _write_error_total metrics, and github_app.ai_slop_cache_hit / _miss / _write_error audit events.

No issue because: this is a small, self-evident production bug fix backed by direct production data (linkedIssuePolicy on this repo is preferred, not required, and this fix is narrowly scoped to one call site).

Operational follow-up (not part of this PR): gate.slop.aiAdvisory was disabled as an emergency stopgap on the self-hosted VPS config for all 3 repos while this bug was live. Once this PR is merged and deployed, it should be re-enabled — that's a VPS config change outside this PR's scope.

Scope

The PR title follows type(scope): short summary Conventional Commit format.
This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
I linked a currently open issue this PR resolves — no open issue; see the no-issue rationale above.

Validation

Safety

No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics (unchanged — this PR only changes when the existing advisory is computed vs. reused, not its content).
Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. N/A — no auth/session/CORS surface touched.
API/OpenAPI/MCP behavior is updated and tested where needed. N/A — no API routes changed; ui:openapi:check confirms no drift.
UI changes use live API data or real empty/error/loading states. N/A — backend-only change.
Visible UI changes include a UI Evidence section. N/A — no visible/UI change.
Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. No changelog edit (not a release-prep PR).

Notes

Migration 0119_ai_slop_cache.sql re-verified as the correct next-free number immediately before every push, given very active concurrent development on this repo (main moved several times over the course of this change).

superagent-security · 2026-07-07T10:49:19Z

Superagent didn't find any vulnerabilities or security issues in this PR.

codecov · 2026-07-07T10:55:37Z

❌ 51 Tests Failed:

Tests completed	Failed	Passed	Skipped
10564	51	10513	7

View the top 3 failed test(s) by shortest run time

test/unit/mcp-server-telemetry.test.ts > MCP server telemetry > falls back when Hono does not expose an execution context

Stack Traces | 0.012s run time

Error: Failed to resolve entry for package "@jsonbored/gittensory-engine". The package may have incorrect main/module/exports specified in its package.json.
 ❯ .../gittensory-miner/lib/opportunity-fanout.js:2:1
 ❯ src/mcp/find-opportunities.ts:11:1

test/unit/mcp-server-telemetry.test.ts > MCP server telemetry > records session-scoped MCP request errors without a tool name

Stack Traces | 0.0238s run time

Error: Failed to resolve entry for package "@jsonbored/gittensory-engine". The package may have incorrect main/module/exports specified in its package.json.
 ❯ .../gittensory-miner/lib/opportunity-fanout.js:2:1
 ❯ src/mcp/find-opportunities.ts:11:1

test/unit/mcp-server-telemetry.test.ts > MCP server telemetry > records sanitized error telemetry when the MCP transport handler throws

Stack Traces | 0.25s run time

Error: Failed to resolve entry for package "@jsonbored/gittensory-engine". The package may have incorrect main/module/exports specified in its package.json.
 ❯ .../gittensory-miner/lib/opportunity-fanout.js:2:1
 ❯ src/mcp/find-opportunities.ts:11:1

View the full list of 48 ❄️ flaky test(s)

test/integration/api.test.ts > test/integration/api.test.ts