Skip to content

feat(review): add durable webhook-invalidated CI-state snapshot cache - #2984

Merged
JSONbored merged 2 commits into
mainfrom
claude/ci-state-snapshot-cache
Jul 4, 2026
Merged

feat(review): add durable webhook-invalidated CI-state snapshot cache#2984
JSONbored merged 2 commits into
mainfrom
claude/ci-state-snapshot-cache

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

PR #2847 ("prioritize live PR work under GitHub rate pressure") explicitly deferred this as needing "a dedicated design pass": every readiness check and disposition pass does a fresh REST/GraphQL CI read (check-runs/status/check-suites), even when the same (repo, PR, head_sha) was already read moments earlier in the same or a prior job — the VPS evidence that originally motivated the fix#3 work showed ~1,720 check-runs reads and ~1,211 status reads in a 2-hour window.

This extends the existing durable pull_request_detail_sync_state cache (#2537's PR-state cache) with 10 CI-state columns, following the exact same pattern: a fresh row (60s TTL, invalidated on a check_run/check_suite completed webhook for the matching head SHA) is served without a GitHub call; a miss falls through to a live fetch and write-throughs the result for the next reader. status/workflow_run events are explicitly not wired to invalidation today (neither is handled for re-review triggering either) — the 60s TTL is the sole backstop for those two event types, documented with a test rather than left silently uncovered.

The act-boundary re-check (refreshLiveCiAggregate, used right before a merge/close decision) always forces a genuinely live read regardless of cache state via a forceRefresh flag threaded through the call chain — mirroring the pre-existing refreshLiveMergeState "never durable-cached" contract exactly. The two services that call the raw CI-aggregate fetcher directly at the actual act boundary (agent-approval-queue.ts, agent-action-executor.ts) are unaffected — they never import the new cached path.

Generic self-host engine behavior only; no repo-specific logic. Follow-up to #2970 (per-installation GitHub-fetch concurrency admission), the other item PR #2847 deferred — independent files, independent risk profile, no dependency between them.

Scope

Validation

  • git diff --check
  • npm run actionlint (not run — no workflow files touched)
  • npm run typecheck
  • npm run test:coverage locally (unsharded); precise diff-coverage cross-reference against coverage/coverage-final.json confirms 100% branch coverage on every changed line across all touched src/** files
  • npm run test:workers (not run — no Cloudflare-Workers-pool-specific code touched)
  • npm run build:mcp / npm run test:mcp-pack (not run — no MCP package changes)
  • npm run ui:openapi:check / npm run ui:lint / npm run ui:typecheck / npm run ui:buildui:openapi:check was run and passes (no API/OpenAPI surface changed); the others not run — no apps/gittensory-ui source changes, only the generated env-reference file was regenerated
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries
  • npm run selfhost:env-reference:check (regenerated; stale only because origin/main moved since PR feat(selfhost): add per-installation GitHub-fetch concurrency admission #2970, unrelated to this PR's own changes)
  • npm run db:migrations:check (new migration 0108_pull_request_ci_state_cache.sql, confirmed contiguous, next-free 0109) and npm run db:schema-drift:check (59 Drizzle tables OK)

If any required check was skipped, explain why:

  • Skipped checks above are for UI/MCP/Workers surfaces this PR does not touch (backend/DB-only change to the CI-read cache path); CI's validate job runs them as a backstop.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — no auth/session/CORS changes.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — no API/OpenAPI/MCP surface changed; this is an internal cache, not exposed.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (N/A — no UI changes.)
  • Visible UI changes include a UI Evidence section below with screenshots. (N/A — no visible UI changes.)
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. (N/A.)

Notes

@JSONbored JSONbored self-assigned this Jul 4, 2026
@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 4, 2026
@loopover-orb

loopover-orb Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-04 09:28:09 UTC

9 files · 1 blocker · readiness 75/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Possible leaked secret in the diff (generic_secret_assignment) — Remove the secret from the diff, rotate the exposed credential, then re-run the gate.

Review summary
AI review is already running for this PR head in another Gittensory pass. Gittensory is holding this PR for manual review until that pass completes.

Nits — 1 non-blocking
  • AI review already in progress for this PR head — The gate is held for a human reviewer rather than passed automatically; it re-evaluates once the in-flight review completes or on the next update.

Concerns raised — review before merging

  • Possible leaked secret in the diff (generic_secret_assignment) — Remove the secret from the diff, rotate the exposed credential, then re-run the gate.
Signal Result Evidence
Code review ❌ 1 blocker No AI review summary
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ⚠️ 1 scoped overlap Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 60 registered-repo PR(s), 51 merged, 421 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 60 PR(s), 421 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 60 PR(s), 421 issue(s).
  • Related work: Titles/paths share 5 meaningful terms. (issue #1681, issue #1683)
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Review top overlaps.
  • Add a concise scope and risk note.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.08%. Comparing base (1516180) to head (69bc6d6).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2984      +/-   ##
==========================================
+ Coverage   96.06%   96.08%   +0.01%     
==========================================
  Files         260      260              
  Lines       28689    28750      +61     
  Branches    10441    10471      +30     
==========================================
+ Hits        27560    27623      +63     
+ Misses        493      492       -1     
+ Partials      636      635       -1     
Files with missing lines Coverage Δ
src/db/repositories.ts 96.33% <100.00%> (+<0.01%) ⬆️
src/db/schema.ts 69.46% <ø> (ø)
src/github/backfill.ts 96.74% <100.00%> (+0.07%) ⬆️
src/queue/processors.ts 92.92% <100.00%> (+0.17%) ⬆️
src/selfhost/metrics.ts 100.00% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored force-pushed the claude/ci-state-snapshot-cache branch from e0a1c5e to 1fa1a66 Compare July 4, 2026 09:02
JSONbored added 2 commits July 4, 2026 02:22
check_run/check_suite completions currently force a fresh REST/GraphQL CI
read on every readiness check and disposition pass, even when the same
(repo, PR, head_sha) was already read moments earlier in the same or a
prior job. Extend the existing pull_request_detail_sync_state cache
(#2537) with CI-state columns: a fresh row (TTL-capped, invalidated on
check_run/check_suite completed) is served without a GitHub call; a miss
falls through to a live fetch and write-throughs the result. The
act-boundary re-check (refreshLiveCiAggregate) always forces a live read
regardless of cache state, preserving its existing always-fresh contract.
…te on legacy CI events

Two correctness gaps in the durable CI-state cache:

1. cachedFetchLiveCiAggregate persisted the live aggregate under the
   normal cache key even when cachedRequiredStatusContexts's lookup
   had failed (fail-open null required contexts), so a transient
   branch-protection read error could mask a repo's real
   required-context state for every reader until TTL expiry. Track
   whether the lookup actually resolved and skip the durable
   write-through when it didn't -- the live-fetched aggregate is still
   used for the current pass's own decision either way.

2. status/workflow_run webhook events weren't wired to invalidate the
   cache at all (only check_run/check_suite were), so a real legacy CI
   transition could leave prReadyForReview reading a stale
   pre-transition aggregate for up to the full TTL. Add a narrower
   invalidation-only handler for these two event types (re-review
   triggering stays out of scope, per the existing MVP boundary) that
   resolves affected PRs via the fast stored-DB head-SHA lookup and
   invalidates their cache entries.

Both fixes verified by temporarily reintroducing the old behavior and
confirming the new regression tests fail against it.
@JSONbored
JSONbored force-pushed the claude/ci-state-snapshot-cache branch from 1fa1a66 to 69bc6d6 Compare July 4, 2026 09:24
@JSONbored
JSONbored merged commit d9207d2 into main Jul 4, 2026
12 checks passed
@JSONbored
JSONbored deleted the claude/ci-state-snapshot-cache branch July 4, 2026 09:32
JSONbored added a commit that referenced this pull request Jul 4, 2026
…exts (#3036)

cachedFetchLiveCiAggregate (#2984) keyed its durable
pull_request_detail_sync_state row from the raw, unresolved
settings.expectedCiContexts config instead of the actually-resolved
required-contexts set that cachedRequiredStatusContexts returns (branch
protection merged with config, via mergeRequiredCiContexts). If branch
protection changes server-side while expectedCiContexts config and head_sha
stay the same, the durable key was unchanged, so the readiness path could
keep serving a stale aggregate computed against the old required-context set
for up to the cache's TTL. Derive the key from the resolved set instead.

Also hardens deserializeCachedCiAggregate: JSON.parse succeeding does not
guarantee the parsed value is an array, so a corrupted/malformed row's
failingDetails/nonRequiredFailingDetails now fail open to a cache miss via an
explicit Array.isArray check rather than handing callers a wrong shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Development

Successfully merging this pull request may close these issues.

1 participant