perf(github): dedup + budget-gate the upstream ref→SHA resolve - #1998
Conversation
|
Warning 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏸️ Gittensory review result - manual review recommendedReview updated: 2026-07-01 07:08:03 UTC
⏸️ Suggested Action - Manual Review
Review summary Nits — 5 non-blocking
Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1998 +/- ##
=======================================
Coverage 95.66% 95.66%
=======================================
Files 220 221 +1
Lines 24453 24457 +4
Branches 8868 8872 +4
=======================================
+ Hits 23394 23398 +4
Misses 436 436
Partials 623 623
🚀 New features to boost your workflow:
|
refreshScoringModelSnapshot and the upstream-drift refresh each did their own
GET /repos/{repo}/commits/{ref} to pin the upstream ref to a commit SHA — the
same read, twice per hour, in the same cron window as the heavy fan-out.
Collapse the two near-identical resolvers into one resolveUpstreamCommitSha:
- dedup: it's a bare /commits/{ref} read, now a short-TTL 'commit' response-cache
class, so the second hourly job hits the cache — one read per window, not two;
- budget-gate: it yields (null → caller falls back to the mutable ref) when the
shared REST budget is depleted, so this best-effort resolve never spends a
scarce request during a crunch.
Advances #1936.
25486fb to
cd8e1b1
Compare
|
Fixed the review finding (budget-gate suppressing free cached resolves). The gate is no longer an upfront check; it's passed as a Regression tests added: a cached resolve is served under budget pressure (two resolves, budget depleted on the second → still one fetch), and a cache-miss under pressure still skips the network (no request, null). Existing |
Summary
The upstream ref→SHA pin (
GET /repos/{repo}/commits/{ref}) was fetched twice per hour — once inrefreshScoringModelSnapshot(refresh-scoring-model) and once in the upstream-drift refresh (refresh-upstream-drift) — via two near-identical private resolvers, both firing in the same hourly cron window as the heavy maintenance fan-out.This collapses them into one shared
resolveUpstreamCommitShaand addresses both halves of the audit item (rec #7):/commits/{ref}read (no/check-runs,/status,/pulls, … suffix) is now a short-TTLcommitGitHub-response-cache class. Both jobs issue the identical read throughtimeoutFetch, so the second one within the window is served from the cache (self-host Redis) — one read per window instead of two. Only these two upstream resolves use the bare-/commits/{ref}shape, so nothing else is affected (the mutable CI subresources under/commits/{sha}stay bypassed, as before).null→ the caller falls back to the mutable ref, its existing degraded path) when the shared REST budget is at/below the low-water floor — so this best-effort audit pin never spends a scarce request during a rate-limit crunch.TTL is configurable (
GITHUB_COMMIT_CACHE_TTL_SECONDS, default 15m), documented in.env.examplealongside the sibling cache TTLs.Advances #1936 (self-host rate-limit RC; rec #7 of the #1942 audit). No
Closes— #1942 is already closed and there's no dedicated open issue this fully resolves.Scope
Validation
git diff --checknpm run typechecknpm run test:coverage— newresolveUpstreamCommitShatests (resolve / budget-gate-skips-the-request / end-to-end dedup: 2 resolves → 1 fetch / fail-open) +commitcache-class + TTL assertions; existing scoring-model + upstream-ruleset suites still green.npm run test:cinpm audit --audit-level=moderateSafety
commitclass caches only the bare ref-resolve at a short TTL; a/commits/{sha}SHA is immutable and a branch ref is at most TTL-stale — and the caller already tolerates the fully-mutable-ref fallback.