fix(selfhost): backpressure scheduled regate sweeps

[JSONbored]

Summary

• add self-host queue snapshot introspection for pending, processing, and dead jobs
• skip scheduled re-gate and backfill fanout while prior re-gate work is still queued
• lower scheduled re-gate batch size so the source workload leaves REST budget headroom

What changed

• Exposes snapshot() on the SQLite and Postgres self-host queues.
• Adds shared queue snapshot aggregation and backlog helpers.
• Uses queue introspection in cron scheduling to avoid adding new scheduled sweep/backfill work into an existing re-gate backlog.
• Uses queue introspection in repo-scoped scheduled sweeps so already-queued sweep jobs do not fan out more per-PR re-gates while prior scheduled work is still pending or processing.
• Keeps fail-open behavior if queue introspection is unavailable, so maintenance does not permanently stall.
• Reduces SWEEP_MAX_PRS from 6 to 3 to cut scheduled REST demand at the source.

Why

The self-host queue can recover from rate-limit cooldowns and immediately rebuild a large scheduled re-gate backlog. That backlog competes with live webhook reviews and can push the installation REST bucket back into exhaustion. This change makes scheduled work observe the queue before adding more re-gate pressure, while preserving live webhook processing and fail-open behavior when introspection is missing.

Validation

• npx vitest run test/unit/queue.test.ts -t "queue introspection throws|prior regate work"
• npm run test:ci
• npm audit --audit-level=moderate

Notes

• npm run test:ci completed successfully, including coverage, workers, MCP packaging, REES tests, OpenAPI checks, UI lint/typecheck/tests, and UI build.
• npm audit --audit-level=moderate found 0 vulnerabilities.

[loopover-orb]

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

_{Review updated: 2026-06-30 23:46:57 UTC}

12 files · 1 AI reviewer · no blockers · readiness 93/100 · CI pending · unknown

⏸️ Suggested Action - Manual Review

• Touches a guarded path — held for manual review

Review summary
The change adds self-host queue snapshot introspection and uses it to backpressure scheduled re-gate sweep/backfill work while letting repo-scoped sweep jobs fan out unless per-PR re-gate work is already queued. The split between src/index.ts counting both sweep and per-PR rows, and src/queue/processors.ts counting only per-PR rows, is intentional and covered by the new tests. Fail-open behavior is preserved when snapshot introspection is missing or throws, and the lower sweep cap aligns with the stated REST-budget goal.

Nits — 5 non-blocking

• nit: src/selfhost/pg-queue.ts:577 should have a real-shape test for the actual pg driver payload/run_after values, because the current mock only exercises string payloads and string timestamps.
• nit: src/queue/processors.ts:1025 silently fails open on snapshot errors, while src/index.ts:109 logs the same condition; consider logging this path too so repo-scoped defer failures are observable.
• src/selfhost/queue-common.ts:113 could accept object payloads with a string type field in addition to JSON strings, which would make the snapshot helper robust across drivers that deserialize JSON columns.
• src/selfhost/pg-queue.ts:577 should add a test using the same payload/run_after shapes returned by the real pg type parser, or normalize those values before passing them into buildSelfHostQueueSnapshot.
• Touches a guarded path — held for manual review — A maintainer must review and merge this change.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ No-issue rationale	PR body explains why no issue is linked.
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (size label size:L; no linked issue context).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 68 registered-repo PR(s), 58 merged, 242 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 68 PR(s), 242 issue(s).
Gate result	⚠️ Not blocking	Advisory; not blocking this PR.

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository registration is not available in the local Gittensory cache.
• Public profile languages: not available
• Official Gittensor activity: 68 PR(s), 242 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Treat this as maintainer-lane context rather than normal contributor-lane activity.
• Triage stale or unlinked PRs.
• No action.
• Link the issue being solved, or explicitly explain why this is a no-issue PR.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[codecov]

Codecov Report

❌ Patch coverage is 96.07843% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.55%. Comparing base (2bdeb19) to head (aa79af0).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/index.ts	90.90%	0 Missing and 1 partial ⚠️
src/selfhost/queue-common.ts	96.66%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1933   +/-   ##
=======================================
  Coverage   95.55%   95.55%           
=======================================
  Files         218      218           
  Lines       24195    24243   +48     
  Branches     8772     8791   +19     
=======================================
+ Hits        23119    23165   +46     
  Misses        438      438           
- Partials      638      640    +2     

Files with missing lines	Coverage Δ
src/queue/processors.ts	89.36% <100.00%> (+0.04%)	⬆️
src/selfhost/sqlite-queue.ts	99.19% <100.00%> (+<0.01%)	⬆️
src/settings/agent-sweep.ts	100.00% <100.00%> (ø)
src/index.ts	94.52% <90.90%> (-0.72%)	⬇️
src/selfhost/queue-common.ts	90.98% <96.66%> (+0.75%)	⬆️

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.