perf(selfhost): tune Postgres autovacuum and document the observation-write batching decision

[JSONbored]

Summary

Closes #2543.

github_rate_limit_observations receives one INSERT per outbound GitHub API response and is pruned in daily bulk deletes by the retention job — an insert-then-bulk-delete pattern that is exactly the shape known to cause dead-tuple bloat under Postgres's stock autovacuum settings (the codebase already has an alert rule watching for this general symptom, which implies the risk was anticipated but not pre-empted for this specific table). Explicitly framed by the issue as a before-larger-scale item, not an active production problem today.

What changed

• src/selfhost/pg-adapter.ts: new tuneGithubRateLimitObservationsAutovacuum(db), applying ALTER TABLE github_rate_limit_observations SET (autovacuum_vacuum_scale_factor = 0.05, autovacuum_vacuum_threshold = 50) — a lower scale factor than Postgres's 0.2 default, so autovacuum reclaims space promptly after each day's bulk delete instead of letting dead tuples accumulate across cycles. Uses the exact same D1Database.exec() surface runSelfHostMigrations already relies on for migrations, so it reuses the existing translateDdl SQL path rather than a second raw-pool mechanism.
• src/server.ts: calls the tuning step once at boot, immediately after runSelfHostMigrations (the table has to exist first) and gated behind the existing usePostgres check — a complete no-op on SQLite, which has no autovacuum concept at all. Best-effort: a failed tune logs a warning and continues rather than blocking boot, since this is an optimization, never a correctness dependency.
• src/github/backfill.ts: documented, in place at recordGitHubResponse (the single call site of recordGitHubRateLimitObservation), the decision not to implement write batching — see Correctness notes below.

Correctness notes — why batching was evaluated and not implemented

The issue explicitly permits "either an implemented batching layer, or a documented decision that the current per-call INSERT pattern is acceptable... with the reasoning written down." I chose the latter:

• The write rate is bounded by GitHub's own REST budget for a single App installation (~5000/hour ≈ 1.4/s sustained, further capped in practice by QUEUE_CONCURRENCY's small worker-pool size) — nowhere near a volume where single-row Postgres INSERTs meaningfully pressure a connection pool.
• shouldWaitForGitHubRateLimit (src/github/rate-limit.ts) reads the latest row from this exact table for admission control across every self-host queue worker — including in a multi-instance/shared-Postgres deployment, where a buffering instance's writes would go stale to every other instance's reads, not just its own.
• A batching window here would trade a real-but-currently-unmeasured write-volume concern for a genuine risk to the admission-control freshness the chore(selfhost): beta-stable release readiness roadmap #1936 rate-limit-reliability campaign (this entire roadmap) was built to protect: a stale "remaining: 500" observation could let a queue worker admit a job it should have deferred, right when conserving the budget matters most.
• The autovacuum tuning above already addresses the dead-tuple-bloat half of the issue, which is the part that was actually observed/anticipated (an alert rule already exists for it).

Verification

Spun up a real Postgres 16 container (docker run postgres:16) and ran the full test/integration/selfhost-pg.test.ts suite against it (normally CI-skipped, gated on PG_TEST_URL) — all 5 tests pass, including a new one asserting the exact pg_class.reloptions values after applying the tuning twice (idempotency).

Tests

• test/unit/selfhost-pg-adapter-autovacuum.test.ts (new): the SQL constant targets the right table with a scale factor below Postgres's 0.2 default and is a single non-destructive ALTER; tuneGithubRateLimitObservationsAutovacuum calls db.exec with the exact SQL, fails open (does not throw) on a rejected db.exec, logs the underlying error message, and handles a non-Error rejection without throwing on .message access. Mutation-tested: reverting the scale factor to Postgres's 0.2 default correctly failed the "below default" assertion.
• test/integration/selfhost-pg.test.ts: new real-Postgres test verifying the autovacuum storage parameters are actually set on the table and that a second apply is a genuine no-op (not an error).

Validation

• npm run typecheck
• npx vitest run test/unit/selfhost-pg-adapter-autovacuum.test.ts
• PG_TEST_URL=... npx vitest run test/integration/selfhost-pg.test.ts (real Postgres 16, Docker)
• npm run test:changed
• npm run test:coverage (unsharded, full suite — src/selfhost/pg-adapter.ts/src/server.ts are Codecov-ignored per codecov.yml, validated instead by the real-Postgres integration test above; src/github/backfill.ts's change is comment-only)
• npm run db:migrations:check (no-op — no schema/migration file changes; this is a runtime storage-parameter tune, not DDL that needs migration-ledger tracking)
• npm run test:ci (the full local gate, exit 0)
• npm audit --audit-level=moderate
• git diff --check
• Manual mutation testing of the autovacuum SQL constant (see Tests above)

Scope

• Change is narrow and limited to the stated problem
• No secrets, wallets, hotkeys, trust scores, or reward values added anywhere
• No edits to site/, CNAME, **/lovable/**, or CHANGELOG.md

Safety

• No auth/session/CORS surface touched
• No behavior change on the SQLite backend (verified: the new call is gated behind usePostgres, never reached otherwise); no regression to rate-limit admission-control accuracy (no batching was introduced — the write path is unchanged, only a storage parameter on the target table)

[loopover-orb]

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

_{Review updated: 2026-07-02 20:33:07 UTC}

5 files · 1 AI reviewer · no blockers · readiness 98/100 · CI green · clean

⏸️ Suggested Action - Manual Review

• Touches a guarded path — held for manual review

Review summary
This PR adds a Postgres-only boot-time storage-parameter tune for github_rate_limit_observations after migrations and documents why response observations remain single-row writes. The implementation is correctly scoped to self-host Postgres, uses the existing D1 exec path, and the integration test verifies the real reloptions on Postgres. The most notable tradeoff is that the production comments are carrying a lot of issue narrative, but the behavior itself is safe enough to proceed.

Nits — 6 non-blocking

• nit: src/selfhost/pg-adapter.ts:86 and src/github/backfill.ts:3541 have very long rationale comments that mix operational context, issue history, and implementation notes; keeping only the invariant and moving the extended rationale to the issue/docs would make the hot-path files easier to scan.
• nit: test/unit/selfhost-pg-adapter-autovacuum.test.ts:42 restores the console.error spy manually at the end of each test, so a failed assertion before mockRestore would leak the spy into later tests; use afterEach/restoreAllMocks or try/finally.
• src/selfhost/pg-adapter.ts:86: shorten the comment to the durable contract: run after migrations, Postgres only, idempotent, best-effort.
• test/unit/selfhost-pg-adapter-autovacuum.test.ts:42: centralize spy cleanup with vi.restoreAllMocks() in afterEach so failure paths do not affect following tests.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
• Touches a guarded path — held for manual review — A maintainer must review and merge this change.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ Linked	#2543
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (size label size:M; 1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 64 registered-repo PR(s), 55 merged, 526 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 64 PR(s), 526 issue(s).
Gate result	⚠️ Not blocking	Advisory; not blocking this PR.

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository registration is not available in the local Gittensory cache.
• Public profile languages: not available
• Official Gittensor activity: 64 PR(s), 526 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Treat this as maintainer-lane context rather than normal contributor-lane activity.
• No action.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.05%. Comparing base (6f46056) to head (4382b60).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2627   +/-   ##
=======================================
  Coverage   96.05%   96.05%           
=======================================
  Files         234      234           
  Lines       26280    26280           
  Branches     9531     9531           
=======================================
  Hits        25244    25244           
  Misses        425      425           
  Partials      611      611           

Files with missing lines	Coverage Δ
src/github/backfill.ts	96.77% <ø> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.