perf(selfhost): skip reporting-exporter's rebuild when the source is unchanged

[JSONbored]

Summary

• scripts/export-grafana-reporting-db.sh (run every 30s by default via reporting-exporter in docker-compose.yml) rebuilt the ENTIRE Grafana reporting SQLite mirror from scratch every cycle — full re-export + re-import of pull_requests/review_audit-derived review_targets and the full ai_usage_events table, regardless of whether anything had actually changed.
• Measured live on a real self-host instance (edge-us-01): this container uses ~2MB of RAM but has generated 91.3GB of cumulative block I/O in 37 hours (~2.5GB/hour). ai_usage_events is an append-only log of every AI call, so this cost only grows as review volume grows — a real, currently-active scalability ceiling.
• Added a cheap COUNT(*) || MAX(<timestamp column>) fingerprint per source table (pull_requests, review_audit, review_targets, ai_usage_events), computed for BOTH the SQLite-source and Postgres-source paths. When the fingerprint matches the last run's AND a last-good output DB already exists, the entire rebuild is skipped.
• Fails open by design: any error or missing piece while computing the fingerprint (missing source file, missing table, unreachable Postgres) falls through to the existing full-rebuild path unchanged — this is purely an optimization on top of proven-correct logic, never a new failure mode or a new way to serve stale data.
• A minor, deliberately-accepted behavior nuance: on a self-host instance whose app DB has literally none of the five reporting tables on two consecutive cycles, this now exits 0 (skipped) on the second occurrence instead of the previous exit 1 ("preserving last good"). Same last-good snapshot is preserved either way — only the exit code/log-noise differs, and it stops the container's while true loop from logging [reporting] export failed every 30s forever on a fresh, not-yet-migrated instance.

Found via a fresh performance/scalability/accuracy hardening audit of the self-host ORB stack (this is the finding with the real measured production evidence). Tracked under #1667.

Scope

• scripts/export-grafana-reporting-db.sh — fingerprint fast-path for both source paths
• test/unit/selfhost-grafana-reporting.test.ts — 3 new tests (SQLite skip, SQLite redo-on-change, Postgres skip); all 14 pre-existing tests pass unmodified

Validation

• sh -n scripts/export-grafana-reporting-db.sh (syntax check)
• npm run typecheck
• npx vitest run test/unit/selfhost-grafana-reporting.test.ts — 17/17 passing (14 pre-existing + 3 new)
• git diff --check clean

Safety

• scripts/** and test/** — no Codecov patch-coverage obligation, but the full existing correctness test suite (gate-decision precedence, IPv6 host parsing, column-fallback compat, BusyBox mktemp compat, fail-closed-on-Postgres-error) passes untouched, proving the rebuild path itself is byte-identical when it does run.
• No secrets, no schema change to the reporting DB itself.

Closes #3895

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.58%. Comparing base (c433e59) to head (c85666c).
⚠️ Report is 13 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3935   +/-   ##
=======================================
  Coverage   93.58%   93.58%           
=======================================
  Files         358      358           
  Lines       34342    34342           
  Branches    12570    12570           
=======================================
  Hits        32138    32138           
  Misses       1580     1580           
  Partials      624      624           

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[loopover-orb]

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

_{Review updated: 2026-07-07 07:25:37 UTC}

2 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
This PR adds a cheap COUNT+MAX fingerprint check per source table (pull_requests, review_audit, review_targets, ai_usage_events) computed identically for both the SQLite and Postgres source paths, and skips the full rebuild when the fingerprint matches the prior run's and a last-good output DB exists. The fail-open design is correctly implemented: any error computing the fingerprint (missing file/table, unreachable psql) leaves CURRENT_FINGERPRINT empty, which cannot satisfy the skip condition, so it always falls through to the existing proven rebuild path. Tests cover the SQLite skip/rebuild-on-change cases and the Postgres skip case, and directly verify the last-good snapshot is untouched (PRAGMA quick_check, row counts) after a skip. The documented exit-code nuance (exit 0 instead of exit 1 on a second no-tables cycle) is intentional, narrow, and doesn't change what's persisted.

Nits — 6 non-blocking

• The COUNT+MAX fingerprint is a heuristic: a same-cycle delete+insert that nets zero row-count change and coincidentally reproduces the same MAX(timestamp) would false-positive as 'unchanged' — worth a one-line comment acknowledging this known limitation near sqlite_source_fingerprint/pg_source_fingerprint in scripts/export-grafana-reporting-db.sh.
• FINGERPRINT_FILE.tmp (written in persist_fingerprint) isn't covered by the cleanup() trap, so a kill mid-write could leave a stray .tmp file in $OUT_DIR indefinitely; consider adding `rm -f "${FINGERPRINT_FILE}.tmp"` to cleanup().
• scripts/export-grafana-reporting-db.sh is now ~597 lines; the new fingerprint helpers (sqlite_source_fingerprint/pg_source_fingerprint/persist_fingerprint) could be split into a sourced helper file for readability, though this is optional given the script's existing size.
• Add the fingerprint-staleness caveat as a code comment so future readers understand it's a heuristic, not an exact change-detection mechanism.
• Fold FINGERPRINT_FILE.tmp cleanup into the existing trap for symmetry with TMP_DB handling.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ Linked	#3895
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 51 registered-repo PR(s), 43 merged, 343 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 51 PR(s), 343 issue(s).
Gate result	✅ Passing	No configured blocker found.

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
• Official Gittensor activity: 51 PR(s), 343 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Treat this as maintainer-lane context rather than normal contributor-lane activity.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[loopover-orb]

Gittensory approves — the gate is satisfied and CI is green.