Skip to content

fix(selfhost): validate reporting exporter fast path - #4079

Merged
JSONbored merged 1 commit into
mainfrom
codex/fix-stale-or-corrupted-reporting-db-issue
Jul 8, 2026
Merged

fix(selfhost): validate reporting exporter fast path#4079
JSONbored merged 1 commit into
mainfrom
codex/fix-stale-or-corrupted-reporting-db-issue

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • The incremental fast-path previously used a weak COUNT(*)+MAX(timestamp) fingerprint and trusted any non-empty output DB, which could preserve stale or corrupted reporting snapshots and break observability freshness.
  • The change aims to make the skip decision safe by detecting in-place edits that don't change row counts or max timestamps and by ensuring the existing reporting DB is valid before skipping.

Description

  • Replace the weak per-table COUNT+MAX fingerprint with a content hash: for SQLite the script sqlite3 "$APP_DB" ".dump <table>" is hashed; for Postgres the script computes an md5 over ordered row_to_json(t) output, and these per-table digests are composed into the source fingerprint in scripts/export-grafana-reporting-db.sh.
  • Add reporting_db_ok() which requires sqlite3 "$OUT_DB" "PRAGMA quick_check;" to return ok before the fast-path can skip a rebuild, and include this check in the early-exit condition.
  • Harden hash_stdin() to robustly prefer sha256sum then shasum, and fail cleanly when neither exists.
  • Add regression tests in test/unit/selfhost-grafana-reporting.test.ts to cover an in-place SQLite edit that keeps count/max unchanged and to ensure a corrupted last-good OUT_DB triggers a rebuild instead of being preserved.

Testing

  • Ran git diff --check and it succeeded.
  • Ran npx vitest run test/unit/selfhost-grafana-reporting.test.ts and the file's test suite passed (19 tests passed), including the two new regression tests.
  • Attempted npm run test:ci but it could not complete due to existing unrelated TypeScript syntax errors in src/queue/processors.ts, so the full CI gate was not run here.
  • Attempted npm audit --audit-level=moderate but the registry audit endpoint returned 403 Forbidden, so the audit step could not be completed.

Codex Task

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.65%. Comparing base (1ec9f94) to head (a507b0c).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4079   +/-   ##
=======================================
  Coverage   93.65%   93.65%           
=======================================
  Files         384      384           
  Lines       35865    35865           
  Branches    13160    13160           
=======================================
  Hits        33589    33589           
  Misses       1618     1618           
  Partials      658      658           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 7, 2026
@loopover-orb

loopover-orb Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-08 03:21:03 UTC

2 files · 2 blockers · readiness 93/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers No AI review summary
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 51 registered-repo PR(s), 43 merged, 505 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 51 PR(s), 505 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 51 PR(s), 505 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 7, 2026
@JSONbored
JSONbored force-pushed the codex/fix-stale-or-corrupted-reporting-db-issue branch from efb21f6 to a507b0c Compare July 8, 2026 02:48
@JSONbored
JSONbored merged commit a6adbcb into main Jul 8, 2026
11 checks passed
@JSONbored
JSONbored deleted the codex/fix-stale-or-corrupted-reporting-db-issue branch July 8, 2026 03:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant