Skip to content

feat(observability): add GitHub issue stats to the maintainer dashboard - #4170

Closed
JSONbored wants to merge 1 commit into
mainfrom
feat/dashboard-github-issues-stats-3716
Closed

feat(observability): add GitHub issue stats to the maintainer dashboard#4170
JSONbored wants to merge 1 commit into
mainfrom
feat/dashboard-github-issues-stats-3716

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • The "Reviews & PRs (maintainer)" dashboard (grafana/dashboards/maintainer-reviews.json) tracked pull requests only. Add a new "GitHub issues" stat row (Issues opened / Issues closed / Issues open) for the same dashboard time window the PR panels already use, so a maintainer sees issue activity without switching to the separate upstream GitHub dashboard.
  • Datasource choice (per the issue's own judgment call): every existing panel on this dashboard already reads the local frser-sqlite-datasource reporting export (uid: gittensory-db), not the grafana-github-datasource live-API plugin the sibling github-prs.json dashboard uses for its own "Open issues" panel. To match this dashboard's own established convention instead of blindly copying the sibling's, the new panels query a local issues table through the same datasource, with the same unixepoch(updated_at) >= ${__from} AND < ${__to} windowing the existing panels use (Issues opened is the one deliberate exception, keyed off created_at instead, since an issue's created_at never changes after the fact — see its panel description). The tradeoff and caveat are documented on the dashboard itself (root description plus the Issues open panel's own description) and cross-link to the sibling dashboard's time-range-independent "Open issues" panel for an upstream-accurate count.
  • The load-bearing part: the local issues table (migrations/0001_initial.sql, webhook-populated by upsertIssueFromGitHub for every repo the app receives issue webhooks for — verified at the call site in src/queue/processors.ts, gated only on payload.issue existing, not a gittensory-specific check) was never mirrored into the redacted reporting export the dashboard actually reads — scripts/export-grafana-reporting-db.sh only ever exported review_targets and ai_usage_events. Without this, the new panels would query a table that doesn't exist in the reporting SQLite file. Extended the exporter with a minimal issues mirror (repo, number, state, created_at, updated_at only — no title/labels/payload, since these are count-only stat panels) across both the Postgres-source and SQLite-source code paths, including the incremental fast-path fingerprint (full-content hash, since issues receive in-place updates like pull_requests/review_targets do).
  • I could not query the live production reporting database directly from this environment to literally spot-check a per-repo row count (as the issue suggests); I instead verified structurally that the webhook handler populating this table is repo-generic (same shape as the already-multi-repo PR pipeline), and proved it end-to-end with a multi-repo synthetic fixture in the new tests.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • This PR touches zero files under src/**, packages/**, or apps/gittensory-ui/** (only grafana/, scripts/, and test/unit/), and npm run test:changed (vitest's real import-graph diff against origin/main) confirms exactly two test files are reachable from this diff — both are included above and pass. test:workers, build:mcp, test:mcp-pack, and the ui:* checks have no path into this diff's blast radius, so I did not re-run them locally.
  • npm run test:coverage (full, unsharded) found exactly one failing test out of 11,997: test/unit/selfhost-update-script.test.ts > selfhost-update.sh > fetches, fast-forwards, rebuilds, and verifies health on a clean checkout, a 15s timeout. That file is untouched by this diff and not in the test:changed set. Re-run in isolation (npx vitest run test/unit/selfhost-update-script.test.ts), all 12 of its tests pass in ~6.6s total (922ms for the specific test), confirming a resource-contention flake from running the entire suite concurrently rather than a real regression.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — no auth/cookie/CORS/session code touched.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — no API/OpenAPI/MCP surface touched.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (N/A — no apps/gittensory-ui changes; this is a Grafana dashboard backed by a real reporting-DB export, not the product UI.)
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository. (N/A — a self-host Grafana dashboard panel, not apps/gittensory-ui; no local Grafana instance to screenshot from in this environment. Panel layout/queries are verified instead via the new automated tests in test/unit/selfhost-grafana-dashboard.test.ts.)
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. (No changelog edit; no public docs reference this dashboard by name.)

UI Evidence

N/A — this is a self-host Grafana dashboard change (grafana/dashboards/maintainer-reviews.json), not an apps/gittensory-ui change. There is no running Grafana instance in this environment to screenshot from. Panel structure, query correctness, and windowing semantics are covered by new automated tests instead (test/unit/selfhost-grafana-dashboard.test.ts, test/unit/selfhost-grafana-reporting.test.ts), and the JSON is validated with python3 -m json.tool, jq, and npm run selfhost:validate-observability.

Notes

  • Also extended scripts/export-grafana-reporting-db.sh (redacted reporting-DB exporter) and its test suite (test/unit/selfhost-grafana-reporting.test.ts) to mirror the issues table — required for the new panels to have real data to query; see Summary for why.
  • Dashboard version bumped 5 → 6, matching this repo's established convention for dashboard edits (e.g. fix(observability): clarify manual/commented/ignored panel semantics #4134).

Closes #3716

The Reviews & PRs (maintainer) dashboard tracked pull requests only --
there was no issue-activity visibility without switching to the
separate upstream GitHub dashboard. Add an "Issues opened/closed/open"
stat row using the same frser-sqlite-datasource plus windowed-updated_at
convention every existing panel on this dashboard already uses, rather
than the live-API grafana-github-datasource the sibling dashboard uses
for its own "Open issues" panel -- matching this dashboard's own
established, webhook-observed philosophy instead of introducing a
second datasource type into one dashboard. The tradeoff and its
caveat are documented on the dashboard itself (root description plus
per-panel descriptions) and cross-link to the sibling dashboard for
an upstream-accurate, time-range-independent count.

That local issues table was never mirrored into the redacted
reporting export the dashboard actually reads (only review_targets
and ai_usage_events were), so also extend
export-grafana-reporting-db.sh with a minimal issues mirror (repo,
number, state, created_at, updated_at only -- no title/labels/payload,
since these are count-only stat panels) across both the Postgres and
SQLite source paths, including the incremental fingerprint that
detects an in-place state change.

Closes #3716
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 8, 2026
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.73%. Comparing base (b453658) to head (dda8de5).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4170   +/-   ##
=======================================
  Coverage   93.73%   93.73%           
=======================================
  Files         387      387           
  Lines       36304    36304           
  Branches    13298    13298           
=======================================
  Hits        34031    34031           
  Misses       1617     1617           
  Partials      656      656           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb

loopover-orb Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-08 09:21:19 UTC

4 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · dirty

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): scripts/export-grafana-reporting-db.sh (matched scripts/**).

Review summary
This adds a redacted `issues` mirror table to the Grafana reporting SQLite export and a matching 'GitHub issues' stat row (Issues opened/closed/open) to the maintainer dashboard, deliberately reusing the dashboard's existing frser-sqlite-datasource convention instead of the sibling dashboard's live-API plugin, with that tradeoff documented on the dashboard itself. The `issues` table is correctly classified as mutable (full-dump hash fingerprint, not the append-only count+max aggregate), and the SQLite-source, Postgres-source, and CSV import paths, table-existence guards, and fingerprint loops were all updated in lockstep with no stale call site visible. Tests cover the redacted-column mirror (no title/labels/payload leak), the pre-issues-table empty case, an in-place `state`-flip regression against the fingerprint, the Postgres COPY path across multiple repos, and the exact time-window/keying semantics of all three new panel queries; CI is green across build, typecheck, tests, and security scans.

Nits — 6 non-blocking
  • The 'Issues closed'/'Issues open' panel queries in grafana/dashboards/maintainer-reviews.json assume the app-side `issues.state` column is always exactly the lowercase strings 'open'/'closed' — that source (upsertIssueFromGitHub) isn't visible in this diff, so it's worth a one-line confirmation from you rather than a blocker.
  • scripts/export-grafana-reporting-db.sh has grown to ~689 lines; this PR's increment is proportionate to the existing per-table pattern, but as more tables get added the per-table export blocks (pull_requests/review_targets/ai_usage_events/issues) may be worth extracting into a shared helper.
  • Add a short comment near the new `issues` CREATE TABLE (or in the PR description) pinning the exact app-side `state` values expected, so a future change to the webhook processor's state strings doesn't silently break these panel counts.
  • Consider a follow-up lifetime/backlog open-issue panel to complement the new in-window-snapshot 'Issues open' panel, mirroring the caveat already documented for 'Manual review' on this same dashboard — not needed for this PR.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3716
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 52 registered-repo PR(s), 43 merged, 486 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 52 PR(s), 486 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Linked issue satisfaction

Addressed
The PR adds a new 'GitHub issues' stat row (Issues opened/closed/open) to maintainer-reviews.json with matching stat-panel visual style, extends the reporting export pipeline to mirror the local issues table for all repos (not just gittensory), and documents the datasource tradeoff via a dashboard-level description plus panel descriptions cross-linking to the sibling github-prs.json dashboard's ca

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 52 PR(s), 486 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 8, 2026
@JSONbored

Copy link
Copy Markdown
Owner Author

Closing unmerged — superseded by #4168, which merged first and implements the same 3 stat panels (Issues opened/closed/open) for the same issue #3716. Re-adding an equivalent-but-differently-designed panel set on top would just create duplicate/conflicting rows on the same dashboard, which is worse than the datasource-consistency tradeoff this version made differently (noted for the record in a comment on #3716). No further action needed here.

@JSONbored JSONbored closed this Jul 8, 2026
@JSONbored
JSONbored deleted the feat/dashboard-github-issues-stats-3716 branch July 8, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(observability): add GitHub Issues stats to the Reviews & PRs maintainer dashboard

1 participant