Skip to content

feat(observability): scrape REES metrics + add a dedicated REES Grafana dashboard - #5412

Merged
JSONbored merged 1 commit into
mainfrom
feat/rees-observability-dashboard
Jul 12, 2026
Merged

feat(observability): scrape REES metrics + add a dedicated REES Grafana dashboard#5412
JSONbored merged 1 commit into
mainfrom
feat/rees-observability-dashboard

Conversation

@JSONbored

@JSONbored JSONbored commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds a rees scrape target to prometheus/prometheus.yml, gated the same way as the other optional-profile targets (postgres/gpu/qdrant) — absent/down when --profile rees isn't active, no alert fires.
  • Ships grafana/dashboards/rees-metrics.json: request outcomes/latency by status, non-ok outcome rate, top analyzers by run rate, analyzer status breakdown, and slowest-analyzer p95 latency (bounded with topk on every per-analyzer breakdown so a future analyzer influx can't unbound the panel legend).
  • Adds resource-hub.json catalog entry + a dedicated dashboard test.
  • This is the second and final PR completing the tracking issue for REES observability — the metrics instrumentation itself shipped in the prior PR (feat(observability): instrument REES + its main-app client with real Prometheus metrics #5398); this PR only wires the scrape target and visualization now that there's real data to query against.

Verified live end to end on edge-nl-01 (not just structural JSON/PromQL validation):

  1. Rebuilt and restarted REES with the merged instrumentation, confirmed /metrics returns real data after triggering a real /v1/enrich call (rees_analyzer_runs_total, rees_enrich_requests_total, both histograms all populated correctly).
  2. Applied this PR's prometheus.yml to the live Prometheus, confirmed the rees target shows up with zero scrape errors.
  3. Ran all 8 of the new dashboard's PromQL expressions directly against the live Prometheus API — all status: success.
  4. Confirmed the dashboard file provisions into Grafana (GET /api/dashboards/uid/gittensory-rees returns the expected title/8 panels) and that Grafana's own datasource proxy returns real query results, not just a loaded-but-empty panel.

Part of #5365. Closes #5367.

Test plan

  • npm run typecheck
  • npm run test:ci (full local gate: 15099 passed / 12 skipped — one unrelated miner-repo-clone.test.ts timeout flake on the first run, confirmed by isolated re-run + a second full clean test:ci pass)
  • npm audit --audit-level=moderate — 0 vulnerabilities
  • node scripts/validate-observability-configs.mjs
  • Live verification on edge-nl-01 as described above

…na dashboard

Adds a rees scrape target to prometheus.yml (gated the same way as the other
optional-profile targets -- absent target just shows down, no alert fires) and a
new Gittensory — REES (review-enrichment) dashboard: request outcomes/latency,
plus per-analyzer run rate, status breakdown, and slowest-analyzer latency.
Completes the remaining deliverables from the REES metrics tracking issue; the
instrumentation itself shipped in the prior PR.

Every PromQL expression verified live against edge-nl-01's Prometheus + Grafana
after rebuilding REES with the new metrics code and sending a real /v1/enrich
request -- confirmed real data flows end to end (exporter -> scrape -> query).

Part of #5365, closes #5367
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.41%. Comparing base (0d78f24) to head (c9b114b).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5412      +/-   ##
==========================================
- Coverage   94.45%   94.41%   -0.04%     
==========================================
  Files         552      552              
  Lines       44310    44310              
  Branches    14659    14659              
==========================================
- Hits        41851    41834      -17     
- Misses       1784     1801      +17     
  Partials      675      675              
Flag Coverage Δ
shard-1 43.90% <ø> (-0.27%) ⬇️
shard-2 34.80% <ø> (+0.18%) ⬆️
shard-3 31.55% <ø> (+0.03%) ⬆️
shard-4 31.36% <ø> (-0.16%) ⬇️
shard-5 32.85% <ø> (-0.49%) ⬇️
shard-6 43.91% <ø> (+0.57%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 12, 2026
@loopover-orb

loopover-orb Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-12 18:22:19 UTC

5 files · 1 AI reviewer · no blockers · readiness 82/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR adds a `rees` Prometheus scrape target gated the same way as the other optional-profile jobs, ships a well-structured Grafana dashboard with 8 panels covering request outcomes/latency and per-analyzer health, and adds a resource-hub link plus dedicated tests for both the dashboard JSON and the scrape/compose wiring. The dashboard consistently uses the `${DS_PROMETHEUS}` datasource variable, bounds every per-analyzer breakdown with `topk` as claimed, and the metric names match what's described as already registered in REES's metrics.ts. This is narrow, well-tested, operator-facing observability work consistent with the maintainer's own established conventions (postgres/gpu/qdrant gating pattern), and it closes the stated tracking issue as the second half of a two-PR sequence.

Nits — 6 non-blocking
  • grafana/dashboards/rees-metrics.json: the 'Analyzer outcomes by status' panel (id 12) has no topk bound, which the test explicitly excludes on the reasoning that AnalyzerStatus is a fixed 5-value enum — worth double-checking that assumption stays true if the enum ever grows.
  • test/unit/selfhost-grafana-rees-dashboard.test.ts: the topk-bounding test only inspects `expr` string content for `by (analyzer)`/`by (le, analyzer)` substrings rather than parsing the PromQL AST, so a future panel using a differently-formatted label list (e.g. `by(analyzer)` without the space) would silently escape the check.
  • Consider asserting the panel count (8) explicitly in the dashboard test to catch accidental panel additions/removals in future edits, not just uid/title/tags.
  • Consider validating scrape_interval (15s) for the `rees` job in selfhost-observability-config.test.ts alongside job_name/static_configs, since that's part of the scrape contract too.
  • nit: grafana/dashboards/rees-metrics.json:12 hard-codes "53 registered analyzers" in operator-facing text, which will drift the next time analyzer count changes; make this count generic or document why this dashboard should track the exact current count.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #5367
Related work ⚠️ 1 scoped overlap Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 457 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 44 PR(s), 457 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ⚠️ ℹ️ Insufficient signal risk: clean · value: insufficient-signal — Nothing measurable for the structural-improvement analyzers on this PR (e.g. no code files changed).
Linked issue satisfaction

Addressed
This PR completes the tracking issue by adding the prometheus.yml scrape target (gated the same way as other optional-profile targets), a dedicated REES Grafana dashboard with per-analyzer success/timeout/latency panels, a resource-hub catalog link, and tests covering both the dashboard JSON/PromQL and the root observability-config compose/scrape wiring — with the underlying metrics instrumentatio

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 44 PR(s), 457 issue(s).
  • Related work: Titles/paths share 7 meaningful terms. (issue #5367, issue #5368)
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Review top overlaps.
  • Add a concise scope and risk note.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 12, 2026
@JSONbored
JSONbored merged commit aec1672 into main Jul 12, 2026
18 checks passed
@JSONbored
JSONbored deleted the feat/rees-observability-dashboard branch July 12, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(observability): instrument REES (review-enrichment service) with real Prometheus metrics

1 participant