feat(miner-selfimprove): render prediction-calibration Prometheus metrics - #4461
Conversation
…rics A pure Prometheus text-exposition renderer for the miner's own predicted-gate accuracy — the miner-side counterpart to the server's src/selfhost/metrics.ts registry, scoped to calibration rather than the server's queue/webhook metrics. Design decision (the issue's open question — CLI vs HTTP, shared vs standalone): gittensory-miner is a local CLI, not a daemon, so this is an on-demand RENDERER a caller prints to stdout for its own scrape/cron setup, not an HTTP-served registry. It mirrors src/selfhost/metrics.ts's metric-naming (gittensory_miner_* _total) and HELP/TYPE/label-escaping conventions rather than importing across the package boundary. It stays a pure, side-effect-free engine function: the caller reads the prediction-ledger (packages/gittensory-miner/lib/prediction- ledger.js `readPredictions`, JSONbored#4263) and passes the rows in — no data collection of its own. Counters: - gittensory_miner_predictions_total{conclusion="..."} — predictions recorded, one series per predicted conclusion. - gittensory_miner_prediction_correct_total / _incorrect_total — the confusion- matrix-shaped correct/incorrect counts, which only move for rows carrying a resolved outcome (unresolved rows count toward predictions_total only), so the surface is meaningful before outcome-pairing exists and grows once it does. - packages/gittensory-engine/src/miner-prediction-metrics.ts: the renderer + metric-name constants + the MinerPredictionMetricRow input type; the counters are documented in the module header. - packages/gittensory-engine/src/index.ts: barrel re-export. - test/unit/miner-prediction-metrics.test.ts: valid-exposition assertions over empty / unresolved / mixed-correctness / label-escaping fixture states. Closes JSONbored#4264
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4461 +/- ##
=======================================
Coverage 94.01% 94.01%
=======================================
Files 418 419 +1
Lines 37410 37434 +24
Branches 13675 13678 +3
=======================================
+ Hits 35171 35195 +24
Misses 1583 1583
Partials 656 656
🚀 New features to boost your workflow:
|
|
Tip 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 ✅ Gittensory review result - approve/merge recommendedReview updated: 2026-07-09 20:15:42 UTC
✅ Suggested Action - Approve/Merge
Review summary Nits — 5 non-blocking
Linked issue satisfactionPartially addressed Review context
Contributor next steps
Signal definitions
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.
|
Closes #4264
What
A pure Prometheus text-exposition renderer for the miner's own predicted-gate accuracy — the miner-side counterpart to the server's
src/selfhost/metrics.tsregistry, scoped to calibration rather than the server's queue/webhook/AI-provider metrics.Design decision (the issue's open question)
gittensory-mineris a local CLI, not a daemon, so this is an on-demand renderer a caller prints to stdout for its own scrape/cron setup — not an HTTP-served registry. It mirrorssrc/selfhost/metrics.ts's metric-naming (gittensory_miner_*_total) and# HELP/# TYPE/label-escaping conventions rather than importing across the package boundary. It stays a pure, side-effect-free engine function: the caller reads the prediction-ledger (prediction-ledger.jsreadPredictions, #4263) and passes the rows in — no data collection of its own lives here.Counters
gittensory_miner_predictions_total{conclusion="..."}— predictions recorded, one series per predicted conclusion (e.g.merge/close/hold).gittensory_miner_prediction_correct_total— predictions whose realized outcome matched the prediction.gittensory_miner_prediction_incorrect_total— predictions whose realized outcome differed.The correct/incorrect counters (the confusion-matrix-shaped buckets, mirroring
computeGateEval) only move for rows carrying a resolved outcome; unresolved rows count towardpredictions_totalonly, so the surface is meaningful before outcome-pairing exists and grows once it does. Output is deterministic (conclusion series sorted) and always emits HELP/TYPE, so it's well-formed even for an empty ledger.API (
packages/gittensory-engine/src/miner-prediction-metrics.ts)renderMinerPredictionMetrics(rows) => string— Prometheus exposition text.MinerPredictionMetricRow—{ conclusion, correct? }(a ledger row joined with its optional resolved outcome).MINER_PREDICTIONS_TOTAL/MINER_PREDICTION_CORRECT_TOTAL/MINER_PREDICTION_INCORRECT_TOTALname constants.The counters are documented in the module header.
Files
packages/gittensory-engine/src/miner-prediction-metrics.ts— the renderer.packages/gittensory-engine/src/index.ts— barrel re-export.test/unit/miner-prediction-metrics.test.ts— tests.Testing
4/4 tests pass (empty / unresolved / mixed-correctness / label-escaping fixtures); typecheck clean. New engine file at 100% line + branch coverage (24/24, 6/6, 4/4).