Skip to content

feat(miner-selfimprove): read-only calibration dashboard view - #4504

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/calibration-dashboard-view
Jul 9, 2026
Merged

feat(miner-selfimprove): read-only calibration dashboard view#4504
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
jeffrey701:feat/calibration-dashboard-view

Conversation

@jeffrey701

Copy link
Copy Markdown
Contributor

Closes #4261

What

computePhase7CalibrationLoop (phase7-calibration-loop.ts) already produces the tracked calibration metric — predicted-gate accuracy combined from the historical_replay and pr_outcome signals, scored against the documented 0.62 baseline. This adds the read-only view that projects a Phase7CalibrationLoopResult into a human-readable dashboard shape (a CLI table, or a panel in the miner dashboard, renders the rows).

Pure — no new computation. It re-shapes an already-computed result (predicted-gate accuracy vs realized pr_outcome is presented, never recomputed here — that stays in the calibration loop). Public-safe: only accuracies, sample sizes, freshness, and the loop's holdReasons are surfaced (no raw scores/rewards).

API (packages/gittensory-engine/src/calibration-dashboard.ts)

  • resolveCalibrationDashboardStatus(result)"on_track" | "below_baseline" | "insufficient_signal" | "disabled".
  • buildCalibrationDashboardView(result){ status, headline, rows, holdReasons } — a headline (e.g. "68% combined (+6pts vs baseline)") plus labeled rows: combined accuracy, signed delta-from-baseline, each source (accuracy + n= sample size + fresh/stale), the replay-harness state, and whether an autonomy increase is permitted. holdReasons pass through verbatim so an operator sees why an increase is (or isn't) allowed.

Files

  • packages/gittensory-engine/src/calibration-dashboard.ts — the view.
  • packages/gittensory-engine/src/index.ts — barrel re-export.
  • test/unit/calibration-dashboard.test.ts — tests.

Testing

npx vitest run test/unit/calibration-dashboard.test.ts
npm run typecheck

5/5 tests pass (on-track, below-baseline negative-delta, insufficient-signal em-dash, and disabled/harness-hold/replay-due branches); typecheck clean. New engine file at 100% line + branch coverage (12/12, 24/24, 5/5).

`computePhase7CalibrationLoop` (phase7-calibration-loop.ts) already produces the
tracked calibration metric — predicted-gate accuracy combined from the
historical-replay and pr_outcome signals, against the documented 0.62 baseline.
This adds the read-only VIEW that projects a Phase7CalibrationLoopResult into a
human-readable dashboard shape (a CLI table or a UI panel renders the rows).

Pure: it re-shapes an already-computed result and adds NO new calibration
computation — accuracy vs realized pr_outcome is presented, never recomputed
here. Public-safe: only accuracies, sample sizes, freshness, and the loop's
holdReasons are surfaced (no raw scores/rewards).

- packages/gittensory-engine/src/calibration-dashboard.ts:
  resolveCalibrationDashboardStatus (on_track / below_baseline /
  insufficient_signal / disabled) and buildCalibrationDashboardView — a headline
  plus labeled rows for combined accuracy, delta-from-baseline (signed points),
  each source (accuracy + sample size + freshness), the replay-harness state,
  and whether an autonomy increase is permitted, with hold reasons verbatim.
- packages/gittensory-engine/src/index.ts: barrel re-export.
- test/unit/calibration-dashboard.test.ts: on-track, below-baseline (negative
  delta), insufficient-signal (em-dash), and disabled/hold/replay-due branches.

Closes JSONbored#4261
@jeffrey701
jeffrey701 requested a review from JSONbored as a code owner July 9, 2026 22:08
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.03%. Comparing base (23b3549) to head (cfd8e16).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4504   +/-   ##
=======================================
  Coverage   94.03%   94.03%           
=======================================
  Files         420      421    +1     
  Lines       37524    37541   +17     
  Branches    13702    13714   +12     
=======================================
+ Hits        35285    35302   +17     
  Misses       1583     1583           
  Partials      656      656           
Files with missing lines Coverage Δ
...ges/gittensory-engine/src/calibration-dashboard.ts 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. labels Jul 9, 2026
@loopover-orb

loopover-orb Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-09 22:20:52 UTC

3 files · 1 AI reviewer · no blockers · readiness 82/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
The AI review returned non-blocking notes for this change but did not include a separate narrative summary. Review the nits below before deciding this PR.

Nits — 5 non-blocking
  • calibration-dashboard.ts:64-90 always renders `Historical replay`/`PR outcome`/`Replay harness`/`Autonomy increase` rows even when `status` is `disabled` or `insufficient_signal`, which can show stale-looking `n=0`/`stale` detail alongside a 'disabled' headline — worth a one-line note in the row detail or a doc comment on intended consumer behavior in that case.
  • test/unit/calibration-dashboard.test.ts imports from `../../packages/gittensory-engine/src/index` (no `.js`) rather than the `.js`-suffixed relative imports used inside the package itself — confirm this matches the existing test-import convention elsewhere in `test/unit/`.
  • calibration-dashboard.ts:9-13 duplicates the module-level doc comment already given in the PR description almost verbatim; consider trimming to what a reader can't get from the exported JSDoc on the two functions.
  • Consider whether `rows` should omit or clearly flag source rows when `status === "disabled"`, so a CLI/UI renderer doesn't need its own special-casing on top of `status`.
  • If a future issue references linking `holdReasons` codes to human text, this view is the natural place to add a `label`/description lookup — not needed now given the pure read-only scope.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4261
Related work ⚠️ 1 scoped overlap Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 106 registered-repo PR(s), 45 merged, 4 issue(s).
Contributor context ✅ Confirmed Gittensor contributor jeffrey701; Gittensor profile; 106 PR(s), 4 issue(s).
Gate result ✅ Passing No configured blocker found.
Linked issue satisfaction

Partially addressed
The PR delivers the read-only dashboard view with the requested combined accuracy/baseline, per-source freshness, holdReasons, and replayRunDue rows, and includes tests for healthy, below-baseline, insufficient-signal, and disabled/hold states. However, the tests hand-construct `Phase7CalibrationLoopResult` objects directly rather than building `HistoricalReplayCalibrationInput`/`PrOutcomeCalibrat

Review context
  • Author: jeffrey701
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 106 PR(s), 4 issue(s).
  • Related work: Titles/paths share 6 meaningful terms. (issue #4261, issue #4307)
Contributor next steps
  • Review top overlaps.
  • Add a concise scope and risk note.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 363305c into JSONbored:main Jul 9, 2026
10 checks passed
@loopover-orb loopover-orb Bot added gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. and removed gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. labels Jul 9, 2026
@JSONbored JSONbored added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. and removed gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 9, 2026
loopover-orb Bot pushed a commit that referenced this pull request Jul 10, 2026
…hot series (#4639)

Extend the single-snapshot calibration dashboard (#4261/#4504) into an accuracy
*trend* over accumulated snapshots (#4268): direction (improving/degrading/flat),
a normalized sparkline, per-point delta vs baseline, and latest/change-over-window.
Read-only and pure. A brand-new install has no snapshots and warming-up snapshots
carry a null accuracy, so empty/single-point history renders an explicit
insufficient-history state instead of a broken chart.

Closes #4268
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(miner-selfimprove): calibration dashboard — predicted-gate accuracy vs real pr_outcome, read-only

2 participants