Skip to content

feat(miner-selfimprove): calibration dashboard — predicted-gate accuracy vs real pr_outcome, read-only #4261

Description

@JSONbored

packages/gittensory-engine/src/phase7-calibration-loop.ts (landed via #3014/PR #3225) is a real, tested, pure engine module: computePhase7CalibrationLoop combines a historical_replay composite score with the live pr_outcome signal into one tracked calibration metric (Phase7CalibrationLoopResult), evaluated against the documented DOCUMENTED_CALIBRATION_BASELINE = 0.62 self-review accuracy baseline (phase7-calibration-loop.ts:11). It already exposes exactly the shape a read-only dashboard needs: combinedAccuracy, deltaFromBaseline, a per-source breakdown (bySource.historical_replay / bySource.pr_outcome, each with accuracy/sampleSize/observedAt/fresh), holdReasons, replayHarnessStatus, and audit.contributingSources/rejectedSources (phase7-calibration-loop.ts:80-102). There's even an existing renderPhase7CalibrationAuditMarkdown() (phase7-calibration-loop.ts:533-596) that already turns a computed result into a deterministic, public-safe Markdown report — a reasonable starting point or reference format for this dashboard's read-only view.

This dashboard item is the consumer of that module's output — a read-only view (CLI table, or a panel under the Phase 6 apps/gittensory-miner-ui/ dashboard shell once that scaffold lands) that renders a Phase7CalibrationLoopResult for a human to read.

Important caveat, cite don't duplicate #4248: #4248 ("feat(miner-selfimprove): wire the historical-replay scorers into the Phase 7 calibration loop", maintainer-only, currently open) found that phase7-calibration-loop.ts is not actually fed real data yet — neither packages/gittensory-miner/lib/replay-objective-anchor.js (#3012) nor lib/replay-task-generation.js (#3011) is wired to call into it, and PR #3225 only ever touched files inside packages/gittensory-engine/. That means this dashboard item can be built and tested today against the module's typed contract (feed it synthetic/fixture Phase7CalibrationLoopResult inputs, or call computePhase7CalibrationLoop directly with hand-built historicalReplay/prOutcome inputs the way phase7-calibration-loop.ts's own tests do) — but it will not show real, live numbers until #4248 lands and actually wires the replay scorers through. Building this now is still useful groundwork: the render/view layer doesn't need to wait on the wiring.

Deliverables

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.gittensor:priorityMaintainer-selected Gittensor priority — scores a 1.5x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions