Skip to content

maintainer-only: personalized gate-prediction tuning per contributor/miner history #2349

Description

@JSONbored

Extend the predicted-gate self-review so a returning contributor/miner's OWN historical accuracy (from the calibration ledger) can adjust confidence in future predictions for that same actor — e.g. a miner whose past predicted-merge verdicts have reliably matched real outcomes earns a tighter confidence band; one with a track record of false-merge predictions gets a more conservative (more likely to warn/hold) verdict. This is exactly the "adjusts live scoring/gate calibration" case the safety tier calls out explicitly, because it changes what verdict buildPredictedGateVerdict (or a future live-gate-adjacent surface) hands back based on WHO is asking — a personalization/authority-weighting decision the owner wants to build and review directly.

Deliverables

  • A per-actor calibration lookup (keyed by contributor/miner login) feeding an adjustment factor into the predicted-gate confidence/readiness output
  • Explicit floor/ceiling clamps so personalization can only narrow or widen a WARNING band, never silently flip a hard blocker off
  • Cold-start behavior for a never-seen actor (falls back to the unweighted baseline, no penalty and no bonus)
  • Full regression coverage: cold-start, strong-track-record, weak-track-record, and a clamp-boundary test proving personalization cannot bypass a hard blocker
  • A design note on where this may NOT be exposed publicly (never reveal a specific actor's personal accuracy/confidence weighting to that actor or anyone else — this is exactly the kind of "raw trust/score" surface src/signals/redaction.ts already exists to keep private)

References

  • src/rules/predicted-gate.ts (263 lines) — buildPredictedGateVerdict, the surface this personalizes
  • src/review/parity.ts (301 lines) — the calibration ledger this reads per-actor history from (once source-scoped per the existing source?: string param on computeGateEval)
  • src/signals/redaction.ts — the existing private-signal boundary this personalization data must stay behind

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions