Skip to content

calibration: unify the tightening loop under the backtest evidence standard #8225

Description

@JSONbored

Problem

Epic #8211 track D capstone. Loosening moves only on Pareto-floored backtests; the legacy TIGHTENING loop (src/review/auto-tune.ts, shadow-soak + precision-floor triggers) predates that standard. Two evidence regimes for opposite directions of the same knobs is incoherent and blocks the drift sentinel's tighter-alternative findings from ever acting.

Requirements

⚠️ Required pattern. Design-first inside the issue before code: map every auto-tune trigger onto the registry model (a tightening candidate ladder above shipped, same splits, same sample floors) and decide per trigger whether it becomes a registry evaluation or stays bespoke WITH WRITTEN REASON. Behavior-pinning tests must hold auto-tune's current outcomes byte-stable until each trigger's migration lands.

  • Registry entries gain an optional tightening ladder; evaluateKnobLoosening generalizes (or gains a sibling) with direction-aware Pareto orientation — tightening trades recall for precision, so the floor's axes orientation must be explicit, not reused blind.
  • Applies remain double-gated per direction (separate env flag for tighten-autonomy; default off).

Deliverables

  • Design section committed to the docs page + the migration map in this issue
  • First migrated trigger end-to-end with byte-stable-pinning tests for the rest

Links & Resources

#8211 (epic), the slop/quality registry issue (blocked-by, shares registry churn), src/review/auto-tune.ts, src/review/selftune-wire.ts

Boundaries

No trigger loses coverage mid-migration: old path keeps running for unmigrated triggers.

maintainer-only — gate-authority unification.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions