Context
Phase 7 (Self-improve) of the gittensory-miner roadmap is 0% started. Its stated scope is outcome-driven calibration of the predicted-gate and self-slop scorer against real pr_outcome, personalized and value-weighted gate prediction, fleet anti-farming feedback, and an engine-parity drift auto-flag. Today the only calibration signal is passively-observed pr_outcome trickling in from live miner submissions as they resolve through the maintainer review stack, which is slow, noisy, and hard to control for confounds like repo mix, task difficulty, or timing.
This epic builds the missing piece underneath all of Phase 7: a rigorous, internal, leakage-safe historical-replay calibration harness. Instead of waiting on live outcomes, we freeze a real target repo's state as it existed at a past commit, replay the miner's actual discover/plan/create/self-review pipeline against that frozen state, and score the result against what the repo's real subsequent history shows actually happened. This gives dense, controllable, ground-truthed calibration signal on demand, using the same predicted-gate (byte-identical to the live maintainer gate) and the deterministic slop/self-review scorer the miner already runs in production.
Why this matters
The self-review calibration gap is not hypothetical: the documented baseline is 62% accuracy against real pr_outcome. Phase 7 exists specifically to close that gap before any autonomy-level increase is considered. The stated ordering rule is never trust the loop before calibration confirms it. Passive pr_outcome accumulation cannot validate that rule fast enough or rigorously enough on its own: it has no ground truth we control, no ability to replay the same freeze point under a changed scorer version, and no way to catch a calibration regression before it reaches production submissions. A controlled historical-replay methodology, with real ground truth pulled from real repo history and real leakage-safety discipline, gives faster and more rigorous signal than waiting on the live loop, and is the only way to responsibly cross the never-trust-the-loop-before-calibration-confirms-it line.
Child issues
Context
Phase 7 (Self-improve) of the gittensory-miner roadmap is 0% started. Its stated scope is outcome-driven calibration of the predicted-gate and self-slop scorer against real pr_outcome, personalized and value-weighted gate prediction, fleet anti-farming feedback, and an engine-parity drift auto-flag. Today the only calibration signal is passively-observed pr_outcome trickling in from live miner submissions as they resolve through the maintainer review stack, which is slow, noisy, and hard to control for confounds like repo mix, task difficulty, or timing.
This epic builds the missing piece underneath all of Phase 7: a rigorous, internal, leakage-safe historical-replay calibration harness. Instead of waiting on live outcomes, we freeze a real target repo's state as it existed at a past commit, replay the miner's actual discover/plan/create/self-review pipeline against that frozen state, and score the result against what the repo's real subsequent history shows actually happened. This gives dense, controllable, ground-truthed calibration signal on demand, using the same predicted-gate (byte-identical to the live maintainer gate) and the deterministic slop/self-review scorer the miner already runs in production.
Why this matters
The self-review calibration gap is not hypothetical: the documented baseline is 62% accuracy against real pr_outcome. Phase 7 exists specifically to close that gap before any autonomy-level increase is considered. The stated ordering rule is never trust the loop before calibration confirms it. Passive pr_outcome accumulation cannot validate that rule fast enough or rigorously enough on its own: it has no ground truth we control, no ability to replay the same freeze point under a changed scorer version, and no way to catch a calibration regression before it reaches production submissions. A controlled historical-replay methodology, with real ground truth pulled from real repo history and real leakage-safety discipline, gives faster and more rigorous signal than waiting on the live loop, and is the only way to responsibly cross the never-trust-the-loop-before-calibration-confirms-it line.
Child issues