Context
Part of the internal historical-replay calibration harness roadmap. A frozen snapshot at commit T is only useful for calibration if there is a well-defined, real, subsequent history to score against, and if nothing in the frozen context lets the miner infer that future history by pattern-matching text rather than reasoning. This issue defines how freeze points are selected and how the frozen context is scrubbed before it is handed to the replay pipeline.
Goal
Select calibration-worthy freeze points and scrub forward references so replay cannot cheat off future information embedded in the frozen context.
Requirements
- Define selection criteria for a freeze point T: minimum prior commit history depth, and a minimum number of subsequent revealed commits/PRs after T to serve as scorable ground truth.
- Scrub the frozen context of forward references: issue/PR numbers and links, deep-links, and raw commit SHAs that only resolve to something created after T.
- Mix freeze-point selection across two pools: repos with activity recent to the calibration run's own model training cutoff, and older or lower-traffic repos, to reduce the chance a judge or planner has memorized the actual outcome. Record which pool each freeze point was drawn from so downstream scoring can be sliced by pool.
- Store the revealed post-T history (the ground truth) separately from the frozen pre-T snapshot so the replay pipeline never has both in scope at once.
- Add a lint/check that fails a freeze-point candidate if scrubbing cannot remove all detected forward references.
Deliverables
Expected outcomes
- No frozen-context sample handed to a replay run contains a resolvable forward reference
- Freeze-point pool is measurably mixed across recency bands, not concentrated in one
Part of the historical-replay calibration roadmap: #2994
Context
Part of the internal historical-replay calibration harness roadmap. A frozen snapshot at commit T is only useful for calibration if there is a well-defined, real, subsequent history to score against, and if nothing in the frozen context lets the miner infer that future history by pattern-matching text rather than reasoning. This issue defines how freeze points are selected and how the frozen context is scrubbed before it is handed to the replay pipeline.
Goal
Select calibration-worthy freeze points and scrub forward references so replay cannot cheat off future information embedded in the frozen context.
Requirements
Deliverables
Expected outcomes
Part of the historical-replay calibration roadmap: #2994