⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.
Context
packages/loopover-engine/src/phase7-calibration-loop.ts's computePhase7CalibrationLoop builds two
CalibrationSourceMetric objects, one per calibration source. historicalReplayMetric.fresh (around
line 410) is a real computation, gated through isHistoricalReplayRunFresh against
observedAt/config.replayFreshnessMaxAgeHours. prOutcomeMetric.fresh (line 374) is a hardcoded
constant:
const prOutcomeMetric: CalibrationSourceMetric = {
source: "pr_outcome",
accuracy: prOutcomeDerived?.accuracy ?? null,
sampleSize: prOutcomeDerived?.sampleSize ?? 0,
observedAt: normalizeObservedAt(input.prOutcome?.observedAt),
fresh: true,
};
fresh is always true regardless of observedAt — even when observedAt is null because no
timestamp was supplied at all. CalibrationSourceMetric.fresh is a single shared field used
identically for both sources by calibration-dashboard.ts's sourceRow (lines 42-48), which renders
detail: n=${sampleSize} · ${fresh ? "fresh" : "stale"} for both the "Historical replay" and "PR
outcome" dashboard rows. The practical effect: the "PR outcome" row can never show "stale" no matter
how old the underlying pr_outcome sample actually is, silently misleading whoever reads the
dashboard about how current that number is. test/unit/phase7-calibration-loop.test.ts (grepped for
"fresh") never asserts a value for prOutcomeMetric.fresh — the constant has zero test coverage of
its own correctness.
Requirements
- Compute
prOutcomeMetric.fresh the same way historicalReplayMetric.fresh is computed: reuse
isHistoricalReplayRunFresh (or extract a source-agnostic freshness helper if that reads better —
the exported/internal function name is up to the implementation, but the freshness semantics must
be identical for both sources: same max-age comparison against config.replayFreshnessMaxAgeHours,
same handling of a missing/unparseable observedAt as NOT fresh) against
prOutcomeMetric.observedAt and the same now/config inputs historicalReplayMetric.fresh
already uses.
- A
null observedAt (no timestamp supplied for the pr_outcome source) must resolve to
fresh: false, matching how isHistoricalReplayRunFresh already treats a missing/invalid
observedAt for the historical-replay source.
- Do not change
prOutcomeDerived, accuracy, or sampleSize computation, and do not change
historicalReplayMetric's own freshness logic.
Deliverables
All three Deliverables are required in this one PR — there is no narrower scope for this issue.
Test Coverage Requirements
packages/loopover-engine/src/** is measured by Codecov via two separate uploads whose hits are
unioned — root test/** AND packages/loopover-engine/test/**. Add regression tests to
packages/loopover-engine/test/phase7-calibration-loop.test.ts (this file's existing test location)
asserting prOutcomeMetric.fresh for: a recent observedAt (within the max-age window) → true; a
stale observedAt (older than the max-age window) → false; and a null/missing observedAt →
false. Also add or extend a test in test/unit/calibration-dashboard.test.ts (root-level)
confirming the "PR outcome" dashboard row renders "stale" when prOutcomeMetric.fresh is false
(previously impossible to test meaningfully, since the value was always true). Target 100% branch
coverage of the new freshness computation.
Expected Outcome
The calibration dashboard's "PR outcome" row can genuinely show "stale," matching the accuracy the
"Historical replay" row already has, so an operator reading the dashboard gets a truthful signal
about how current the pr_outcome calibration sample actually is.
Links & Resources
packages/loopover-engine/src/phase7-calibration-loop.ts (line 374 prOutcomeMetric.fresh,
isHistoricalReplayRunFresh at line 293, historicalReplayMetric construction around line 410)
packages/loopover-engine/src/calibration-dashboard.ts (lines 42-48 sourceRow)
packages/loopover-engine/test/phase7-calibration-loop.test.ts
test/unit/calibration-dashboard.test.ts
- Milestone: Miner Wave 4.6 — AMS Hardening Round 3
Context
packages/loopover-engine/src/phase7-calibration-loop.ts'scomputePhase7CalibrationLoopbuilds twoCalibrationSourceMetricobjects, one per calibration source.historicalReplayMetric.fresh(aroundline 410) is a real computation, gated through
isHistoricalReplayRunFreshagainstobservedAt/config.replayFreshnessMaxAgeHours.prOutcomeMetric.fresh(line 374) is a hardcodedconstant:
freshis alwaystrueregardless ofobservedAt— even whenobservedAtisnullbecause notimestamp was supplied at all.
CalibrationSourceMetric.freshis a single shared field usedidentically for both sources by
calibration-dashboard.ts'ssourceRow(lines 42-48), which rendersdetail: n=${sampleSize} · ${fresh ? "fresh" : "stale"}for both the "Historical replay" and "PRoutcome" dashboard rows. The practical effect: the "PR outcome" row can never show "stale" no matter
how old the underlying
pr_outcomesample actually is, silently misleading whoever reads thedashboard about how current that number is.
test/unit/phase7-calibration-loop.test.ts(grepped for"fresh") never asserts a value forprOutcomeMetric.fresh— the constant has zero test coverage ofits own correctness.
Requirements
prOutcomeMetric.freshthe same wayhistoricalReplayMetric.freshis computed: reuseisHistoricalReplayRunFresh(or extract a source-agnostic freshness helper if that reads better —the exported/internal function name is up to the implementation, but the freshness semantics must
be identical for both sources: same max-age comparison against
config.replayFreshnessMaxAgeHours,same handling of a missing/unparseable
observedAtas NOT fresh) againstprOutcomeMetric.observedAtand the samenow/configinputshistoricalReplayMetric.freshalready uses.
nullobservedAt(no timestamp supplied for thepr_outcomesource) must resolve tofresh: false, matching howisHistoricalReplayRunFreshalready treats a missing/invalidobservedAtfor the historical-replay source.prOutcomeDerived,accuracy, orsampleSizecomputation, and do not changehistoricalReplayMetric's own freshness logic.Deliverables
prOutcomeMetric.freshistruewheninput.prOutcome.observedAtis withinconfig.replayFreshnessMaxAgeHoursofnow, andfalsewhen it is older, using the samecomparison
historicalReplayMetric.freshuses.prOutcomeMetric.freshisfalsewheninput.prOutcomeis present but itsobservedAtisnull/missing/unparseable (matching the historical-replay source's existing handling of theequivalent case).
calibration-dashboard.ts'ssourceRowrendering of the "PR outcome" row's freshness detailreflects the newly-computed value (no changes needed to
calibration-dashboard.tsitself ifthe fix is confined to
phase7-calibration-loop.ts— verify this is the case as part of thisissue, and if it is not, fix whatever additional wiring is needed to make the dashboard
actually display the real freshness).
All three Deliverables are required in this one PR — there is no narrower scope for this issue.
Test Coverage Requirements
packages/loopover-engine/src/**is measured by Codecov via two separate uploads whose hits areunioned — root
test/**ANDpackages/loopover-engine/test/**. Add regression tests topackages/loopover-engine/test/phase7-calibration-loop.test.ts(this file's existing test location)asserting
prOutcomeMetric.freshfor: a recentobservedAt(within the max-age window) →true; astale
observedAt(older than the max-age window) →false; and anull/missingobservedAt→false. Also add or extend a test intest/unit/calibration-dashboard.test.ts(root-level)confirming the "PR outcome" dashboard row renders
"stale"whenprOutcomeMetric.freshisfalse(previously impossible to test meaningfully, since the value was always
true). Target 100% branchcoverage of the new freshness computation.
Expected Outcome
The calibration dashboard's "PR outcome" row can genuinely show "stale," matching the accuracy the
"Historical replay" row already has, so an operator reading the dashboard gets a truthful signal
about how current the
pr_outcomecalibration sample actually is.Links & Resources
packages/loopover-engine/src/phase7-calibration-loop.ts(line 374prOutcomeMetric.fresh,isHistoricalReplayRunFreshat line 293,historicalReplayMetricconstruction around line 410)packages/loopover-engine/src/calibration-dashboard.ts(lines 42-48sourceRow)packages/loopover-engine/test/phase7-calibration-loop.test.tstest/unit/calibration-dashboard.test.ts