You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Review-decision accuracy is already computed and already public — computeGateEval (src/review/parity.ts:88-169) joins every gate decision (the predicted merge/close/hold verdict) against the realized pr_outcome once a PR actually closes, reversal-discounted via recordReversalSignals (src/review/outcomes-wire.ts:402-471, catching both a contributor reopening a bot-closed PR and someone reverting a bot-merged PR). This already powers the homepage's live "Decision accuracy %" (shipped via #1059/#1567) and an existing auto-tune circuit breaker (runSelfTuneBreaker, src/review/outcomes-wire.ts:499-573) that self-tightens per-repo if precision drops below a floor.
What's missing is history. The current public number is a live snapshot computed at request time — there is no persisted time series, so there's no way to show (or even internally verify) whether accuracy is improving, stable, or degrading over weeks/months. This is distinct from #4261/#2352, which track the miner's own self-review calibration (packages/gittensory-engine/src/phase7-calibration-loop.ts, a different subsystem, different audience) — this issue is about the maintainer-facing review engine's own accuracy trend.
Requirements
Add a periodic (weekly, matching the existing rollup cadence precedent) snapshot job that calls computeGateEval and persists the result — combined precision, per-verdict-type precision, sample size, and the reversal count/rate — into a new small rollup table (or extends the pr_handling_stats rollup already described in feat(stats): public proof-of-power — lifetime PRs-handled + accuracy counter on homepage hero #1059's deliverables, if that table already exists with room for these fields — check before adding a new one).
Never let a low-sample-size week produce a misleading precision number: apply the same kind of "insufficient history" guard feat(signals): add queue pressure and review velocity trend windows #110 already uses for its trend windows, and consider carrying forward computeGateEval's existing sample-size field into the guard condition.
Extend GET /v1/public/stats with the accuracy trend (reuse the existing response shape, per the parent epic's guidance).
Add a trend chart to the homepage//stats page alongside the existing live accuracy number — the live number stays as the "right now" headline, the trend is the new "is this improving" story.
Document the accuracy definition publicly and precisely (reversal-grounded: a decision is "correct" unless a human later overrides/reverts it) — feat(stats): public proof-of-power — lifetime PRs-handled + accuracy counter on homepage hero #1059 already required this for the live number; make sure the trend's methodology is described identically so the two never appear inconsistent to a reader.
Insufficient-history state handled explicitly, not silently rendered as a real number
GET /v1/public/stats extended with the accuracy trend; OpenAPI regenerated
Homepage//stats page shows an accuracy-over-time chart alongside the existing live number
Public methodology write-up matches the existing reversal-grounded definition exactly
Tests: snapshot computation, low-sample guard, public endpoint shape
Expected outcome
"Decision accuracy %" stops being a single live number and becomes a real, honest trend line the community (and we, internally) can watch over time — reusing the accuracy computation that already exists rather than inventing a second one.
References
src/review/parity.ts:88-169 (computeGateEval, the existing accuracy computation — reuse, don't duplicate)
Part of #4445.
Context
Review-decision accuracy is already computed and already public —
computeGateEval(src/review/parity.ts:88-169) joins every gate decision (the predicted merge/close/hold verdict) against the realizedpr_outcomeonce a PR actually closes, reversal-discounted viarecordReversalSignals(src/review/outcomes-wire.ts:402-471, catching both a contributor reopening a bot-closed PR and someone reverting a bot-merged PR). This already powers the homepage's live "Decision accuracy %" (shipped via #1059/#1567) and an existing auto-tune circuit breaker (runSelfTuneBreaker,src/review/outcomes-wire.ts:499-573) that self-tightens per-repo if precision drops below a floor.What's missing is history. The current public number is a live snapshot computed at request time — there is no persisted time series, so there's no way to show (or even internally verify) whether accuracy is improving, stable, or degrading over weeks/months. This is distinct from #4261/#2352, which track the miner's own self-review calibration (
packages/gittensory-engine/src/phase7-calibration-loop.ts, a different subsystem, different audience) — this issue is about the maintainer-facing review engine's own accuracy trend.Requirements
computeGateEvaland persists the result — combined precision, per-verdict-type precision, sample size, and the reversal count/rate — into a new small rollup table (or extends thepr_handling_statsrollup already described in feat(stats): public proof-of-power — lifetime PRs-handled + accuracy counter on homepage hero #1059's deliverables, if that table already exists with room for these fields — check before adding a new one).computeGateEval's existing sample-size field into the guard condition.GET /v1/public/statswith the accuracy trend (reuse the existing response shape, per the parent epic's guidance)./statspage alongside the existing live accuracy number — the live number stays as the "right now" headline, the trend is the new "is this improving" story.Deliverables
GET /v1/public/statsextended with the accuracy trend; OpenAPI regenerated/statspage shows an accuracy-over-time chart alongside the existing live numberExpected outcome
"Decision accuracy %" stops being a single live number and becomes a real, honest trend line the community (and we, internally) can watch over time — reusing the accuracy computation that already exists rather than inventing a second one.
References
src/review/parity.ts:88-169(computeGateEval, the existing accuracy computation — reuse, don't duplicate)src/review/outcomes-wire.ts:402-471(recordReversalSignals)src/review/outcomes-wire.ts:499-573(runSelfTuneBreaker, the existing live consumer of this same computation)src/review/public-stats.ts:179-346,src/api/routes.ts:949-957(existing public stats endpoint to extend)pr_outcomeplumbing this trend builds on)Effort
M