Context
Parent: #8828 (Phase 1 — measurement; blocks the rest of the epic). The homepage and fleet analytics publish a single accuracy scalar. A scalar at unstated coverage is gameable (raising the hold rate mechanically raises it) and, at current decision volumes, has a confidence interval wider than the gap between 96% and 99.5% — so no tuning claim is currently verifiable.
Requirements
- Extend
computeFleetAnalytics (src/orb/analytics.ts) and the public stats payload (src/review/public-stats.ts) with, per window and per repo hash:
mergePrecision and closePrecision published separately (both already computed per-instance; surface them),coverage = (merge+close verdicts) / (merge+close+hold signals),- Wilson score intervals (never Wald — Wald degenerates near p→1) on each published proportion,
- the sample sizes behind every figure.
- Every published accuracy figure must carry its coverage: "X% at Y% coverage". The UI hero tile hint should include coverage.
- Add a power statement to the fairness page: by the rule of three, ≤0.5% error requires ~600 consecutive clean decisions; when the window is under-powered, render the goal-vs-measured distinction honestly rather than a bare point estimate.
- Cluster standard errors by repo (contributor clustering is a follow-up once per-actor data exists in an exportable form; do NOT add per-actor identity to the fleet pipeline — it is anonymized by design).
Test Coverage Requirements
99%+ patch coverage, branch-counted; Wilson interval math property-tested at the p→0 and p→1 edges.
Expected Outcome
No published accuracy number without coverage, interval, and sample size attached. The "raise holds to raise accuracy" gaming vector is visible on the same surface it would distort.
Context
Parent: #8828 (Phase 1 — measurement; blocks the rest of the epic). The homepage and fleet analytics publish a single accuracy scalar. A scalar at unstated coverage is gameable (raising the hold rate mechanically raises it) and, at current decision volumes, has a confidence interval wider than the gap between 96% and 99.5% — so no tuning claim is currently verifiable.
Requirements
computeFleetAnalytics(src/orb/analytics.ts) and the public stats payload (src/review/public-stats.ts) with, per window and per repo hash:mergePrecisionandclosePrecisionpublished separately (both already computed per-instance; surface them),coverage = (merge+close verdicts) / (merge+close+hold signals),Test Coverage Requirements
99%+ patch coverage, branch-counted; Wilson interval math property-tested at the p→0 and p→1 edges.
Expected Outcome
No published accuracy number without coverage, interval, and sample size attached. The "raise holds to raise accuracy" gaming vector is visible on the same surface it would distort.