Context
src/review/public-stats.ts's getPublicStats computes the homepage's global accuracyPct
(line 401) as accuracyPct(totals.merged, totals.closed, totals.reversed). totals.reversed is
accumulated only from loopover's own allowlisted repos (line 358), but totals.merged/
totals.closed are folded with the registered Orb fleet's aggregate counts first (lines 381-382,
via getOrbGlobalStats, which — per this file's own comment at line 378 — "has no reversal concept
at all"). accuracyPct's formula (line 151) is 1 - min(1, reversed / (merged + closed)). Because
the denominator (merged + closed) grows with the whole registered fleet while the numerator
(reversed) stays fixed to a handful of own-ledger repos, the published accuracy percentage trends
toward 100% as more self-hosted installations register, independent of the fleet's actual
merge/close reversal behavior — which this number is supposed to represent.
This file's header comment is otherwise very thorough about reasoning through this fold's
tradeoffs (explicitly noting reversals stay own-ledger-only), but doesn't address this specific
downstream skew on the published accuracyPct value.
Requirements
- The publicly-displayed global
accuracyPct must not be computable in a way where its numerator
(reversed count) and denominator (merged+closed count) are drawn from structurally different,
independently-growing populations. Acceptable fixes include (pick one, and document the choice
in a code comment next to accuracyPct's call site):
- Compute the global
accuracyPct from own-ledger totals only (i.e. call accuracyPct with the
pre-fold ownLedger-scoped merged/closed, not the fleet-folded totals.merged/totals.closed),
leaving the Orb-fleet merged/closed counts to inflate only the raw reviewed/handled/
minutesSaved totals as they do today (which have no numerator/denominator mismatch problem);
- Or, if
getOrbGlobalStats can be extended to also report a fleet-wide reversal count, fold
that into totals.reversed alongside merged/closed so numerator and denominator scale
together.
Do not silently drop the Orb-fleet fold from merged/closed/reviewed/handled entirely —
those aggregate totals are working as intended per the file's own comments; only the
accuracyPct computation's numerator/denominator mismatch needs fixing.
- Do not change
accuracyPct's own formula/signature, byProject's per-project accuracyPct
values (each project's is already computed from same-scope merged/closed/reversed — verify this
stays true), or any other field in the returned stats object.
Deliverables
Test Coverage Requirements
This repo's Codecov patch gate is 99%+ hard (branch-counted) on every changed line/branch in
src/**. The new regression test must exercise the real getPublicStats function (mocking
getOrbGlobalStats and the own-ledger disposition query) end-to-end, not just unit-test
accuracyPct in isolation with hand-picked numbers.
Expected Outcome
The homepage's publicly-displayed global accuracy percentage can no longer be pulled toward 100%
purely as an artifact of more self-hosted installations registering into the Orb fleet — it either
reflects the same own-ledger population its numerator has always used, or fleet-wide reversal data
that scales with the same denominator.
Links & Resources
src/review/public-stats.ts:151-163 (accuracyPct), :358 (own-ledger totals.reversed
accumulation), :376-401 (the Orb-fleet fold and the final accuracyPct call site),
getOrbGlobalStats (src/orb/outcomes.ts, confirm whether a reversal-count extension is already
tracked elsewhere before choosing option 2 above).
Context
src/review/public-stats.ts'sgetPublicStatscomputes the homepage's globalaccuracyPct(line 401) as
accuracyPct(totals.merged, totals.closed, totals.reversed).totals.reversedisaccumulated only from loopover's own allowlisted repos (line 358), but
totals.merged/totals.closedare folded with the registered Orb fleet's aggregate counts first (lines 381-382,via
getOrbGlobalStats, which — per this file's own comment at line 378 — "has no reversal conceptat all").
accuracyPct's formula (line 151) is1 - min(1, reversed / (merged + closed)). Becausethe denominator (
merged + closed) grows with the whole registered fleet while the numerator(
reversed) stays fixed to a handful of own-ledger repos, the published accuracy percentage trendstoward 100% as more self-hosted installations register, independent of the fleet's actual
merge/close reversal behavior — which this number is supposed to represent.
This file's header comment is otherwise very thorough about reasoning through this fold's
tradeoffs (explicitly noting reversals stay own-ledger-only), but doesn't address this specific
downstream skew on the published
accuracyPctvalue.Requirements
accuracyPctmust not be computable in a way where its numerator(reversed count) and denominator (merged+closed count) are drawn from structurally different,
independently-growing populations. Acceptable fixes include (pick one, and document the choice
in a code comment next to
accuracyPct's call site):accuracyPctfrom own-ledger totals only (i.e. callaccuracyPctwith thepre-fold
ownLedger-scoped merged/closed, not the fleet-foldedtotals.merged/totals.closed),leaving the Orb-fleet merged/closed counts to inflate only the raw
reviewed/handled/minutesSavedtotals as they do today (which have no numerator/denominator mismatch problem);getOrbGlobalStatscan be extended to also report a fleet-wide reversal count, foldthat into
totals.reversedalongsidemerged/closedso numerator and denominator scaletogether.
Do not silently drop the Orb-fleet fold from
merged/closed/reviewed/handledentirely —those aggregate totals are working as intended per the file's own comments; only the
accuracyPctcomputation's numerator/denominator mismatch needs fixing.accuracyPct's own formula/signature,byProject's per-projectaccuracyPctvalues (each project's is already computed from same-scope merged/closed/reversed — verify this
stays true), or any other field in the returned stats object.
Deliverables
totals.accuracyPctcomputation reworked so its numerator and denominator aredrawn from the same population (own-ledger-only per option 1, or fleet-wide-both per option 2)
file's existing house style of documenting non-obvious metric-composition choices
getOrbGlobalStatsmerged/closed count no longer produces
accuracyPcttrending toward 100 purely from thefleet fold — assert the value reflects only the reversal-comparable population chosen above
getPublicStats's other fields (reviewed,handled,minutesSaved,per-project
accuracyPct) continue to pass unchangedTest Coverage Requirements
This repo's Codecov patch gate is 99%+ hard (branch-counted) on every changed line/branch in
src/**. The new regression test must exercise the realgetPublicStatsfunction (mockinggetOrbGlobalStatsand the own-ledger disposition query) end-to-end, not just unit-testaccuracyPctin isolation with hand-picked numbers.Expected Outcome
The homepage's publicly-displayed global accuracy percentage can no longer be pulled toward 100%
purely as an artifact of more self-hosted installations registering into the Orb fleet — it either
reflects the same own-ledger population its numerator has always used, or fleet-wide reversal data
that scales with the same denominator.
Links & Resources
src/review/public-stats.ts:151-163(accuracyPct),:358(own-ledgertotals.reversedaccumulation),
:376-401(the Orb-fleet fold and the finalaccuracyPctcall site),getOrbGlobalStats(src/orb/outcomes.ts, confirm whether a reversal-count extension is alreadytracked elsewhere before choosing option 2 above).