feat(calibration): pure confusion-matrix scorer for candidate rule classifiers

[kai392]

Summary

• Add packages/loopover-engine/src/calibration/backtest-score.ts with BacktestScoreReport and scoreBacktest(ruleId, cases, classify): replays a caller-supplied synchronous classifier over a labeled BacktestCase[] corpus (calibration: pure BacktestCase corpus builder from RuleFiredEvent/HumanOverrideEvent pairs #8083) and scores it against the real human verdicts — the "would the proposed rule have gotten more of these right?" primitive, mirroring src/review/auto-tune.ts's GateEvalRow confusion-matrix shape at a backtest-replay grain.
• Convention (pinned in a comment attached to the type, per the issue): "reversed" is the positive class — correctly predicting that a firing was wrong is the true positive.
• precision/recall are null when their denominator is 0, never coerced to 0 or 1 — the same "unknown stays unknown" discipline as RulePrecisionReport.precision in signal-tracking.ts; cases for a different ruleId are excluded from every count, caseCount included, mirroring computeRulePrecision's defensive override filter.
• Barrel export added to packages/loopover-engine/src/index.ts on its own line immediately after the backtest-corpus.js line, per the issue's deliverable.

Closes #8085

Test plan

• packages/loopover-engine/test/backtest-score.test.ts (node:test, per the issue's deliverable): all-correct classifier → precision 1 / recall 1; all-wrong classifier → precision 0 / recall 0 with the misses in the right cells; mixed classifier accumulating all four cells; empty corpus → zero counts with both ratios null; separate explicit null-precision (classifier never predicts reversed) and null-recall (no reversed labels) cases; non-matching ruleId excluded from every count including caseCount
• test/unit/backtest-score-engine.test.ts (vitest, importing the engine source): same cases — this is what exercises the .ts for codecov/patch, since @loopover/engine imports resolve to dist (the trap that has already failed prior PRs in this issue family)
• Local coverage on the new file: 15/15 lines, 12/12 branches (all four matrix cells + both nullable-ratio sides + both rule-filter sides)
• npm run test --workspace @loopover/engine green; npm run typecheck green
• CI validate

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.00%. Comparing base (423a3d1) to head (c470097).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8098      +/-   ##
==========================================
- Coverage   92.01%   92.00%   -0.01%     
==========================================
  Files         755      756       +1     
  Lines       77201    77218      +17     
  Branches    23337    23344       +7     
==========================================
+ Hits        71034    71047      +13     
  Misses       5061     5061              
- Partials     1106     1110       +4     

Flag	Coverage Δ
shard-1	53.65% <100.00%> (-5.94%)	⬇️
shard-2	57.06% <0.00%> (+5.97%)	⬆️
shard-3	51.55% <0.00%> (+1.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
.../loopover-engine/src/calibration/backtest-score.ts	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

[loopover-orb]

Tip

✅ LoopOver review result - approve/merge recommended

_{Review updated: 2026-07-22 22:39:24 UTC}

4 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
The AI review returned non-blocking notes for this change but did not include a separate narrative summary. Review the nits below before deciding this PR.

Nits — 2 non-blocking

• Consider extracting the shared corpusCase() test helper (duplicated verbatim in both test files) into a small shared test-fixtures module if more backtest-scoring tests are added in follow-up PRs.
• The barrel export in index.ts:167 is correctly placed immediately after backtest-corpus.js per the issue's deliverable — no change needed there.

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#8085
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 93 registered-repo PR(s), 49 merged, 2 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	kai392; Gittensor profile; 93 PR(s), 2 issue(s).
Improvement	✅ Minor	risk: clean · value: minor · LLM: moderate

Review context

• Author: kai392
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: JavaScript, Kotlin, Perl, Python, TypeScript, Vue
• Official Gittensor activity: 93 PR(s), 2 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Keep the PR focused and include validation evidence before maintainer review.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[loopover-orb]

LoopOver approves — the gate is satisfied and CI is green.