feat(calibration): Pareto-floor comparator between two BacktestScoreReports

[kai392]

Summary

• Add packages/loopover-engine/src/calibration/backtest-compare.ts with BacktestComparison and compareBacktestScores(baseline, candidate): the dual-axis no-regression comparator for a baseline-vs-candidate pair of BacktestScoreReports (calibration: pure confusion-matrix scorer for a candidate rule classifier against the backtest corpus #8085) — a candidate rule change may not regress on ANY measured axis even while improving another; "trading one axis for the other" is a regression, not a net win.
• Per-axis rules exactly as specified: strictly-less = regressed, strictly-greater = improved, equal = neither; an axis with null on either side is excluded from both lists (insufficient decided data is never treated as 0 or as "no change"). verdict is "regressed" whenever even a single axis regressed — the Pareto-floor rule, deliberately NOT a weighted/averaged score — else "improved" when any axis improved, else "unchanged".
• Mismatched ruleIds throw a plain Error naming both rules — comparing scores for two different rules is a caller bug, not a valid comparison.
• Barrel export added to packages/loopover-engine/src/index.ts on its own line immediately after the backtest-score.js line, per the issue's deliverable.

Closes #8086

Test plan

• packages/loopover-engine/test/backtest-compare.test.ts (node:test, per the issue's deliverable): both axes improve → improved with empty regressedAxes; the Pareto-floor case — one axis improves while the other regresses → regressed; null on either side excludes the axis from both lists; equal axes land in neither list → unchanged; mismatched ruleId throws naming both rules
• test/unit/backtest-compare-engine.test.ts (vitest, importing the engine source): same cases plus the both-axes-regressed case — this is what exercises the .ts for codecov/patch, since @loopover/engine imports resolve to dist
• Local coverage on the new file: 11/11 lines, 14/14 branches (rule-mismatch guard both sides, both null-exclusion sides, all three per-axis outcomes, all three verdict arms)
• npm run test --workspace @loopover/engine green; npm run typecheck green
• CI validate

[superagent-security]

Superagent did not find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.01%. Comparing base (d07b057) to head (05ab1b5).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8108      +/-   ##
==========================================
- Coverage   92.01%   92.01%   -0.01%     
==========================================
  Files         757      758       +1     
  Lines       77228    77240      +12     
  Branches    23346    23351       +5     
==========================================
+ Hits        71061    71069       +8     
  Misses       5061     5061              
- Partials     1106     1110       +4     

Flag	Coverage Δ
shard-1	59.56% <0.00%> (+4.47%)	⬆️
shard-2	51.08% <0.00%> (-0.08%)	⬇️
shard-3	50.45% <100.00%> (-5.73%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...oopover-engine/src/calibration/backtest-compare.ts	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

[loopover-orb]

Tip

✅ LoopOver review result - approve/merge recommended

_{Review updated: 2026-07-22 23:01:06 UTC}

4 files · 1 AI reviewer · no blockers · readiness 88/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
The AI review returned non-blocking notes for this change but did not include a separate narrative summary. Review the nits below before deciding this PR.

Nits — 3 non-blocking

• The PR body claims to close calibration: Pareto-floor comparator between two BacktestScoreReports #8086, but per repo convention every contributor PR must close or clearly link an eligible open issue — confirm calibration: Pareto-floor comparator between two BacktestScoreReports #8086 is actually open and eligible since I cannot verify issue status from the diff alone.
• packages/loopover-engine/src/calibration/backtest-compare.ts:10-11 defines ComparisonAxis but the exported BacktestComparison type re-spells the same union inline (Array<"precision" | "recall">) instead of reusing ComparisonAxis[], which is a minor DRY miss.
• Consider using `regressedAxes: ComparisonAxis[]` and `improvedAxes: ComparisonAxis[]` in the BacktestComparison type instead of repeating the literal union, per backtest-compare.ts:15-16.

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#8086
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	❌ 8/20	High review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 93 registered-repo PR(s), 49 merged, 2 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	kai392; Gittensor profile; 93 PR(s), 2 issue(s).
Improvement	✅ Minor	risk: clean · value: minor · LLM: moderate

Linked issue satisfaction

Addressed
The diff adds backtest-compare.ts implementing BacktestComparison and compareBacktestScores exactly per spec (ruleId mismatch throws with both IDs, null-exclusion per axis, strict less/greater rules, and regressed verdict whenever any axis regresses regardless of improvement elsewhere), adds the barrel export in the correct position, and includes the required Pareto-floor mixed-improve/regress tes

Review context

• Author: kai392
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: JavaScript, Kotlin, Perl, Python, TypeScript, Vue
• Official Gittensor activity: 93 PR(s), 2 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Start here: Add a concise scope and risk note.
• Then work through the remaining 1 step in the Signals table above.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[loopover-orb]

LoopOver approves — the gate is satisfied and CI is green.