feat(engine): render backtest score and comparison reports as Markdown - #8116
Conversation
JSONbored#8088) BacktestScoreReport (JSONbored#8085) and BacktestComparison (JSONbored#8086) are plain data with no human-readable rendering. Add calibration/backtest-report.ts: renderBacktestScoreReport (a Markdown table of the rule ID, case count, all four confusion-matrix counts, and precision/recall) and renderBacktestComparison (regressed axes under a Regressed heading, improved axes under a visually separate Improved heading -- an empty section is omitted entirely so nothing ever reads as regressed when it isn't -- plus a closing verdict line). Null precision/recall render as the literal N/A, never 0 or the word null, mirroring the reports' own null-is-not-zero discipline. The regressed closing line is exactly "Verdict: REGRESSED — do not merge." so the follow-up CI wiring can detect it by string match without re-implementing the comparison. Both functions are pure and byte-identical for identical input. Barrel export added directly after the backtest-compare line, per the issue's placement requirement. Tests in both suites (engine node:test deliverable + root vitest for the coverage gate): snapshot-exact table render, N/A for both null axes, the literal REGRESSED + do-not-merge line with section ordering asserted, improved-only with no regressed claim, unchanged with neither section, and byte-identical determinism for both renderers.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8116 +/- ##
==========================================
- Coverage 92.02% 92.01% -0.01%
==========================================
Files 759 760 +1
Lines 77328 77348 +20
Branches 23376 23382 +6
==========================================
+ Hits 71159 71175 +16
Misses 5061 5061
- Partials 1108 1112 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
Tip ✅ LoopOver review result - approve/merge recommendedReview updated: 2026-07-22 23:32:57 UTC
Review summary Nits — 5 non-blocking
Decision drivers
Context & advisory signals — never blocks the verdict
Linked issue satisfactionAddressed Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
Summary
BacktestScoreReport(calibration: pure confusion-matrix scorer for a candidate rule classifier against the backtest corpus #8085) andBacktestComparison(calibration: Pareto-floor comparator between two BacktestScoreReports #8086) are plain data with no human-readable rendering. Addspackages/loopover-engine/src/calibration/backtest-report.ts:renderBacktestScoreReport(a Markdown table of the rule ID, case count, all four confusion-matrix counts, and precision/recall) andrenderBacktestComparison(regressed axes under a Regressed heading, improved axes under a visually separate Improved heading — an empty section is omitted entirely, so nothing ever reads as regressed when it isn't — plus a closing verdict line).N/A— never0, the wordnull, or an empty cell — mirroring the reports' own null-is-not-zero discipline. The regressed closing line is exactlyVerdict: REGRESSED — do not merge.(containing the literal case-sensitive wordREGRESSEDand a do-not-merge statement), so the follow-up CI wiring can detect the regressed case by string match without re-implementing the comparison logic. Both functions are pure (string in, string out; no IO, no wall-clock reads) and byte-identical for byte-identical input.backtest-compare.jsline inpackages/loopover-engine/src/index.ts, exactly where the issue requires.Scope
type(scope): short summaryConventional Commit format, for examplefix(api): restore profile access checks.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Closes #123) — a linked open issue is required for every contributor PR.Validation
git diff --checknpm run typechecknpm run actionlintnpm run test:coveragelocally;codecov/patchrequires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.npm run test:workersnpm run build:mcpnpm run test:mcp-packnpm run ui:openapi:checknpm run ui:lintnpm run ui:typechecknpm run ui:buildnpm audit --audit-level=moderateIf any required check was skipped, explain why:
npm run build && npm run test --workspace @loopover/engine(all green, including the newbacktest-report.test.ts), the roottest/unit/backtest-report.test.ts(8 tests green), and the full rootnpm run typecheck. Verified per-diff-line coverage via lcov: every changed line and branch inbacktest-report.tsis covered by the root suite — the snapshot-exact table render,N/Afor both null axes, the literalREGRESSED+ do-not-merge line with section ordering asserted, improved-only with no regressed claim, unchanged with neither section, and byte-identical determinism for both renderers — and simulated the scoped-CI shard condition with the exact CI invocation (--changed=origin/main --coverage.all=false): the lcov is non-empty and contains the changed instrumented file.actionlint/workers/mcp/ui checks are untouched surfaces; CI runs them all.Safety
UI Evidencesection below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.UI Evidence
Not applicable — engine-only pure-function addition (no UI, docs, or extension surface touched).
Notes
packages/loopover-engine/test/backtest-report.test.tsis the issue's deliverable (node:test againstdist/), andtest/unit/backtest-report.test.tsimports the source directly so the repo's Codecov patch gate measures every changed line and branch — the engine workspace's ownnode --testrun does not feed Codecov.