feat(engine): render backtest score and comparison reports as deterministic Markdown - #8117
Closed
cleanjunc wants to merge 1 commit into
Closed
feat(engine): render backtest score and comparison reports as deterministic Markdown#8117cleanjunc wants to merge 1 commit into
cleanjunc wants to merge 1 commit into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8117 +/- ##
==========================================
- Coverage 92.02% 92.01% -0.01%
==========================================
Files 759 760 +1
Lines 77328 77348 +20
Branches 23376 23383 +7
==========================================
+ Hits 71159 71175 +16
Misses 5061 5061
- Partials 1108 1112 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
|
JSONbored
force-pushed
the
feat/8088-backtest-report
branch
from
July 22, 2026 23:18
d820540 to
be6f2df
Compare
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #8088.
Adds
packages/loopover-engine/src/calibration/backtest-report.ts— the deterministic Markdown "receipt" renderers for backtest results:renderBacktestScoreReport(rule ID, case count, all four confusion-matrix counts, precision/recall withnullrendered as the literalN/A— never0,null, or an empty cell) andrenderBacktestComparison(regressed and improved axes under clearly separated headings — an axis only ever appears under its own section — plus a closing verdict line whose regressed form contains the literal wordREGRESSEDand statesdo not merge, the exact string a future automated consumer can match without re-deriving the comparison).Both functions are pure (string in → string out; no IO, no wall-clock reads) and byte-identically deterministic — pinned by test. When
regressedAxesis empty the output contains no Regressed section at all, so nothing can read as regressed. The barrel export lands on its own line immediately after thebacktest-compare.jsexport inpackages/loopover-engine/src/index.ts, exactly where the issue specifies.Tests land in both suites:
packages/loopover-engine/test/backtest-report.test.ts(the issue's mandated file: an exact-string snapshot for a non-null fixture, theN/Anull path, theREGRESSEDliteral + section-separation case, improved-with-no-regressed-claim, unchanged verdict, byte-identical determinism) plustest/unit/backtest-report.test.ts, the root-side vitest coverage twin importing the engine src directly — the pattern this module's merged #8083/#8086-era tests established (the engine'snode --testsuite runs againstdist/, outside root vitest's coverage instrumentation, so the twin is what makes the file's coverage visible tocodecov/patch). All three verdict branches and both null/non-null render paths are covered.Scope
type(scope): short summaryConventional Commit format, for examplefix(api): restore profile access checks.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Closes #123) — a linked open issue is required for every contributor PR.Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coveragelocally;codecov/patchrequires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.npm run test:workersnpm run build:mcpnpm run test:mcp-packnpm run ui:openapi:checknpm run ui:lintnpm run ui:typechecknpm run ui:buildnpm audit --audit-level=moderateIf any required check was skipped, explain why:
npm run build --workspace @loopover/enginegreen, enginenode --test6/6, root vitest coverage twin 4/4,npm run typecheck0 errors. To beat the batch's scoop window the final assembly was pushed without re-running the full local chain against the freshly-merged base — the only delta is the dependency's own merged, CI-green code, and real CI now re-runs everything here. Unchecked boxes above are exactly the ones not re-run post-rebase, stated honestly.Safety
UI Evidencesection below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.The three unchecked Safety boxes are N/A: no auth/session/UI change — a pure, additive engine module plus tests.
UI Evidence
N/A — no UI change (pure engine module).
Notes
Verdict: REGRESSED — do not mergeliteral is the machine-matchable hook the epic's follow-up CI wiring consumes; the renderer never re-derives any comparison logic (the axis lists and verdict carry everything).