Skip to content

feat(engine): render backtest score and comparison reports as deterministic Markdown - #8117

Closed
cleanjunc wants to merge 1 commit into
JSONbored:mainfrom
cleanjunc:feat/8088-backtest-report
Closed

feat(engine): render backtest score and comparison reports as deterministic Markdown#8117
cleanjunc wants to merge 1 commit into
JSONbored:mainfrom
cleanjunc:feat/8088-backtest-report

Conversation

@cleanjunc

Copy link
Copy Markdown
Contributor

Summary

Closes #8088.

Adds packages/loopover-engine/src/calibration/backtest-report.ts — the deterministic Markdown "receipt" renderers for backtest results: renderBacktestScoreReport (rule ID, case count, all four confusion-matrix counts, precision/recall with null rendered as the literal N/A — never 0, null, or an empty cell) and renderBacktestComparison (regressed and improved axes under clearly separated headings — an axis only ever appears under its own section — plus a closing verdict line whose regressed form contains the literal word REGRESSED and states do not merge, the exact string a future automated consumer can match without re-deriving the comparison).

Both functions are pure (string in → string out; no IO, no wall-clock reads) and byte-identically deterministic — pinned by test. When regressedAxes is empty the output contains no Regressed section at all, so nothing can read as regressed. The barrel export lands on its own line immediately after the backtest-compare.js export in packages/loopover-engine/src/index.ts, exactly where the issue specifies.

Tests land in both suites: packages/loopover-engine/test/backtest-report.test.ts (the issue's mandated file: an exact-string snapshot for a non-null fixture, the N/A null path, the REGRESSED literal + section-separation case, improved-with-no-regressed-claim, unchanged verdict, byte-identical determinism) plus test/unit/backtest-report.test.ts, the root-side vitest coverage twin importing the engine src directly — the pattern this module's merged #8083/#8086-era tests established (the engine's node --test suite runs against dist/, outside root vitest's coverage instrumentation, so the twin is what makes the file's coverage visible to codecov/patch). All three verdict branches and both null/non-null render paths are covered.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • This module (source + both test files) was validated in full against the spec's shapes immediately before the dependency (calibration: Pareto-floor comparator between two BacktestScoreReports #8086) landed: npm run build --workspace @loopover/engine green, engine node --test 6/6, root vitest coverage twin 4/4, npm run typecheck 0 errors. To beat the batch's scoop window the final assembly was pushed without re-running the full local chain against the freshly-merged base — the only delta is the dependency's own merged, CI-green code, and real CI now re-runs everything here. Unchecked boxes above are exactly the ones not re-run post-rebase, stated honestly.
  • workers/mcp/ui/audit/actionlint: untouched surfaces — one new pure engine file, one barrel line, two test files; zero dependency changes.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

The three unchecked Safety boxes are N/A: no auth/session/UI change — a pure, additive engine module plus tests.

UI Evidence

N/A — no UI change (pure engine module).

Notes

  • The Verdict: REGRESSED — do not merge literal is the machine-matchable hook the epic's follow-up CI wiring consumes; the renderer never re-derives any comparison logic (the axis lists and verdict carry everything).

@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.01%. Comparing base (c52d862) to head (be6f2df).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8117      +/-   ##
==========================================
- Coverage   92.02%   92.01%   -0.01%     
==========================================
  Files         759      760       +1     
  Lines       77328    77348      +20     
  Branches    23376    23383       +7     
==========================================
+ Hits        71159    71175      +16     
  Misses       5061     5061              
- Partials     1108     1112       +4     
Flag Coverage Δ
shard-1 57.40% <0.00%> (+3.42%) ⬆️
shard-2 51.09% <0.00%> (-3.72%) ⬇️
shard-3 54.46% <100.00%> (+0.59%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...loopover-engine/src/calibration/backtest-report.ts 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

@JSONbored
JSONbored force-pushed the feat/8088-backtest-report branch from d820540 to be6f2df Compare July 22, 2026 23:18
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#8116 was opened first, which links to close the same issue.

@JSONbored JSONbored closed this Jul 22, 2026
@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

calibration: render a backtest score/comparison report as Markdown

2 participants