⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue.
Context
Verified live on 2026-07-29 against api.loopover.ai:
GET /v1/public/stats -> rulePrecision.rules[0] = { ruleId: "ai_consensus_defect",
decided: 460, confirmed: 287, precision: 0.624 }
rulePrecision.latestBacktestRun = null
GET /v1/public/eval-corpus?ruleId=ai_consensus_defect
-> caseCount 460, truncated false,
checksum 470e234e8923461e8642807c46265f01393f6dc9e4e40e52363f41767b5c3c10
GET /v1/public/eval-scores -> { "records": [] }
The corpus is real, complete, and its checksum is independently re-derivable: running
checksumPublicEvalCorpus over the cases array exactly as downloaded reproduces 470e234e…. The 460
published cases also match the 460 decided in rulePrecision exactly — they are the same dataset.
Yet the records array is empty, because buildEvalScoreRecordsFromRulePrecision
(src/review/eval-score-records.ts) sources its commitment from precision.latestBacktestRun, which
loadPublicRulePrecision reads from a persisted calibration.*_backtest_run audit event. #9639 fixes the
in-Worker writer, but that writer runs inside a review pass — and hosted review execution is retired
(src/index.ts acks-and-drops isReviewExecutionJob messages when !isSelfHostedReviewRuntime(env)). So on
loopover.ai that event never exists, and #9639 alone leaves this surface empty.
The user-visible consequence is that apps/loopover-ui/content/docs/verify-this-review.mdx §4 tells an
anonymous reader to curl .../v1/public/eval-scores | jq '.records' and re-derive recordDigest. They get
[]. The doc explains the empty array as "the latest backtest run's corpus is empty" — which is now
inaccurate: the corpus is not empty, it is 460 cases and downloadable.
The fix
Commit each record to its own rule's public-corpus checksum — the exact bytes /v1/public/eval-corpus
serves for that ruleId — when no persisted backtest run supplies one. That is not a synthetic or placeholder
commitment (which #9639 explicitly forbids): it is a checksum over an artifact the reader can download and
re-hash, which is the property reproducible trust tier asserts.
This also fixes a latent per-rule bug in the current code: latestBacktestRun.corpusChecksum is a single
value stamped onto every rule's record. With more than one rule in rulePrecision.rules, every record but
one would commit to a different rule's corpus. Only one rule is published today, which is why it has not
surfaced.
Requirements
- The commitment for a rule's record must be the checksum of that rule's own corpus, never another rule's.
- Window alignment is load-bearing: the corpus endpoint and
PUBLIC_PRECISION_WINDOW_DAYS must agree, or the
record commits to a different slice than the one its decided/confirmed were computed over. Assert it.
- A truncated corpus (
truncated: true, at PUBLIC_EVAL_CORPUS_MAX_CASES) must NOT be published as a
commitment: the checksum would cover a subset while the score covers the whole window.
EMPTY_CORPUS_CHECKSUM must still be refused, for the reason eval-score-records.ts already documents.
- A persisted backtest run, when one exists, must still win — self-host must not silently change behaviour.
verify-this-review.mdx §4's explanation of an empty records array must be corrected: it currently
attributes it to an empty corpus, which is no longer the only cause.
Deliverables
Test Coverage Requirements
99%+ Codecov patch coverage, branch-counted, on src/**. Both arms of every changed conditional.
Expected Outcome
An anonymous reader can complete verify-this-review.mdx end to end against loopover.ai: fetch the corpus,
re-derive its checksum, fetch the records, confirm each record's recordDigest and confirm its
commitments.corpusChecksum equals the corpus they just hashed.
Links & Resources
src/review/eval-score-records.ts, src/review/public-eval-corpus.ts, src/review/public-rule-precision.ts,
src/api/routes.ts (the /v1/public/eval-scores handler), apps/loopover-ui/content/docs/verify-this-review.mdx.
Related: #9639, #9636, #9266, #9215.
Context
Verified live on 2026-07-29 against
api.loopover.ai:The corpus is real, complete, and its checksum is independently re-derivable: running
checksumPublicEvalCorpusover thecasesarray exactly as downloaded reproduces470e234e…. The 460published cases also match the 460
decidedinrulePrecisionexactly — they are the same dataset.Yet the records array is empty, because
buildEvalScoreRecordsFromRulePrecision(
src/review/eval-score-records.ts) sources its commitment fromprecision.latestBacktestRun, whichloadPublicRulePrecisionreads from a persistedcalibration.*_backtest_runaudit event. #9639 fixes thein-Worker writer, but that writer runs inside a review pass — and hosted review execution is retired
(
src/index.tsacks-and-dropsisReviewExecutionJobmessages when!isSelfHostedReviewRuntime(env)). So onloopover.ai that event never exists, and #9639 alone leaves this surface empty.
The user-visible consequence is that
apps/loopover-ui/content/docs/verify-this-review.mdx§4 tells ananonymous reader to
curl .../v1/public/eval-scores | jq '.records'and re-deriverecordDigest. They get[]. The doc explains the empty array as "the latest backtest run's corpus is empty" — which is nowinaccurate: the corpus is not empty, it is 460 cases and downloadable.
The fix
Commit each record to its own rule's public-corpus checksum — the exact bytes
/v1/public/eval-corpusserves for that
ruleId— when no persisted backtest run supplies one. That is not a synthetic or placeholdercommitment (which #9639 explicitly forbids): it is a checksum over an artifact the reader can download and
re-hash, which is the property
reproducibletrust tier asserts.This also fixes a latent per-rule bug in the current code:
latestBacktestRun.corpusChecksumis a singlevalue stamped onto every rule's record. With more than one rule in
rulePrecision.rules, every record butone would commit to a different rule's corpus. Only one rule is published today, which is why it has not
surfaced.
Requirements
PUBLIC_PRECISION_WINDOW_DAYSmust agree, or therecord commits to a different slice than the one its
decided/confirmedwere computed over. Assert it.truncated: true, atPUBLIC_EVAL_CORPUS_MAX_CASES) must NOT be published as acommitment: the checksum would cover a subset while the score covers the whole window.
EMPTY_CORPUS_CHECKSUMmust still be refused, for the reasoneval-score-records.tsalready documents.verify-this-review.mdx§4's explanation of an emptyrecordsarray must be corrected: it currentlyattributes it to an empty corpus, which is no longer the only cause.
Deliverables
rule's own checksum, on a deployment with no persisted backtest run.
/v1/public/eval-corpuspayload for thesame rule — the reader's actual workflow, end to end.
verify-this-review.mdx§4 updated to describe the real conditions for an empty array.Test Coverage Requirements
99%+ Codecov patch coverage, branch-counted, on
src/**. Both arms of every changed conditional.Expected Outcome
An anonymous reader can complete
verify-this-review.mdxend to end against loopover.ai: fetch the corpus,re-derive its checksum, fetch the records, confirm each record's
recordDigestand confirm itscommitments.corpusChecksumequals the corpus they just hashed.Links & Resources
src/review/eval-score-records.ts,src/review/public-eval-corpus.ts,src/review/public-rule-precision.ts,src/api/routes.ts(the/v1/public/eval-scoreshandler),apps/loopover-ui/content/docs/verify-this-review.mdx.Related: #9639, #9636, #9266, #9215.