What's wrong
Three defects on the public verifiability surface, verified live against production on 2026-07-28.
1. The reproducibility freeze point commits to an empty corpus
GET https://api.loopover.ai/v1/public/stats reports:
"latestBacktestRun": { "corpusChecksum": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945", "at": "2026-07-28T21:18:14.855Z" }That value is checksumCases([]) — SHA-256 over the canonicalized empty case list (printf '[]' | shasum -a 256 reproduces it exactly). The /fairness page renders it as a "Reproducibility freeze point", and GET /v1/public/eval-scores publishes two records committed to it carrying trust.tier: "reproducible" alongside decided: 460, confirmed: 287.
A hash over zero cases is byte-identical for every rule, every window and every deployment, so it commits to nothing a reader could re-derive the scores from. The scores and the commitment also come from different datasets: decided/confirmed are computed live from signal.human_override:* audit events over a trailing 90 days, while the checksum is one frozen value from the most recent backtest run.
Why the corpus is empty: .github/workflows/calibration-advisory.yml runs backtest-corpus-export.ts hardcoded to --rule-id linked_issue_scope_mismatch. That rule's signal.rule_fired events are only written on a path gated on linkedIssueSatisfactionGateMode === "block", which defaults to "off", so buildBacktestCorpus returns [] and nothing refuses to publish the result.
2. The walkthrough's stats reference 404s
apps/loopover-ui/content/docs/verify-this-review.mdx step 4 cites a bare `/v1/public/stats`. A reader on loopover.ai/docs/verify-this-review resolves that to https://loopover.ai/v1/public/stats, which the UI worker does not serve — only api.loopover.ai does. Sibling docs (what-you-can-verify.mdx) already use the absolute host.
3. The walkthrough overstates who can run it
The page says "Nothing posts anywhere, nothing needs an API key" and what-you-can-verify.mdx says "Anyone can do this for public repositories". But step 1 (backtest-corpus-export.ts --remote) and the track-record command shell out to wrangler d1 execute --remote, reading the deployment's own Cloudflare D1 with that deployment's credentials. A third party cannot run step 1 at all. There is no HTTP corpus-export route — any unmatched /v1/* path returns 401 {"error":"unauthorized"} from the catch-all auth middleware, which is what makes a guessed corpus URL look auth-gated rather than absent.
Scope of this issue
Stop publishing a commitment that verifies nothing, and make the walkthrough's host references and audience claims accurate.
Explicitly out of scope (each needs its own decision and issue): publishing an actual anonymous corpus snapshot (BacktestCase carries targetKey and firing metadata, so it needs a redaction design before anything is published), making corpusChecksum per-rule and window-bound, and backfilling the missing rule_fired signal.
What's wrong
Three defects on the public verifiability surface, verified live against production on 2026-07-28.
1. The reproducibility freeze point commits to an empty corpus
GET https://api.loopover.ai/v1/public/statsreports:That value is
checksumCases([])— SHA-256 over the canonicalized empty case list (printf '[]' | shasum -a 256reproduces it exactly). The/fairnesspage renders it as a "Reproducibility freeze point", andGET /v1/public/eval-scorespublishes two records committed to it carryingtrust.tier: "reproducible"alongsidedecided: 460, confirmed: 287.A hash over zero cases is byte-identical for every rule, every window and every deployment, so it commits to nothing a reader could re-derive the scores from. The scores and the commitment also come from different datasets:
decided/confirmedare computed live fromsignal.human_override:*audit events over a trailing 90 days, while the checksum is one frozen value from the most recent backtest run.Why the corpus is empty:
.github/workflows/calibration-advisory.ymlrunsbacktest-corpus-export.tshardcoded to--rule-id linked_issue_scope_mismatch. That rule'ssignal.rule_firedevents are only written on a path gated onlinkedIssueSatisfactionGateMode === "block", which defaults to"off", sobuildBacktestCorpusreturns[]and nothing refuses to publish the result.2. The walkthrough's stats reference 404s
apps/loopover-ui/content/docs/verify-this-review.mdxstep 4 cites a bare`/v1/public/stats`. A reader onloopover.ai/docs/verify-this-reviewresolves that tohttps://loopover.ai/v1/public/stats, which the UI worker does not serve — onlyapi.loopover.aidoes. Sibling docs (what-you-can-verify.mdx) already use the absolute host.3. The walkthrough overstates who can run it
The page says "Nothing posts anywhere, nothing needs an API key" and
what-you-can-verify.mdxsays "Anyone can do this for public repositories". But step 1 (backtest-corpus-export.ts --remote) and the track-record command shell out towrangler d1 execute --remote, reading the deployment's own Cloudflare D1 with that deployment's credentials. A third party cannot run step 1 at all. There is no HTTP corpus-export route — any unmatched/v1/*path returns401 {"error":"unauthorized"}from the catch-all auth middleware, which is what makes a guessed corpus URL look auth-gated rather than absent.Scope of this issue
Stop publishing a commitment that verifies nothing, and make the walkthrough's host references and audience claims accurate.
Explicitly out of scope (each needs its own decision and issue): publishing an actual anonymous corpus snapshot (
BacktestCasecarriestargetKeyand firingmetadata, so it needs a redaction design before anything is published), makingcorpusChecksumper-rule and window-bound, and backfilling the missingrule_firedsignal.