Skip to content

feat(public-stats): measured per-rule precision + reproducibility freeze point on /v1/public/stats - #8241

Merged
JSONbored merged 1 commit into
mainfrom
feat/public-rule-precision
Jul 23, 2026
Merged

feat(public-stats): measured per-rule precision + reproducibility freeze point on /v1/public/stats#8241
JSONbored merged 1 commit into
mainfrom
feat/public-rule-precision

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

What

The #8230 trust surface (epic #8211 track G): a rulePrecision block on /v1/public/stats.

How

  • Per-rule precision over the trailing 90 days computed from the SAME human-verdict events the internal calibration reads — one number, no second computation path, so the public claim and the internal measurement can never diverge.
  • Sparse rules are null, never 0%: the public sample floor (10 decided) is deliberately stiffer than the internal trend's weekly floor — a public percentage carries more weight than an operator chart.
  • All three reversal shapes counted over the window (reopened / reverted / superseded) — the counted-against-ourselves number.
  • Reproducibility freeze point: the latest persisted backtest run's corpus checksum + timestamp (Research: is TEE hardware attestation necessary, or does a reproducible-backtest checksum approach cover the real threat? #8136's reproducible-verification conclusion, operationalized) — a run without a checksum can never shadow an older one that carries it.
  • Same surface, flag, manifest override, and cache as the sibling trends; OpenAPI schema extended + regenerated.
  • Boundary invariant suite: pins that no target key, repo, confidence, or private term can appear in the serialized payload; fail-safe degradation on a broken store pinned too.

Validation

Full npm run test:ci green (TEST_CI_EXIT=0, zero failed files); 100% line+branch on the new module; route-level assertions on the extended payload.

Closes #8230

…eze point on /v1/public/stats

The #8230 trust surface (epic #8211 track G): per-rule precision over the
trailing 90 days computed from the SAME human-verdict events the internal
calibration reads — one number, no second computation path. Sparse rules
report null below a public sample floor deliberately stiffer than the
internal trend's (a public percentage carries more weight than an operator
chart). All three reversal shapes count over the window — the
counted-against-ourselves number — and the latest persisted backtest run's
corpus checksum + timestamp surface as the freeze point a skeptic needs to
independently re-run the comparison (#8136's reproducibility conclusion,
operationalized). Aggregates and rule ids only: the invariant suite pins
that no target key, repo, confidence, or private term can appear.

Rides the existing /v1/public/stats surface: same flag, same manifest
override, same cache, OpenAPI schema extended and regenerated.

Closes #8230
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 23, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
loopover-ui 971712b Commit Preview URL

Branch Preview URL
Jul 23 2026, 02:05 PM

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

1 similar comment
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Bundle Report

Changes will increase total bundle size by 2.77kB (0.04%) ⬆️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
loopover-ui 7.41MB 2.77kB (0.04%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: loopover-ui

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/add-scalar-classes-yRGbxbi8.js (New) 2.17MB 2.17MB 100.0% 🚀
assets/tanstack-vendor-D-9xxwTs.js (New) 740.12kB 740.12kB 100.0% 🚀
openapi.json 2.05kB 528.12kB 0.39%
assets/docs.fumadocs-spike-api-reference-F0m6F0q-.js (New) 442.92kB 442.92kB 100.0% 🚀
assets/AgentScalarChatInterface.vue--XB86iL7.js (New) 201.71kB 201.71kB 100.0% 🚀
assets/modal-fPwGXEDf.js (New) 184.38kB 184.38kB 100.0% 🚀
assets/client-7CnoVmyL.js (New) 146.06kB 146.06kB 100.0% 🚀
assets/maintainer-panel-OHe8Wtfy.js (New) 78.99kB 78.99kB 100.0% 🚀
assets/ui-vendor-u2BLuUbg.js (New) 57.04kB 57.04kB 100.0% 🚀
assets/routes-DqHOu71N.js (New) 34.98kB 34.98kB 100.0% 🚀
assets/owner-panel-ONGUPQH1.js (New) 27.36kB 27.36kB 100.0% 🚀
assets/app-DLrK7DjB.js (New) 25.7kB 25.7kB 100.0% 🚀
assets/app.runs-DvqoU__l.js (New) 23.56kB 23.56kB 100.0% 🚀
assets/miner-panel-BDMgrRi9.js (New) 20.28kB 20.28kB 100.0% 🚀
assets/api._op-cgmxeCq3.js (New) 17.57kB 17.57kB 100.0% 🚀
assets/self-hosting-docs-audit-BrvtWujI.js (New) 16.61kB 16.61kB 100.0% 🚀
assets/app.index-raryYHSS.js (New) 15.62kB 15.62kB 100.0% 🚀
assets/docs._slug-hXc0I1ed.js (New) 15.22kB 15.22kB 100.0% 🚀
assets/playground-panel-B7p8tmbe.js (New) 14.49kB 14.49kB 100.0% 🚀
assets/app.audit-BZv2Wp4G.js (New) 10.11kB 10.11kB 100.0% 🚀
assets/app.config-generator-B8lm1Isl.js (New) 10.09kB 10.09kB 100.0% 🚀
assets/extension-JSjWS1tZ.js (New) 10.04kB 10.04kB 100.0% 🚀
assets/fairness-DEP-PUEB.js (New) 8.34kB 8.34kB 100.0% 🚀
assets/maintainers-DWae-n8I.js (New) 8.09kB 8.09kB 100.0% 🚀
assets/miners-CblF_Cf9.js (New) 7.91kB 7.91kB 100.0% 🚀
assets/agents-CiXiiG-_.js (New) 7.8kB 7.8kB 100.0% 🚀
assets/commands-panel-DrFgsjf_.js (New) 6.65kB 6.65kB 100.0% 🚀
assets/maintainer-workflow-BIvvwKAJ.js (New) 6.52kB 6.52kB 100.0% 🚀
assets/roadmap-DztXuchF.js (New) 6.33kB 6.33kB 100.0% 🚀
assets/digest-panel-OjuDiEPo.js (New) 6.18kB 6.18kB 100.0% 🚀
assets/repos._owner._repo.quality-C5eHyUeo.js (New) 6.14kB 6.14kB 100.0% 🚀
assets/docs.index-CyL9VXlm.js (New) 5.73kB 5.73kB 100.0% 🚀
assets/docs-nav-B3i9Qz_4.js (New) 5.53kB 5.53kB 100.0% 🚀
assets/api.index-D_pcCWS9.js (New) 4.7kB 4.7kB 100.0% 🚀
assets/docs-D7MLIk9l.js (New) 2.7kB 2.7kB 100.0% 🚀
assets/api-CP8KMyyB.js (New) 2.69kB 2.69kB 100.0% 🚀
assets/docs-page-DLRZXa8R.js (New) 2.1kB 2.1kB 100.0% 🚀
assets/table-DV8ecPPE.js (New) 1.75kB 1.75kB 100.0% 🚀
assets/app.workbench-CHJUWPpK.js (New) 1.58kB 1.58kB 100.0% 🚀
assets/session-FVvZiVlF.js (New) 1.45kB 1.45kB 100.0% 🚀
assets/tooltip-BBXAFs4d.js (New) 1.45kB 1.45kB 100.0% 🚀
assets/tabs-DF7Ctwds.js (New) 1.39kB 1.39kB 100.0% 🚀
assets/app.repos-CV7_DM5C.js (New) 1.07kB 1.07kB 100.0% 🚀
assets/input-f0-45aZI.js (New) 796 bytes 796 bytes 100.0% 🚀
assets/file-cog-DfmwlPkF.js (New) 758 bytes 758 bytes 100.0% 🚀
assets/app.maintainer-Ck60tcph.js (New) 502 bytes 502 bytes 100.0% 🚀
assets/sparkles-DC4nD0M_.js (New) 494 bytes 494 bytes 100.0% 🚀
assets/app.owner-BYmBpTdx.js (New) 474 bytes 474 bytes 100.0% 🚀
assets/app.commands-BgtoZ9Sy.js (New) 455 bytes 455 bytes 100.0% 🚀
assets/app.playground-Ceh3jcxA.js (New) 442 bytes 442 bytes 100.0% 🚀
assets/reveal-BYa0J_t0.js (New) 442 bytes 442 bytes 100.0% 🚀
assets/index-jYOyyQG2.js (New) 438 bytes 438 bytes 100.0% 🚀
assets/app.digest-tyqznl5s.js (New) 430 bytes 430 bytes 100.0% 🚀
assets/eye-off-BggPryEd.js (New) 430 bytes 430 bytes 100.0% 🚀
assets/app.miner-Cvdh8msy.js (New) 422 bytes 422 bytes 100.0% 🚀
assets/key-round-hzNdJVbX.js (New) 355 bytes 355 bytes 100.0% 🚀
assets/bot-DI2cFrJg.js (New) 328 bytes 328 bytes 100.0% 🚀
assets/trash-2-CJsmWUgX.js (New) 328 bytes 328 bytes 100.0% 🚀
assets/save-CIsFm35L.js (New) 327 bytes 327 bytes 100.0% 🚀
assets/git-pull-request-arrow-BlpCYM7-.js (New) 321 bytes 321 bytes 100.0% 🚀
assets/wrench-QrBgQWrx.js (New) 303 bytes 303 bytes 100.0% 🚀
assets/list-checks-COKNLuwn.js (New) 279 bytes 279 bytes 100.0% 🚀
assets/shield-CD8vOP_e.js (New) 272 bytes 272 bytes 100.0% 🚀
assets/workflow-Eg184qjf.js (New) 265 bytes 265 bytes 100.0% 🚀
assets/compass-B0T3wacV.js (New) 251 bytes 251 bytes 100.0% 🚀
assets/history-0yTe-cnQ.js (New) 237 bytes 237 bytes 100.0% 🚀
assets/activity-CSdjVMD0.js (New) 234 bytes 234 bytes 100.0% 🚀
assets/message-square-BXTTGpID.js (New) 233 bytes 233 bytes 100.0% 🚀
assets/lock-VQIMLPm4.js (New) 206 bytes 206 bytes 100.0% 🚀
assets/rotate-cw-BBsojun2.js (New) 201 bytes 201 bytes 100.0% 🚀
assets/play-B3grGX2-.js (New) 190 bytes 190 bytes 100.0% 🚀
assets/circle-check-BjM63g_a.js (New) 178 bytes 178 bytes 100.0% 🚀
assets/search-D0jWxHWX.js (New) 174 bytes 174 bytes 100.0% 🚀
assets/circle-Cpe2uP77.js (New) 130 bytes 130 bytes 100.0% 🚀
assets/add-scalar-classes-Vo6khwrj.js (Deleted) -2.17MB 0 bytes -100.0% 🗑️
assets/tanstack-vendor-DwRrbLVG.js (Deleted) -739.41kB 0 bytes -100.0% 🗑️
assets/docs.fumadocs-spike-api-reference-CeIfx7Ms.js (Deleted) -442.92kB 0 bytes -100.0% 🗑️
assets/AgentScalarChatInterface.vue-CLVJPor-.js (Deleted) -201.71kB 0 bytes -100.0% 🗑️
assets/modal-DsAHlH2e.js (Deleted) -184.38kB 0 bytes -100.0% 🗑️
assets/client-DeZupq4J.js (Deleted) -146.06kB 0 bytes -100.0% 🗑️
assets/maintainer-panel-Da1hS6--.js (Deleted) -78.99kB 0 bytes -100.0% 🗑️
assets/ui-vendor-CFGUHXJi.js (Deleted) -57.04kB 0 bytes -100.0% 🗑️
assets/routes-DWP0271E.js (Deleted) -34.98kB 0 bytes -100.0% 🗑️
assets/owner-panel-BjaJBR6l.js (Deleted) -27.36kB 0 bytes -100.0% 🗑️
assets/app-p3CcOTh2.js (Deleted) -25.7kB 0 bytes -100.0% 🗑️
assets/app.runs-DJfFC5Xm.js (Deleted) -23.56kB 0 bytes -100.0% 🗑️
assets/miner-panel-ChnLB8W6.js (Deleted) -20.28kB 0 bytes -100.0% 🗑️
assets/api._op-2Xbkr33X.js (Deleted) -17.57kB 0 bytes -100.0% 🗑️
assets/self-hosting-docs-audit-BjHcBMqx.js (Deleted) -16.61kB 0 bytes -100.0% 🗑️
assets/app.index-0L0WC_Ul.js (Deleted) -15.62kB 0 bytes -100.0% 🗑️
assets/docs._slug-B0jxlHRk.js (Deleted) -15.22kB 0 bytes -100.0% 🗑️
assets/playground-panel-BcFJdQ4M.js (Deleted) -14.49kB 0 bytes -100.0% 🗑️
assets/app.audit-BxC95U1i.js (Deleted) -10.11kB 0 bytes -100.0% 🗑️
assets/app.config-generator-0M_ShS5R.js (Deleted) -10.09kB 0 bytes -100.0% 🗑️
assets/extension-CG646fcg.js (Deleted) -10.04kB 0 bytes -100.0% 🗑️
assets/fairness-CgvurdWc.js (Deleted) -8.34kB 0 bytes -100.0% 🗑️
assets/maintainers-DlkK2BDr.js (Deleted) -8.09kB 0 bytes -100.0% 🗑️
assets/miners-PyAvgGtx.js (Deleted) -7.91kB 0 bytes -100.0% 🗑️
assets/agents-CRqWjdm7.js (Deleted) -7.8kB 0 bytes -100.0% 🗑️
assets/commands-panel-B2-ettNh.js (Deleted) -6.65kB 0 bytes -100.0% 🗑️
assets/maintainer-workflow-DSZqia8n.js (Deleted) -6.52kB 0 bytes -100.0% 🗑️
assets/roadmap-BULjprxu.js (Deleted) -6.33kB 0 bytes -100.0% 🗑️
assets/digest-panel-DfO3NOkX.js (Deleted) -6.18kB 0 bytes -100.0% 🗑️
assets/repos._owner._repo.quality-Cn1jT-wl.js (Deleted) -6.14kB 0 bytes -100.0% 🗑️
assets/docs.index-bOpitwAm.js (Deleted) -5.73kB 0 bytes -100.0% 🗑️
assets/docs-nav-BCkQfk2d.js (Deleted) -5.53kB 0 bytes -100.0% 🗑️
assets/api.index-BzVGLWsS.js (Deleted) -4.7kB 0 bytes -100.0% 🗑️
assets/docs-CxMPttKv.js (Deleted) -2.7kB 0 bytes -100.0% 🗑️
assets/api-Y7ZzDdvm.js (Deleted) -2.69kB 0 bytes -100.0% 🗑️
assets/docs-page-BLbF6c_t.js (Deleted) -2.1kB 0 bytes -100.0% 🗑️
assets/table-Bmoxombt.js (Deleted) -1.75kB 0 bytes -100.0% 🗑️
assets/app.workbench-CXo5OKwC.js (Deleted) -1.58kB 0 bytes -100.0% 🗑️
assets/session-C5vhxb3H.js (Deleted) -1.45kB 0 bytes -100.0% 🗑️
assets/tooltip-CKUC5NX0.js (Deleted) -1.45kB 0 bytes -100.0% 🗑️
assets/tabs-CHAA9MEu.js (Deleted) -1.39kB 0 bytes -100.0% 🗑️
assets/app.repos-R9xwbWC5.js (Deleted) -1.07kB 0 bytes -100.0% 🗑️
assets/input-B_hLTRC5.js (Deleted) -796 bytes 0 bytes -100.0% 🗑️
assets/file-cog-ejP82umm.js (Deleted) -758 bytes 0 bytes -100.0% 🗑️
assets/app.maintainer-Dup3PtkC.js (Deleted) -502 bytes 0 bytes -100.0% 🗑️
assets/sparkles-DrR0NRmB.js (Deleted) -494 bytes 0 bytes -100.0% 🗑️
assets/app.owner-Dn1TWmsp.js (Deleted) -474 bytes 0 bytes -100.0% 🗑️
assets/app.commands-CN-i7MKt.js (Deleted) -455 bytes 0 bytes -100.0% 🗑️
assets/app.playground-B_mTl7Gl.js (Deleted) -442 bytes 0 bytes -100.0% 🗑️
assets/reveal-DMe2hWBH.js (Deleted) -442 bytes 0 bytes -100.0% 🗑️
assets/index-RR9w12a0.js (Deleted) -438 bytes 0 bytes -100.0% 🗑️
assets/app.digest-CgZQZW5I.js (Deleted) -430 bytes 0 bytes -100.0% 🗑️
assets/eye-off-g0V65IOR.js (Deleted) -430 bytes 0 bytes -100.0% 🗑️
assets/app.miner-DPi3oPrK.js (Deleted) -422 bytes 0 bytes -100.0% 🗑️
assets/key-round-C4hjTnLs.js (Deleted) -355 bytes 0 bytes -100.0% 🗑️
assets/bot-BlQAovdP.js (Deleted) -328 bytes 0 bytes -100.0% 🗑️
assets/trash-2-BDrM3E6W.js (Deleted) -328 bytes 0 bytes -100.0% 🗑️
assets/save-D5sPicBi.js (Deleted) -327 bytes 0 bytes -100.0% 🗑️
assets/git-pull-request-arrow-CkB5BX9T.js (Deleted) -321 bytes 0 bytes -100.0% 🗑️
assets/wrench-BjfUBzUs.js (Deleted) -303 bytes 0 bytes -100.0% 🗑️
assets/list-checks-BGNNxImZ.js (Deleted) -279 bytes 0 bytes -100.0% 🗑️
assets/shield-CogAXx8V.js (Deleted) -272 bytes 0 bytes -100.0% 🗑️
assets/workflow-276JoZZ-.js (Deleted) -265 bytes 0 bytes -100.0% 🗑️
assets/compass-DXK91S_w.js (Deleted) -251 bytes 0 bytes -100.0% 🗑️
assets/history-C6FHXefi.js (Deleted) -237 bytes 0 bytes -100.0% 🗑️
assets/activity-B8vKFA17.js (Deleted) -234 bytes 0 bytes -100.0% 🗑️
assets/message-square-DWo1NJ_y.js (Deleted) -233 bytes 0 bytes -100.0% 🗑️
assets/lock-C9G7tJEz.js (Deleted) -206 bytes 0 bytes -100.0% 🗑️
assets/rotate-cw-DvDhAUJ1.js (Deleted) -201 bytes 0 bytes -100.0% 🗑️
assets/play-CI8-G74c.js (Deleted) -190 bytes 0 bytes -100.0% 🗑️
assets/circle-check-DJcjPIH_.js (Deleted) -178 bytes 0 bytes -100.0% 🗑️
assets/search-BHrhkAzI.js (Deleted) -174 bytes 0 bytes -100.0% 🗑️
assets/circle-DbHh_38M.js (Deleted) -130 bytes 0 bytes -100.0% 🗑️

@JSONbored JSONbored self-assigned this Jul 23, 2026
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.58%. Comparing base (8017db1) to head (971712b).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8241      +/-   ##
==========================================
- Coverage   92.11%   91.58%   -0.53%     
==========================================
  Files         779      780       +1     
  Lines       78430    78447      +17     
  Branches    23696    23697       +1     
==========================================
- Hits        72245    71848     -397     
- Misses       5062     5524     +462     
+ Partials     1123     1075      -48     
Flag Coverage Δ
shard-1 53.87% <36.84%> (-1.89%) ⬇️
shard-2 49.96% <100.00%> (+0.16%) ⬆️
shard-3 54.81% <73.68%> (-1.68%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/api/routes.ts 95.29% <100.00%> (ø)
src/openapi/schemas.ts 100.00% <ø> (ø)
src/review/public-rule-precision.ts 100.00% <100.00%> (ø)

... and 4 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 23, 2026
@loopover-orb

loopover-orb Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Important

🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪

🔍 LoopOver is reviewing…

AI analysis is in progress. This comment will update when the review is complete.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed · 🟪 Reviewing

@JSONbored
JSONbored merged commit 0a596ca into main Jul 23, 2026
17 checks passed
@JSONbored
JSONbored deleted the feat/public-rule-precision branch July 23, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

public-stats: measured per-rule precision endpoint (aggregates only, sanitizer-invariant)

1 participant