feat(review): per-rule (not just per-project) gate-decision precision tracking

[JSONbored]

Closes #7984.

Summary

• computeGateEval (parity.ts) scores prediction-vs-ground-truth AGGREGATED PER PROJECT — one systematically wrong deterministic rule (like the 2026-07-21/22 hotkey/coldkey regex bug, Fix bare-word hotkey/coldkey false positives in containsSecretLikeText + static-message redaction bug #7981) can sit at effectively 0% precision while hiding inside an otherwise-healthy project-wide close-precision number, diluted by every OTHER correct close reason the same project produces. The precision-over-time circuit breaker (auto-tune.ts) can never isolate and react to a single broken rule this way, even in principle.
• New src/review/rule-gate-eval.ts adds that missing dimension by RE-AGGREGATING data already recorded — review_audit's gate_decision rows already carry a reason code (summary) — no new collection pipeline, no new table. Mirrors contributor-gate-eval.ts's own established "new dimension, same fold" pattern exactly:
  • computeRuleGateEval: per-(project, ruleCode) rows, so a maintainer can see "rule X: 0/4 correct" on one repo even while that repo's own aggregate still looks healthy.
  • computeBlendedRuleGateEval: the SAME cells pooled ACROSS every project a rule has fired on, volume-weighted — a rule's trustworthiness is a property of the rule, not of any one repo it happened to trip. This is the report Replace the blanket CONCRETE_EVIDENCE_BLOCKER_CODES breaker exemption with a per-rule track record #7986 will read.
  • rulesBelowClosePrecisionFloor: which blended rows have cleared enough sample (>= AUTOTUNE_MIN_DECIDED) but sit below the SAME AUTOTUNE_CLOSE_PRECISION_FLOOR the project-level breaker uses — the exact lookup Replace the blanket CONCRETE_EVIDENCE_BLOCKER_CODES breaker exemption with a per-rule track record #7986 needs, and the exact "rule X: 0/4 correct" signal this issue exists to surface.
• Wired into operator-dashboard.ts (an existing operator-facing read path — Per-rule (not just per-project) precision tracking in the gate-eval pipeline #7984's own stated deliverable) as a new "Rules below close-precision floor" metric card, alongside the existing contributor-fairness tiles.
• Read/reporting only — no gate/disposition decision changes here; that's Replace the blanket CONCRETE_EVIDENCE_BLOCKER_CODES breaker exemption with a per-rule track record #7986's job.

Test plan

• npm run typecheck
• New test/unit/rule-gate-eval.test.ts (28 cases, 100% statement/line/function coverage — the 2 remaining untested branches mirror computeGateEval's own identical, equally-untested "unmatched truth value" defensive branch): fold correctness, incident replay at both per-rule and blended levels, reversal-discount parity, sort order, source/minerOnly scoping, D1 fail-safe, real review_audit integration reads, rulesBelowClosePrecisionFloor's sample/floor gating
• New operator-dashboard.test.ts case: end-to-end replay of an isolated 0/12-correct rule on an otherwise-healthy project (20 correct closes on a different reason) surfacing correctly on the new dashboard card, with the healthy rule's code never appearing anywhere in the payload
• Full unsharded npm run test:coverage: 1093/1093 files, 20416 tests, 0 failures
• npm run engine-parity:drift-check: clean, no version bump needed

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

❌ Patch coverage is 97.36842% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.56%. Comparing base (321c192) to head (fe5c0ec).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/review/rule-gate-eval.ts	97.29%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8099      +/-   ##
==========================================
- Coverage   92.01%   89.56%   -2.45%     
==========================================
  Files         754       99     -655     
  Lines       77188    22689   -54499     
  Branches    23335     3900   -19435     
==========================================
- Hits        71021    20322   -50699     
+ Misses       5061     2187    -2874     
+ Partials     1106      180     -926     

Flag	Coverage Δ
shard-1	62.93% <51.31%> (+7.83%)	⬆️
shard-2	64.65% <94.73%> (+13.47%)	⬆️
shard-3	36.20% <14.47%> (-20.00%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/services/operator-dashboard.ts	100.00% <100.00%> (ø)
src/review/rule-gate-eval.ts	97.29% <97.29%> (ø)

... and 656 files with indirect coverage changes

[loopover-orb]

Caution

🛑 LoopOver review result - fixes required

_{Review updated: 2026-07-22 22:40:11 UTC}

4 files · 1 AI reviewer · no blockers · CI failing · unstable

🛑 Suggested Action - Manual Review

Review summary
This adds a per-(project, ruleCode) and cross-project blended gate-precision fold, re-aggregating existing review_audit rows (no new table), and wires the blended report's below-floor rule codes into the operator dashboard as a new metric tile. The implementation is a faithful structural mirror of contributor-gate-eval.ts (same cell query pattern, same weighted-precision fold, same fail-safe-to-empty-report posture), and rule-gate-eval.ts itself has thorough unit coverage including the incident-replay scenario from the issue. The one place I can't confirm full coverage is the new dashboard tile's falsy ternary arm.

Nits — 6 non-blocking

• `computeRuleGateEval`/`computeBlendedRuleGateEval` don't apply `resolveEligibleFairnessAnalyticsProjects` the way `computeContributorGateEval` does (contributor-gate-eval.ts) — worth a one-line note on why a per-repo opt-out doesn't need to apply here (ruleCode isn't PII, but the per-project `RuleGateEvalRow.project` field still surfaces repo names, even if unused by this diff's own dashboard wiring).
• operator-dashboard.ts:303's `rulesBelowFloor.length > 0 ? ... : "no rule below floor"` ternary's falsy arm doesn't look covered by any test in test/unit/operator-dashboard.test.ts, which likely explains the reported 97.36% patch coverage (target 99%).
• The magic numbers 730/90/86_400_000 in rule-gate-eval.ts:97-98 mirror the same unnamed literals already present in contributor-gate-eval.ts's queryContributorGateCells, so this is pre-existing convention rather than new drift, but a shared constant would be cleaner in both places.
• rule-gate-eval.ts's `foldCell` if/else-if chain (line ~149) is flagged by static analysis as depth 5 — purely structural, no logic issue, but consider extracting the merge/close branches into small helpers if this file grows further.
• Add a dashboard test asserting the zero-flagged-rules case renders `delta: "no rule below floor"` to close the coverage gap and satisfy the 99% patch target.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

CI checks failing

• codecov/patch — 97.36% of diff hit (target 99.00%)

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#7984
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 14 registered-repo PR(s), 14 merged, 242 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 14 PR(s), 242 issue(s).
Improvement	✅ Minor	risk: clean · value: minor

Linked issue satisfaction

Addressed
The PR adds a new rule-gate-eval.ts module implementing computeRuleGateEval (per-project+rule) and computeBlendedRuleGateEval (cross-project pooled by rule) that re-aggregates existing gate_decision/pr_outcome data exactly as the issue proposes, includes rulesBelowClosePrecisionFloor using the same AUTOTUNE constants, and wires the blended report into operator-dashboard.ts as a new metric card, sa

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, Ruby, Go, JavaScript, MDX, Shell, Solidity
• Official Gittensor activity: 14 PR(s), 242 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review