feat(review): record MCP predicted-gate verdicts and measure predicted-vs-live gate agreement

[JSONbored]

Summary

gittensory_predict_gate/gittensory_explain_gate_disposition compute a verdict but never record it — there was no way to measure how often the MCP tool's pre-submission prediction actually agrees with the REAL gate decision a contributor's PR later receives.

• src/review/predicted-gate-calls.ts (new, migration 0134): records every predict-gate call into a local-only, login-keyed predicted_gate_calls table, structurally a sibling of contributor_gate_history (migration 0126) — review_audit (migration 0049) is deliberately actor-login-free (feeds the anonymized orb-collector export), so this follows that exact precedent rather than adding a login column there. Wired into computePredictedGateVerdict (src/mcp/server.ts), shared by both tools.
• src/review/predicted-gate-agreement.ts (new): a sibling to computeGateEval/computeGateParity (src/review/parity.ts), answering a different question than either — joins predicted_gate_calls against the already-existing contributor_gate_history (the login-keyed real-decision data recorded at the same call sites as recordNativeGateDecision) by (project, login, timestamp), since a predicted call has no PR number to key on the way a real gate decision does. Folds into a 2-way (merge/hold — the gate never predicts "close") agreement matrix with an explicit unsafe-direction count (predicted merge, real hold).
• GET /v1/internal/predicted-agreement (new endpoint, src/api/routes.ts): same bearer/flag gate as the existing /v1/internal/parity, surfaces the new metric.
• scripts/check-schema-drift.mjs: allowlists predicted_gate_calls as raw-SQL-only (matching the existing convention for contributor_gate_history etc.)

Deliberately scoped out (documented in code): a per-login breakdown of this report is explicitly issue #4517's separate, deliberately-scoped concern (a login-keyed disagreement ledger with its own data-ownership boundary) — this PR's output stays project-level aggregate only.

Fixes #4516

Test plan

• npx tsc --noEmit clean
• npm run db:migrations:check / db:schema-drift:check clean
• New tests: writer (predicted-gate-calls.test.ts, 100% coverage), reader/join logic (predicted-gate-agreement.test.ts — pairing, multi-candidate iteration, cross-login/cross-project isolation, non-binary real-decision skip-through, correlation-window boundary, hasSignal threshold, fail-safe on read errors, the new endpoint), MCP wiring (mcp-predict-gate.test.ts — both tools record a row, cloud-worker default is a no-op)
• Full related suite green: 124 tests across all 7 touched/adjacent test files

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.09%. Comparing base (80d623f) to head (8f053bd).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4558   +/-   ##
=======================================
  Coverage   94.08%   94.09%           
=======================================
  Files         427      429    +2     
  Lines       37961    38023   +62     
  Branches    13864    13882   +18     
=======================================
+ Hits        35715    35777   +62     
  Misses       1586     1586           
  Partials      660      660           

Files with missing lines	Coverage Δ
src/api/routes.ts	94.24% <100.00%> (+<0.01%)	⬆️
src/mcp/server.ts	95.57% <100.00%> (+<0.01%)	⬆️
src/review/predicted-gate-agreement.ts	100.00% <ø> (ø)
src/review/predicted-gate-calls.ts	100.00% <100.00%> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[loopover-orb]

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

_{Review updated: 2026-07-10 02:47:52 UTC}

9 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

⏸️ Suggested Action - Manual Review

• Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): scripts/check-schema-drift.mjs (matched scripts/**).

Review summary
This PR adds a local-only predicted_gate_calls table (migration 0134) that records every MCP predict_gate/explain_gate_disposition verdict, plus a computePredictedGateAgreement module that joins those rows against the existing contributor_gate_history by (project, login, timestamp) to measure predicted-vs-live gate agreement, exposed via a bearer- and flag-gated /v1/internal/predicted-agreement endpoint. The design correctly follows established precedent (a separate login-keyed local table rather than adding a login column to the login-free review_audit, reuse of nativeGateActionFromConclusion, the same best-effort/fail-safe write contract as recordNativeGateDecision) and is backed by thorough boundary/error-path tests; the migration itself is D1-remote-safe with no temp/pragma/transaction issues. The main caveats are a stale migration-number reference in the new module's comments and that the actual join/query implementation (predicted-gate-agreement.ts) wasn't visible to confirm directly, only inferred from its test suite.

Nits — 7 non-blocking

• src/review/predicted-gate-calls.ts's header and inline comments reference "migrations/0132" as this table's migration three times, but the actual added file is migrations/0134_predicted_gate_calls.sql — fix the comments to say 0134 so future readers aren't misled about which migration created the table.
• migrations/0134_predicted_gate_calls.sql:31 uses a static `DEFAULT CURRENT_TIMESTAMP` for created_at, which produces a non-ISO-8601 string if the column is ever inserted without an explicit value, deviating from the repo's `$defaultFn(() => nowIso())` convention; harmless today since predicted-gate-calls.ts always binds nowIso() explicitly, but worth aligning for consistency.
• Confirm issue feat(review): record MCP predicted-gate verdicts to review_audit and measure predicted-vs-live gate agreement #4516's linkage language in the full PR description (truncated in what was provided) actually closes/references an eligible open issue per the repo's contributor-PR policy, since the external brief flagged only partial coverage.
• The row `id` uses `Math.random().toString(36).slice(2, 8)` for uniqueness alongside timestamp/login/project — fine for a non-security-critical telemetry key, but a UUID (as used elsewhere, e.g. the test helpers' `crypto.randomUUID()`) would be more consistent with the rest of the codebase.
• src/review/predicted-gate-calls.ts's `reasonCode`/`project` truncation both hardcode `.slice(0, 200)`; consider a small shared bound helper if this pattern recurs elsewhere in the review-audit family.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
• Touches a guarded path — held for manual review — A maintainer must review and merge this change.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ Linked	#4516
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 48 registered-repo PR(s), 40 merged, 347 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 48 PR(s), 347 issue(s).
Gate result	⚠️ Not blocking	Advisory; not blocking this PR.

Linked issue satisfaction

Addressed
The PR records every predict_gate/explain_gate_disposition call into a new login-keyed table with a documented (project, login, timestamp) correlation-key design exactly as the issue anticipated (since there's no PR number pre-submission), adds a sibling agreement-computation function that joins predicted calls against real gate history into a distinct agreement metric with its own endpoint, and i

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: not available
• Official Gittensor activity: 48 PR(s), 347 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Treat this as maintainer-lane context rather than normal contributor-lane activity.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review