Skip to content

feat(review): per-repo opt-in to let a confident AI-judgment blocker gate the merge - #4171

Merged
JSONbored merged 1 commit into
mainfrom
feat/ai-judgment-blocker-gate-optin-3907
Jul 8, 2026
Merged

feat(review): per-repo opt-in to let a confident AI-judgment blocker gate the merge#4171
JSONbored merged 1 commit into
mainfrom
feat/ai-judgment-blocker-gate-optin-3907

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Same repro as fix(review): deterministic linked-issue gate never checks issue-PR scope match, only existence/openness #3906 (metagraphed PR fix(review): preserve gate verdict on auto-review skip #3910): the AI reviewer correctly flagged a real content-correctness defect under "Blockers" (a registry provider slug semantically wrong for its domain — "gittensory" instead of "gittensor"), yet the gate's top-line disposition stayed approve/merge recommended. Root cause: applySurfaceGate's guard feat(scoring): add situational score projections #3 unconditionally lets a decisive deterministic surface merge override even an AI-judgment-only failure — correct and intentional for gittensory's own code repo (fix(review): decide whether AI-judgment blockers should gate the verdict, not just annotate it #2592: other deterministic nets like codecov/patch/tests/lint/secret-scan make AI judgment supplementary there), but content/registry repos like metagraphed have no equivalent net for "this value is semantically wrong" defects.
  • Adds gate.aiJudgmentBlockers: "gate" | "advisory" (default "advisory", byte-identical everywhere that doesn't opt in) as a YML-only config knob — deliberately mirroring contentLane's own YML-only shape (no DB column, no dashboard toggle) rather than the heavier DB-backed gate-setting-wiring template, since this only ever has an effect for a repo already running the registry content lane.
  • When "gate": applySurfaceGate's AI-judgment-only override is skipped, so the finding falls through to the same union+failure path any other blocker takes — decision is no longer merge. evaluateWithSurfaceLane's companion advisory.findings cleanup (which strips the AI finding from the public-comment feed when the override fires) is also skipped in "gate" mode, since the finding is now a real blocker and the public comment needs to keep showing it.
  • Preserves the existing "advisory" (default) behavior byte-identically — confirmed by running all 46 pre-existing content-lane-wire.test.ts tests unchanged and green, plus a new explicit test proving aiJudgmentBlockersMode: "advisory" produces output toEqual the unset-default case.

Scope

Validation

  • git diff --check
  • npm run typecheck
  • npm run docs:drift-check — unaffected (this field lives only in FocusManifestGateConfig, not RepositorySettings, so it's outside GATE_MODE_MANIFEST's scope, which only scans src/types.ts).
  • npx vitest run test/unit/content-lane-wire.test.ts test/unit/focus-manifest.test.ts test/unit/config-templates.test.ts — 612/612 passing, including all 46 pre-existing content-lane-wire.test.ts cases unchanged.
  • New tests: applySurfaceGate with aiJudgmentBlockersMode: "gate" reproducing PR fix(review): preserve gate verdict on auto-review skip #3910's exact shape (a confident ai_consensus_defect provider-misattribution finding survives a decisive surface merge, decision becomes failure); the explicit-"advisory"-equals-default proof above; and two full evaluateWithSurfaceLane end-to-end tests — one opted-in (gate.aiJudgmentBlockers: "gate" via a real parsed manifest, confirms decision !== "success" AND the finding stays in advisory.findings) and one opted-out (zero-config manifest, confirms decision === "success" and the finding IS stripped, byte-identical to the pre-existing REGRESSION test).
  • Scoped coverage: src/review/content-lane-wire.ts 100% lines / 98.75% branches (the one remaining branch is in runRegistrySurfaceGate, a function this PR does not touch — confirmed via git diff, zero changes in that function). packages/gittensory-engine/src/focus-manifest.ts's uncovered spots are likewise pre-existing and unrelated (verified each one against git diff — none fall within the aiJudgmentBlockersMode field I added).
  • npm run test:workers, build:mcp, test:mcp-pack, ui:openapi:check, ui:lint, ui:typecheck, ui:build, npm audit — not re-run locally (no worker/MCP/OpenAPI/UI-component code touched); CI runs them authoritatively.

Safety

  • No secrets, wallet/hotkey/trust-score/reward data anywhere.
  • No behavior change for any repo that doesn't opt in — confirmed byte-identical via the full existing test suite passing unchanged.
  • No UI changes — no UI Evidence section needed.
  • No docs/changelog changes needed (.gittensory.yml.example + config/examples/gittensory.full.yml documentation is the deliverable itself, not a changelog).
  • Real design tension, documented in the code (not hidden): the guard this issue makes opt-outable is a deliberate, incident-motivated safety net (its own existing regression test cites a prior hallucinated-RAG-claim false positive as the reason AI-judgment-only findings must never veto a decisive deterministic verdict by default). Opting a repo into "gate" mode reintroduces exactly that risk as an explicit, per-repo, documented trade-off — the doc comments on both the manifest field and applySurfaceGate itself say so plainly, and the default stays "advisory" everywhere.

Notes

  • Deliberately out of scope, per the issue's own 4th deliverable: actually setting gate.aiJudgmentBlockers: "gate" in JSONbored/metagraphed's own .gittensory.yml is a separate follow-up PR in that other repo, gated on this shipping first — not bundled here.
  • Confidence bar, as specified: only a finding the AI reviewer itself already placed under "Blockers" (not "Nits") is eligible — this PR doesn't add a new confidence-bar mechanism, it reuses the existing isAiJudgmentOnlyFailure/AI_JUDGMENT_BLOCKER_CODES classification that already gates which findings are "AI-judgment-only" in the first place.

…gate the merge

Content/registry repos (metagraphed today) have no schema/lint/codecov
net to catch a semantically-wrong-but-structurally-valid defect -- their
own AI reviewer's judgment is the only thing that ever catches it, yet
applySurfaceGate's guard #3 lets a decisive deterministic surface merge
unconditionally override even a confidently-flagged AI-judgment blocker.

Adds gate.aiJudgmentBlockers: "gate" | "advisory" (default "advisory",
byte-identical everywhere that doesn't opt in), YML-only config mirroring
contentLane's own shape since this only matters for repos already running
the registry content lane. When "gate", the AI-judgment-only override is
skipped and the finding survives into the deterministic gate's own
blockers, demoting the decision away from merge -- reproducing exactly
the PR #3910 shape (correct-in-prose, wrong-in-disposition) as
structurally impossible once a repo opts in.

Setting metagraphed's own .gittensory.yml to gate.aiJudgmentBlockers:
"gate" is a separate follow-up PR in that repo, gated on this shipping.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui a80e7ae Commit Preview URL

Branch Preview URL
Jul 08 2026, 09:19 AM

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.72%. Comparing base (11fc55b) to head (a80e7ae).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
packages/gittensory-engine/src/focus-manifest.ts 0.00% 0 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (83.33%) is below the target coverage (99.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4171      +/-   ##
==========================================
- Coverage   93.72%   93.72%   -0.01%     
==========================================
  Files         387      387              
  Lines       36309    36312       +3     
  Branches    13300    13303       +3     
==========================================
+ Hits        34032    34034       +2     
  Misses       1621     1621              
- Partials      656      657       +1     
Files with missing lines Coverage Δ
src/review/content-lane-wire.ts 98.55% <100.00%> (+0.04%) ⬆️
packages/gittensory-engine/src/focus-manifest.ts 99.06% <0.00%> (-0.10%) ⬇️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit 4664ad2 into main Jul 8, 2026
11 of 12 checks passed
@JSONbored
JSONbored deleted the feat/ai-judgment-blocker-gate-optin-3907 branch July 8, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Development

Successfully merging this pull request may close these issues.

1 participant