Skip to content

feat(review): per-repo opt-in to let a confident AI-judgment blocker gate the merge decision (not just annotate it) #3907

Description

@JSONbored

Context

Same repro as #3906: JSONbored/metagraphed PR #3910. Alongside the linked-issue-scope miss (tracked separately in #3906), the AI reviewer also correctly identified a real, verifiable data-integrity defect: the new registry surface's provider field is set to "gittensory" — a distinct, unrelated third-party tool's slug — instead of "gittensor". Confirmed directly: every other surface on the api.gittensor.io/gittensor.io domain in registry/subnets/gittensor.json uses "provider": "gittensor"; "gittensory" is reserved for the unrelated gittensory-api.aethereal.dev/gittensory.aethereal.dev entries in the same file. This was correctly rendered under the comment's "Blockers" heading, yet the top-line disposition was still approve/merge recommended and the "Code review" signal row read ✅ No blockers.

Why this isn't just re-litigating #2592

#2592 was resolved (intentionally, per the rationale now documented at src/review/unified-comment.ts:349-350) as: a gate decision: merge stays authoritative-ready even with AI-judgment blockers present, because gittensory's own code-repo review has other deterministic nets — codecov/patch, tests, lint, secret-scan, manifest/slop checks. AI judgment is supplementary there, so leaving it advisory-only is reasonable.

Content/registry repos like metagraphed don't have that net. There is no schema rule, lint, or codecov equivalent that can ever catch "this provider slug is semantically wrong for this domain" — that determination only ever comes from AI judgment reading the surrounding data. Under the current universal design, a metagraphed content-correctness defect this clear-cut and this confidently flagged can never gate a merge — every future PR shaped like this renders "approve/merge, no blockers" even when the reviewer got it right.

Requirements

  • Add a per-repo .gittensory.yml knob (e.g. gate.aiJudgmentBlockers: "gate" | "advisory", default "advisory" — byte-identical to today everywhere that doesn't opt in) that, when set to "gate", promotes a sufficiently-confident AI-judgment finding from the rendered "Blockers" list into the gate's own deterministic blockers array, demoting decision away from merge.
  • This mirrors option 1 originally proposed (and left undecided in favor of "confirm as intended" for the general case) in fix(review): decide whether AI-judgment blockers should gate the verdict, not just annotate it #2592 — scoped now as an explicit per-repo opt-in rather than a universal behavior change, so it does not reopen or reverse that decision for gittensory or any other repo that doesn't opt in.
  • Confidence bar: only a finding the AI reviewer itself placed under "Blockers" (as opposed to "Nits") is eligible for promotion — never promote nits.
  • Regression test reproducing PR fix(review): preserve gate verdict on auto-review skip #3910's provider-misattribution shape on an opted-in repo config: confirm decision is no longer merge. Confirm an opted-out repo (e.g. gittensory's own config) is unaffected — byte-identical to current tests.

Deliverables

  • gate.aiJudgmentBlockers config knob: schema + resolver + .gittensory.yml + openapi, per the standard gate-setting-wiring template
  • Promotion logic wiring confident AI "Blockers" findings into the deterministic gate blockers array when opted in
  • Regression tests: opted-in repo demotes merge; opted-out repo (default) is unaffected
  • Once shipped, default metagraphed's own .gittensory.yml to gate.aiJudgmentBlockers: "gate"

Expected outcomes

  • Registry/content repos (metagraphed today, others later) can require their own AI reviewer's confidently-flagged content-correctness findings to actually block a merge, without changing behavior anywhere that hasn't opted in.
  • The exact PR fix(review): preserve gate verdict on auto-review skip #3910 shape (correct-in-prose, wrong-in-disposition) becomes structurally impossible once metagraphed opts in.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions