Skip to content

fix(review): preserve gate verdict on auto-review skip - #3910

Merged
JSONbored merged 1 commit into
mainfrom
codex/fix-vulnerability-in-auto-review-system
Jul 7, 2026
Merged

fix(review): preserve gate verdict on auto-review skip#3910
JSONbored merged 1 commit into
mainfrom
codex/fix-vulnerability-in-auto-review-system

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • A benign autoReviewSkipReason was being used to publish a completed skipped Gate check unconditionally, which could mask deterministic hard blockers (secret leaks, pre-merge checks, linked-issue failures) and make required status checks non-blocking.
  • The skip flag should only suppress the AI review itself; it must not override or replace the deterministic gate conclusion when that gate yields a hold/failure/neutral outcome.

Description

  • Restrict the skip-path in the gate publication flow so createOrUpdateSkippedGateCheckRun is only used when autoReviewSkipReason is set and the computed gateEvaluation?.conclusion === "success", preserving the real gate verdict otherwise (src/queue/processors.ts).
  • Update the recorded persisted summary so skipped is set only in the same safe, already-passing case and otherwise the actual gateEvaluation conclusion is stored (src/queue/processors.ts).
  • Add a regression unit test that covers a draft PR with auto_review skipped where the deterministic gate does not pass, asserting the gate check and stored summary keep the non-skipped conclusion while AI review remains skipped (test/unit/queue.test.ts).

Testing

  • Ran git diff --check to validate whitespace/conflict hygiene, which succeeded.
  • Ran targeted unit tests with npx vitest run test/unit/queue.test.ts -t "auto-review eligibility" --reporter=verbose and npx vitest run test/unit/queue.test.ts -t "deterministic gate conclusion" --reporter=verbose, both passing locally.
  • The added regression test (publishes the deterministic gate conclusion when auto-review eligibility is skipped but the gate does not pass) executed and validated that AI review is skipped while the gate publishes its true (non-skipped) conclusion.

Codex Task

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 7, 2026
@loopover-orb

loopover-orb Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-07 06:51:51 UTC

2 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This diff correctly scopes the auto-review skip to only suppress the AI review status, not the deterministic gate outcome: `createOrUpdateSkippedGateCheckRun` and the `skipped` summary conclusion are now gated on `gateEvaluation?.conclusion === "success"` in addition to the existing `autoReviewSkipReason && !publicSurfaceSkipped` check (src/queue/processors.ts, both the check-run publish call and the persisted `conclusion` field). This closes the described masking bug where a benign skip reason (draft/WIP/docs-only/etc.) could publish a quiet 'skipped' status over a real hold/failure/neutral gate verdict. The added regression test exercises a draft PR with `requireLinkedIssue: true` and no linked issue, asserting the gate check publishes 'neutral' and the stored summary is not 'skipped' while AI review is skipped (aiCalls === 0), which matches the fix's intent.

Nits — 4 non-blocking
  • The external secret-scan flag on test/unit/queue.test.ts:3760 is almost certainly a false positive on the literal fixture string "installation-credential" used as a fake auth token, but worth a quick glance to confirm no real credential leaked in.
  • The two duplicated conditions `autoReviewSkipReason && !publicSurfaceSkipped && gateEvaluation?.conclusion === "success"` (processors.ts) could be hoisted into a single named boolean (e.g. `canPublishSkippedGate`) to avoid drift if one call site is updated later without the other.
  • Consider extracting the shared skip-eligibility condition into a local const in maybePublishPrPublicSurface so both the check-run call and the persisted summary conclusion stay in sync by construction rather than by duplicated inline expressions.
  • The new test's `AI: { run: ... }` mock plus `expect(aiCalls).toBe(0)` is a good direct assertion that AI review was actually skipped — keep this pattern for future skip-path regression tests.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 51 registered-repo PR(s), 43 merged, 343 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 51 PR(s), 343 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 51 PR(s), 343 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.57%. Comparing base (ac8414c) to head (28c4911).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3910   +/-   ##
=======================================
  Coverage   93.57%   93.57%           
=======================================
  Files         340      340           
  Lines       33516    33516           
  Branches    12262    12262           
=======================================
  Hits        31364    31364           
  Misses       1528     1528           
  Partials      624      624           
Files with missing lines Coverage Δ
src/queue/processors.ts 94.53% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 7, 2026
@JSONbored
JSONbored merged commit 1f8af3e into main Jul 7, 2026
10 checks passed
@JSONbored
JSONbored deleted the codex/fix-vulnerability-in-auto-review-system branch July 7, 2026 07:14
JSONbored added a commit that referenced this pull request Jul 8, 2026
…gate the merge (#4171)

Content/registry repos (metagraphed today) have no schema/lint/codecov
net to catch a semantically-wrong-but-structurally-valid defect -- their
own AI reviewer's judgment is the only thing that ever catches it, yet
applySurfaceGate's guard #3 lets a decisive deterministic surface merge
unconditionally override even a confidently-flagged AI-judgment blocker.

Adds gate.aiJudgmentBlockers: "gate" | "advisory" (default "advisory",
byte-identical everywhere that doesn't opt in), YML-only config mirroring
contentLane's own shape since this only matters for repos already running
the registry content lane. When "gate", the AI-judgment-only override is
skipped and the finding survives into the deterministic gate's own
blockers, demoting the decision away from merge -- reproducing exactly
the PR #3910 shape (correct-in-prose, wrong-in-disposition) as
structurally impossible once a repo opts in.

Setting metagraphed's own .gittensory.yml to gate.aiJudgmentBlockers:
"gate" is a separate follow-up PR in that repo, gated on this shipping.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant