Skip to content

fix(review): hold required automated-review skips from PR metadata - #3721

Merged
JSONbored merged 1 commit into
mainfrom
codex/propose-fix-for-ai-review-bypass-vulnerability-rt8wlp
Jul 6, 2026
Merged

fix(review): hold required automated-review skips from PR metadata#3721
JSONbored merged 1 commit into
mainfrom
codex/propose-fix-for-ai-review-bypass-vulnerability-rt8wlp

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • Prevent contributor-controlled PR metadata (title keywords or base-branch selection) from quietly suppressing a configured blocking automated review and allowing an automatic merge path.
  • Treat metadata-driven skips that would otherwise bypass an enforced automated-review gate as a human-review hold so the gate cannot be silently bypassed.

Description

  • Add a classifier isContributorControlledAutoReviewSkipReason that marks title-keyword and base-branch skip reasons as contributor-controlled.
  • Add maybeAddRequiredAutoReviewSkipHold in the review processor to append an inconclusive, non-blocking hold finding when a contributor-controlled skip occurs while the repository requires a blocking automated review.
  • Wire the hold helper into the PR processing flow before the automated-review-run decision so the gate sees the hold (and becomes neutral/held) instead of passing.
  • Add unit tests covering the new classifier and the required-review hold wiring and update manifest tests to exercise the new behavior.

Testing

  • Ran git diff --check with no errors.
  • Ran the targeted unit tests with npx vitest run test/unit/focus-manifest.test.ts test/unit/auto-review-wiring.test.ts --reporter=dot, and all exercised tests passed (2 files, 470 tests).
  • Ran type checks with npm run typecheck, which completed successfully.
  • Attempted a full gate run with npm run test:ci, which started but was not completed locally due to transient network/setup and a long coverage phase; actionlint fell back and the coverage phase was terminated in this environment.
  • npm audit --audit-level=moderate could not complete here due to the registry audit endpoint returning 403 Forbidden.

Codex Task

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 6, 2026
@loopover-orb

loopover-orb Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-06 07:50:09 UTC

4 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · dirty

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This PR closes a real gap: contributor-controlled PR metadata (WIP title, base branch) could silently satisfy a blocking auto-review requirement and let a PR merge without AI review ever running. The new isContributorControlledAutoReviewSkipReason classifier correctly limits itself to the two attacker-controllable skip reasons (title keyword, base branch), and maybeAddRequiredAutoReviewSkipHold only fires when the repo actually requires blocking review, pushing a non-blocking 'ai_review_inconclusive' finding rather than failing the run outright. Tests cover the classifier's true/false cases and the hold wiring (added, not-added-for-label-skip, not-added-when-review-mode-off), and CI/typecheck/codecov all passed on this commit.

Nits — 5 non-blocking
  • Whether the pushed 'ai_review_inconclusive' finding actually causes the gate/disposition logic to hold rather than pass is not visible in this diff (that logic lives outside the shown files) — worth double-checking the gate's finding-severity handling for this code, since if it's ever treated as non-blocking the whole PR wouldn't achieve its stated goal.
  • processors.ts: maybeAddRequiredAutoReviewSkipHold's args includes skipAiReview but the function body never reads it directly — confirm it's genuinely needed by shouldRequirePublicAiReviewForAdvisory (called with the same args object) rather than being dead plumbing.
  • The hold call is placed before the authorBlacklisted/isFrozenForManualReview checks later in the flow — worth a comment or test noting the hold is independent of those states so a future refactor doesn't accidentally gate it on them.
  • Add a code comment at the maybeAddRequiredAutoReviewSkipHold call site in processors.ts noting why it must run before the aiReviewWillRun decision, since the ordering is load-bearing for the fix.
  • Consider a follow-up test that exercises the full PR-processing path (not just the unit-level hold helper) to confirm the finding actually changes the end-to-end gate disposition on a repo with review.auto_review configured to block.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 54 registered-repo PR(s), 46 merged, 439 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 54 PR(s), 439 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 54 PR(s), 439 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 6, 2026
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.30%. Comparing base (ede8961) to head (462d0fc).
⚠️ Report is 5 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3721   +/-   ##
=======================================
  Coverage   93.29%   93.30%           
=======================================
  Files         317      317           
  Lines       32368    32375    +7     
  Branches    11872    11875    +3     
=======================================
+ Hits        30199    30206    +7     
  Misses       1537     1537           
  Partials      632      632           
Files with missing lines Coverage Δ
src/queue/processors.ts 94.12% <100.00%> (+0.01%) ⬆️
src/signals/focus-manifest.ts 99.10% <100.00%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored force-pushed the codex/propose-fix-for-ai-review-bypass-vulnerability-rt8wlp branch from 4b81b2b to 462d0fc Compare July 6, 2026 09:17
@JSONbored
JSONbored merged commit 2d0a2e7 into main Jul 6, 2026
8 checks passed
@JSONbored
JSONbored deleted the codex/propose-fix-for-ai-review-bypass-vulnerability-rt8wlp branch July 6, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant