feat(review): port reviewbot blockers[]/nits[] shape for extensive reviews - #1089
Merged
Conversation
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1089 +/- ##
==========================================
- Coverage 94.76% 94.76% -0.01%
==========================================
Files 153 153
Lines 18542 18540 -2
Branches 6711 6705 -6
==========================================
- Hits 17572 17570 -2
Misses 418 418
Partials 552 552 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JSONbored
force-pushed
the
feat/review-depth-blockers-nits
branch
from
June 23, 2026 17:48
ff3a875 to
5f9e587
Compare
…views The free Workers-AI pair produced shallow, hedging reviews under the open-ended assessment/suggestions/risks/criticalDefect shape. reviewbot's gold-standard depth comes from forcing the model to ENUMERATE findings via explicit blockers[] (concrete must-fix defects) + nits[] (non-blocking) arrays with a severity rubric. Port it: - REVIEW_SYSTEM_PROMPT: blockers[]/nits[]/suggestions arrays + severity discipline (a blocker points to a real diff defect; nits/hypotheticals never block; CI status is never a code finding), 'do not rubber-stamp', no hedging language. - ModelReview / parseModelReview: parse blockers + nits (finding cap 6 -> 12). - composeAdvisoryNotes: render **Blockers** + **Nits** (nits + suggestions merged). - consensusDefectOf: a consensus defect = a concrete blocker in BOTH reviews (severity-disciplined; a lone blocker is a split) -> replaces the numeric floor. Internal to services/ai-review.ts; the external advisoryNotes string + consensusDefect types are unchanged, so the gate + unified comment are unaffected. Diff patches were verified present in storage -- the gap was purely the prompt structure.
JSONbored
force-pushed
the
feat/review-depth-blockers-nits
branch
from
June 23, 2026 17:51
5f9e587 to
9947d13
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The free Workers-AI pair was producing shallow, hedging reviews ("appears to…", "seems to…") even with the diff fully present. Confirmed via #1049: the stored file payloads contain real patches (
has_patch=1, 5–6 KB each) and the AI re-ran on the re-run checkbox — so the diff was never the problem. The gap is the prompt structure.reviewbot's gold-standard depth comes from forcing the model to enumerate findings into explicit
blockers[](concrete must-fix) +nits[](non-blocking) arrays under a severity rubric — not an open-ended "be thorough".What
Port reviewbot's recipe into
src/services/ai-review.ts:REVIEW_SYSTEM_PROMPT→blockers[]/nits[]/suggestionsarrays + severity discipline (a blocker points to a real diff defect; nits/hypotheticals never block; CI status is never a code finding), "do not rubber-stamp", no hedging.ModelReview/parseModelReview→ parseblockers+nits(finding cap 6 → 12).composeAdvisoryNotes→ render Blockers + Nits (nits + suggestions merged).consensusDefectOf→ a consensus defect = a concrete blocker in both reviews (severity-disciplined; a lone blocker is a split, not a hard block) — replaces the numeric confidence floor.Internal to
ai-review.ts: the externaladvisoryNotesstring +consensusDefecttypes are unchanged, so the gate and unified comment are unaffected.Tests
Full suite green (3483 passed). Updated
ai-review,ai-review-advisory,queuefixtures/assertions to the new shape; added a nits-don't-block consensus test.