You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The system prompt (~99) instructs the model to bail with INCOHERENT_DIFF_ASSESSMENTand return empty blockers when the diff is unusable. A model that bails, violates that instruction, and happens to write a ≥40-character rationale has its blockers promoted into a usable review.
Under combine: "single" — which our live config uses (claude-code,ollama = single + fallback) — combineReviews (~2333-2342) turns any lone blocker into a full ai_consensus_defect: severity critical, published as "AI reviewers agree on a likely critical defect" (src/queue/ai-review-orchestration.ts ~758) for a one-reviewer run. Under aiReviewGateMode: block that is a close.
Before the #8789 reclassification this produced ai_review_inconclusive → neutral hold, which is the correct fail-closed answer.
plannedReviewerCount (~2788) is computed and has no consumer. Under consensus, a 2→1 degradation correctly fails closed to inconclusive (~2386) — but under synthesis + either, missing > 0 with one flagging reviewer returns a full-strength defect (~2361-2366) with nothing recording that only one reviewer ran. Emit loopover_ai_review_reviewer_degraded_total{planned,actual} and put plannedReviewerCount in the decision record so calibration can segment single-reviewer blocks.
src/services/ai-review.ts~911-928:The system prompt (~99) instructs the model to bail with
INCOHERENT_DIFF_ASSESSMENTand return empty blockers when the diff is unusable. A model that bails, violates that instruction, and happens to write a ≥40-character rationale has itsblockerspromoted into a usable review.Under
combine: "single"— which our live config uses (claude-code,ollama= single + fallback) —combineReviews(~2333-2342) turns any lone blocker into a fullai_consensus_defect: severitycritical, published as "AI reviewers agree on a likely critical defect" (src/queue/ai-review-orchestration.ts~758) for a one-reviewer run. UnderaiReviewGateMode: blockthat is a close.Before the #8789 reclassification this produced
ai_review_inconclusive→ neutral hold, which is the correct fail-closed answer.Fix
valueAssessment(the point of fix(ai-review): the incoherent-diff bail misfires on title-vs-diff scope mismatch, turning reviewable PRs into inconclusive holds #8789) but dropblockers— a model that said it could not read the diff has not earned blocker authority over it.plannedReviewerCount === 1— see orb(ai-review): 'AI reviewers agree' is never verified — consensus fires whenever BOTH have any blocker, not the same one (427 closes) #9074, where the same string asserts agreement that was never checked.Related, same file
plannedReviewerCount(~2788) is computed and has no consumer. Underconsensus, a 2→1 degradation correctly fails closed toinconclusive(~2386) — but undersynthesis+either,missing > 0with one flagging reviewer returns a full-strength defect (~2361-2366) with nothing recording that only one reviewer ran. Emitloopover_ai_review_reviewer_degraded_total{planned,actual}and putplannedReviewerCountin the decision record so calibration can segment single-reviewer blocks.Refs #9074, #8789, #8833.