Summary
Live incident, self-inflicted by #4994. claude --print --output-format json is documented (claude --help) as returning a "single result" — it is fully buffered/non-streaming. Confirmed live on edge-nl-01: a realistic 274KB prompt at effort:high took 116 seconds to complete successfully, producing zero stdout bytes for the entire duration, then all output arriving at once. The 30-second (default) firstOutputTimeoutMs fast-fail added by #4994 cannot distinguish "hung" from "working on a large diff" for this CLI invocation shape — every review slower than the threshold was being killed and misclassified as claude_stalled_no_output.
Combined with the per-provider circuit breaker (src/selfhost/ai.ts, opens after 3 consecutive failures) and this self-host box's AI_PROVIDER=claude-code (no fallback provider configured), this caused a total AI-review outage: every review attempt failed deterministically, tripping the circuit repeatedly and blocking all reviews fleet-wide.
codex's own identical mechanism (resolveCodexFirstOutputTimeoutMs, pre-dating #4994) is NOT affected — codex exec --json genuinely streams JSONL progress to stdout, confirmed by its own code comment ("real JSONL progress from codex --json always lands on stdout"). #4994 copied that pattern onto claude-code without verifying claude's own CLI streams the same way. It doesn't.
Fix
resolveClaudeFirstOutputTimeoutMs's default/ceiling made the fast-fail window effectively equal the full timeoutMs budget by default (matching pre-#4994 behavior exactly) -- an operator can still opt into a shorter window via CLAUDE_AI_FIRST_OUTPUT_TIMEOUT_MS if they accept the false-positive risk, but the default no longer kills legitimate slow-but-working reviews. codex's mechanism is untouched.
Deliverables
Summary
Live incident, self-inflicted by #4994.
claude --print --output-format jsonis documented (claude --help) as returning a "single result" — it is fully buffered/non-streaming. Confirmed live on edge-nl-01: a realistic 274KB prompt ateffort:hightook 116 seconds to complete successfully, producing zero stdout bytes for the entire duration, then all output arriving at once. The 30-second (default)firstOutputTimeoutMsfast-fail added by #4994 cannot distinguish "hung" from "working on a large diff" for this CLI invocation shape — every review slower than the threshold was being killed and misclassified asclaude_stalled_no_output.Combined with the per-provider circuit breaker (
src/selfhost/ai.ts, opens after 3 consecutive failures) and this self-host box'sAI_PROVIDER=claude-code(no fallback provider configured), this caused a total AI-review outage: every review attempt failed deterministically, tripping the circuit repeatedly and blocking all reviews fleet-wide.codex's own identical mechanism (resolveCodexFirstOutputTimeoutMs, pre-dating #4994) is NOT affected —codex exec --jsongenuinely streams JSONL progress to stdout, confirmed by its own code comment ("real JSONL progress from codex --json always lands on stdout"). #4994 copied that pattern onto claude-code without verifying claude's own CLI streams the same way. It doesn't.Fix
resolveClaudeFirstOutputTimeoutMs's default/ceiling made the fast-fail window effectively equal the fulltimeoutMsbudget by default (matching pre-#4994 behavior exactly) -- an operator can still opt into a shorter window viaCLAUDE_AI_FIRST_OUTPUT_TIMEOUT_MSif they accept the false-positive risk, but the default no longer kills legitimate slow-but-working reviews.codex's mechanism is untouched.Deliverables
resolveClaudeFirstOutputTimeoutMsno longer false-positives on a genuinely slow (but successful) buffered--output-format jsonresponse.claude_stalled_no_outputvolume drops to near-zero (only firing for genuine full-timeout exhaustion).