Skip to content

fix(selfhost): claude-code fast-fail timeout false-positives on --output-format json (buffered, non-streaming) #5053

Description

@JSONbored

Summary

Live incident, self-inflicted by #4994. claude --print --output-format json is documented (claude --help) as returning a "single result" — it is fully buffered/non-streaming. Confirmed live on edge-nl-01: a realistic 274KB prompt at effort:high took 116 seconds to complete successfully, producing zero stdout bytes for the entire duration, then all output arriving at once. The 30-second (default) firstOutputTimeoutMs fast-fail added by #4994 cannot distinguish "hung" from "working on a large diff" for this CLI invocation shape — every review slower than the threshold was being killed and misclassified as claude_stalled_no_output.

Combined with the per-provider circuit breaker (src/selfhost/ai.ts, opens after 3 consecutive failures) and this self-host box's AI_PROVIDER=claude-code (no fallback provider configured), this caused a total AI-review outage: every review attempt failed deterministically, tripping the circuit repeatedly and blocking all reviews fleet-wide.

codex's own identical mechanism (resolveCodexFirstOutputTimeoutMs, pre-dating #4994) is NOT affected — codex exec --json genuinely streams JSONL progress to stdout, confirmed by its own code comment ("real JSONL progress from codex --json always lands on stdout"). #4994 copied that pattern onto claude-code without verifying claude's own CLI streams the same way. It doesn't.

Fix

resolveClaudeFirstOutputTimeoutMs's default/ceiling made the fast-fail window effectively equal the full timeoutMs budget by default (matching pre-#4994 behavior exactly) -- an operator can still opt into a shorter window via CLAUDE_AI_FIRST_OUTPUT_TIMEOUT_MS if they accept the false-positive risk, but the default no longer kills legitimate slow-but-working reviews. codex's mechanism is untouched.

Deliverables

  • resolveClaudeFirstOutputTimeoutMs no longer false-positives on a genuinely slow (but successful) buffered --output-format json response.
  • Regression test pinning the new default behavior.
  • Confirm via Sentry after deploy that claude_stalled_no_output volume drops to near-zero (only firing for genuine full-timeout exhaustion).

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintainer-onlyOwner-only work — yields no Gittensor points.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions