Skip to content

fix(llm): clarify independent timeouts and deadline diagnostics - #1159

Merged
lizhengfeng101 merged 6 commits into
alibaba:mainfrom
nolanchic:fix/timeout-extends-request-deadline-1155
Oct 8, 2026
Merged

lizhengfeng101 merged 6 commits into
alibaba:mainfrom
nolanchic:fix/timeout-extends-request-deadline-1155

Conversation

@nolanchic

@nolanchic nolanchic commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Clarify the independent task and request timeouts and improve deadline diagnostics, without changing timeout defaults or retry behavior. This revision replaces the original proposal to derive the HTTP request timeout from --timeout.

Behavior

  • --timeout remains the concurrent-task budget in minutes (default 15; 0 disables the task deadline). Review multiplies this setting by the number of effort rounds.
  • The per-request HTTP timeout remains independently configured through OCR_LLM_TIMEOUT or provider timeout_sec (also supported in the legacy llm section). Precedence remains environment > configuration file > the 300-second client default. Increasing --timeout does not extend an individual request's timeout.
  • Diagnostics distinguish LLM request timeout from caller deadline exceeded, preserving the original error chain and errors.Is behavior. The shared LLM helper does not attribute arbitrary caller deadlines to --timeout: background memory compression and ocr llm test have their own deadlines. Per-file review failure reasons distinguish request timeouts from task deadlines.
  • Existing SDK retries and legacy truncated-response recovery remain unchanged. This PR does not add automatic replay when a request deadline expires, including during response-body or stream consumption.

For a slow request that legitimately needs more than five minutes, configure the request timeout explicitly, for example:

OCR_LLM_TIMEOUT=900 ocr review --timeout 30

Changes and tests

  • Clarify CLI help and synchronize configuration and CLI reference documentation in English, Chinese, Japanese, Korean, and Russian.
  • Add regression coverage for independent timeout configuration, request versus caller deadline diagnostics, an expired independent caller while the main review remains active, cancellation, and unchanged SDK retry limits.
  • Verify that request timeouts do not cause extra replay across OpenAI Chat Completions, OpenAI Responses, Anthropic, streaming responses, and Bedrock.

Validation

  • make check, make test (including race detection), and make coverage passed; total coverage is 91.2%.
  • Documentation structure and anchor checks, 10 rendering-related tests, and the site build passed. The build reports two bundle-size warnings.
  • GitHub checks passed for 05fb953b, including CI, Pages CI, translation sync, CodeQL, plugin contract, and CLA.

Scope

Related to #1155. Completed-plan reuse on resume is not implemented here and remains a separate follow-up. No new timeout-retry policy is introduced.

AI assistance

The maintainer follow-up used Codex CLI (session model identifier: gpt-6-astra) for implementation, tests, and documentation, and ocr (configured review model: claude-opus-5) for automated code review. This disclosure covers the maintainer follow-up, not the original author's tooling.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

✅ OpenCodeReview: Review complete: 0 finding(s) across 3 selected item(s).

The --timeout flag documents a concurrent-task timeout in minutes, but
the per-request HTTP deadline stayed at the client's independent
5-minute default. A review of one slow file therefore failed with
classification=timeout at duration_to_headers_ms≈300000 even when the
caller granted --timeout 30, and no CLI setting could let the file
review finish (issue alibaba#1155).

ResolveOptions gains a TaskTimeout field: when no env
(OCR_LLM_TIMEOUT) or config-file (timeout_sec) request timeout is set,
the task budget raises the request deadline so a single request can use
the whole task budget. Existing precedence is unchanged — env and
config-file timeouts still win, and leaving --timeout at its default
keeps the 5-minute client default.

ocr review and ocr scan both pass their --timeout value through.

Fixes alibaba#1155
@nolanchic
nolanchic force-pushed the fix/timeout-extends-request-deadline-1155 branch from 5af41ca to 8707e4f Compare September 28, 2026 17:55
main replaced the per-file timeout flag with --timeout binding
concurrentTaskTimeout, and the agent applies it as a context deadline
across review rounds — but the per-request HTTP deadline stayed at the
client's independent 5-minute default, so a single slow file request
could still die inside a task that had plenty of budget left (alibaba#1155's
original symptom).

Wire ResolveOptions.TaskTimeout from the live
opts.concurrentTaskTimeout (minutes) in both review and scan commands.
Precedence is unchanged: env OCR_LLM_TIMEOUT and config-file
timeout_sec still win, and an unset task timeout keeps the 5-minute
client default.
@lizhengfeng101

Copy link
Copy Markdown
Contributor

Thanks for working on this, @nolanchic. To clarify the maintainer direction: we should not adopt option 1 from #1155. Please keep the task timeout and per-request HTTP timeout independent.

--timeout controls the overall concurrent-task budget. Individual LLM requests already have separate controls through OCR_LLM_TIMEOUT and provider timeout_sec; their default should not change implicitly with the task budget.

Please revise this PR toward option 2:

  • Clarify the two timeout controls in CLI help/documentation and point users to the existing request-timeout settings.
  • Distinguish a request timeout from an exhausted task deadline in diagnostics.
  • Verify request-timeout retry behavior and address any gaps while respecting the remaining task budget and retry limits.

Completed-plan reuse on resume can remain a separate follow-up. Thanks!

@lizhengfeng101 lizhengfeng101 changed the title fix(llm): let --timeout raise the per-request HTTP deadline fix(llm): clarify independent timeouts and deadline diagnostics Oct 8, 2026
@lizhengfeng101

Copy link
Copy Markdown
Contributor

Hi @nolanchic, since I haven't heard back and would like to move this PR forward, I've taken over the follow-up work on this PR. Thanks for getting this started!

@lizhengfeng101 lizhengfeng101 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lizhengfeng101
lizhengfeng101 merged commit 2d67596 into alibaba:main Oct 8, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants