Skip to content

feature/Pipellm-returns-empty-text · L-260903-2b8290 - #1414

Merged
lchoquel merged 6 commits into
devfrom
feature/Pipellm-returns-empty-text
Oct 11, 2026
Merged

lchoquel merged 6 commits into
devfrom
feature/Pipellm-returns-empty-text

Conversation

@lchoquel

@lchoquel lchoquel commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Every LLM worker now reads its provider's stop signal through one classifier in `pipelex/cogt/llm/completion_stop.py`, so a completion cut at its token cap or context window, refused, filtered or blocked fails its pipe with `LLMCompletionTruncatedError` or `LLMCompletionRefusedError` instead of passing an empty or partial text as a success. Each error tells a person and an agent what happened and what to change, the billed usage of a refused answer is still reported once, and the Anthropic worker logs when it lowers a structured output's cap. A normal stop and every configured retry behave as before.

Closes L-260903-2b8290

🤖 Generated with Claude Code


Summary by cubic

LLM workers now classify provider stop signals and fail pipes with LLMCompletionTruncatedError or LLMCompletionRefusedError instead of returning incomplete, refused, or filtered text as a success.

  • Both errors are non-retryable input content errors with a change_input next step; messages identify the model, pipe, and stop value without exposing partial text.
  • Truncation guidance distinguishes max_tokens from the context window, and advises setting a limit when none was sent.
  • Billed usage is reported once for refused or truncated answers; Anthropic logs lowered structured-output limits only when relevant to an error or timeout.
  • Normal stops and configured retries remain unchanged.

Closes L-260903-2b8290

Written for commit abf8000. Summary will update on new commits.

View guided diff

lchoquel and others added 6 commits October 11, 2026 01:55
Every LLM worker now reads its response's stop value before it returns a text, through one classifier in pipelex/cogt/llm/completion_stop.py that reads the union of the provider vocabularies (OpenAI chat completions, so a gateway's pass-through of another provider's value too, the Responses API's incomplete status, Anthropic, Bedrock Converse, Gemini and Mistral). A text cut at its output limit or context window raises the new LLMCompletionTruncatedError, and one the model refused or a filter stopped raises LLMCompletionRefusedError: content errors in the input domain, not retryable, with a change_input next step and a message naming the model, the pipe, the stop value and the token counts, never the partial text. An unknown stop value is logged and taken as normal. Structured generation and its configured re-asks are untouched.

The Bedrock client now returns Converse's stopReason with the text and usage in a BedrockChatResult, joining every text block so an answer with none no longer raises IndexError. The Anthropic structured call logs a max_tokens it lowers to fit its timeout, and an error it raises says the limit was lowered and to what.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gen_text reported a job's usage only after _gen_text returned, so a text
cut at its limit or refused, which the provider answered and billed, emitted
no usage report and its span carried no usage. When a truncation or refusal
propagates from _gen_text with usage recorded on the job, the base now
finalizes and reports it once and records it on the span before re-raising.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…prompt is a refusal

A Responses answer with the incomplete status and an absent or unknown
reason was classified normal, so its partial text passed for a result; the
status alone says the text is unfinished, so such a reason is now taken as a
truncation, with the warning kept. A Gemini prompt blocked before any
candidate fell to the generic "No candidates returned" error; its block
reason, read against the SDK's BlockedReason enum, now raises
LLMCompletionRefusedError naming it, the generic error kept for an answer
with neither candidate nor block reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ax_tokens

The classifier mapped Anthropic's and Bedrock's model_context_window_exceeded
and Mistral's model_length to the same truncation as an output-cap stop, and
LLMCompletionTruncatedError always advised raising max_tokens, which cannot
help when the input and the output filled the window. The classifier now
returns a context-window outcome for those values, and the error carries the
limit hit as truncation_limit (max_tokens or context_window): at the context
window its message and user action say to shorten the input, lower the
reasoning effort or choose a model with a larger window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xplain

The Anthropic structured call appended "max_tokens was lowered" to every
error, rate limits, authentication failures, overloads, server errors and
lost connections included, and since the default cap is lowered on almost
every structured call, each of those pointed at max_tokens. The note now
rides only an output that failed validation or came back incomplete once the
re-asks are spent, and a timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…are corrected

A MAX_TOKENS truncation always advised raising the pipe's max_tokens, even
when the request sent none and the provider's default cut the text; it now
says no max_tokens was sent and advises setting the pipe's up to the model's
limit, lowering its reasoning effort or shortening its input. The truncation
page now says an incomplete Responses answer is named by its reason when it
gives one, the new log fields join the logging vocabulary table, and the
stopped-usage test annotates its mocks with pytest-mock's MockType.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lchoquel
lchoquel merged commit 4b6e141 into dev Oct 11, 2026
37 checks passed
@lchoquel
lchoquel deleted the feature/Pipellm-returns-empty-text branch October 11, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant