Repository navigation
feature/Pipellm-returns-empty-text · L-260903-2b8290 - #1414
Merged
Merged
Conversation
Every LLM worker now reads its response's stop value before it returns a text, through one classifier in pipelex/cogt/llm/completion_stop.py that reads the union of the provider vocabularies (OpenAI chat completions, so a gateway's pass-through of another provider's value too, the Responses API's incomplete status, Anthropic, Bedrock Converse, Gemini and Mistral). A text cut at its output limit or context window raises the new LLMCompletionTruncatedError, and one the model refused or a filter stopped raises LLMCompletionRefusedError: content errors in the input domain, not retryable, with a change_input next step and a message naming the model, the pipe, the stop value and the token counts, never the partial text. An unknown stop value is logged and taken as normal. Structured generation and its configured re-asks are untouched. The Bedrock client now returns Converse's stopReason with the text and usage in a BedrockChatResult, joining every text block so an answer with none no longer raises IndexError. The Anthropic structured call logs a max_tokens it lowers to fit its timeout, and an error it raises says the limit was lowered and to what. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gen_text reported a job's usage only after _gen_text returned, so a text cut at its limit or refused, which the provider answered and billed, emitted no usage report and its span carried no usage. When a truncation or refusal propagates from _gen_text with usage recorded on the job, the base now finalizes and reports it once and records it on the span before re-raising. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…prompt is a refusal A Responses answer with the incomplete status and an absent or unknown reason was classified normal, so its partial text passed for a result; the status alone says the text is unfinished, so such a reason is now taken as a truncation, with the warning kept. A Gemini prompt blocked before any candidate fell to the generic "No candidates returned" error; its block reason, read against the SDK's BlockedReason enum, now raises LLMCompletionRefusedError naming it, the generic error kept for an answer with neither candidate nor block reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ax_tokens The classifier mapped Anthropic's and Bedrock's model_context_window_exceeded and Mistral's model_length to the same truncation as an output-cap stop, and LLMCompletionTruncatedError always advised raising max_tokens, which cannot help when the input and the output filled the window. The classifier now returns a context-window outcome for those values, and the error carries the limit hit as truncation_limit (max_tokens or context_window): at the context window its message and user action say to shorten the input, lower the reasoning effort or choose a model with a larger window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xplain The Anthropic structured call appended "max_tokens was lowered" to every error, rate limits, authentication failures, overloads, server errors and lost connections included, and since the default cap is lowered on almost every structured call, each of those pointed at max_tokens. The note now rides only an output that failed validation or came back incomplete once the re-asks are spent, and a timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…are corrected A MAX_TOKENS truncation always advised raising the pipe's max_tokens, even when the request sent none and the provider's default cut the text; it now says no max_tokens was sent and advises setting the pipe's up to the model's limit, lowering its reasoning effort or shortening its input. The truncation page now says an incomplete Responses answer is named by its reason when it gives one, the new log fields join the logging vocabulary table, and the stopped-usage test annotates its mocks with pytest-mock's MockType. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every LLM worker now reads its provider's stop signal through one classifier in `pipelex/cogt/llm/completion_stop.py`, so a completion cut at its token cap or context window, refused, filtered or blocked fails its pipe with `LLMCompletionTruncatedError` or `LLMCompletionRefusedError` instead of passing an empty or partial text as a success. Each error tells a person and an agent what happened and what to change, the billed usage of a refused answer is still reported once, and the Anthropic worker logs when it lowers a structured output's cap. A normal stop and every configured retry behave as before.
Closes L-260903-2b8290
🤖 Generated with Claude Code
Summary by cubic
LLM workers now classify provider stop signals and fail pipes with
LLMCompletionTruncatedErrororLLMCompletionRefusedErrorinstead of returning incomplete, refused, or filtered text as a success.change_inputnext step; messages identify the model, pipe, and stop value without exposing partial text.max_tokensfrom the context window, and advises setting a limit when none was sent.Closes L-260903-2b8290
Written for commit abf8000. Summary will update on new commits.