[Winoren]

Which issue(s) does this pull-request address?

None; found by comparing the LiteLLM adapter with the LangChain and Transformers adapters, which both honour stop_sequences.

Closes: #

Description

stop_sequences is part of ChatModelParameters, is documented in the ChatModel.run docstring, and is demonstrated in python/examples/backend/chat.py:

llm.parameters.stop_sequences = ["q", "quit", "ahhhhhhhhh"]

On the LiteLLM path (which backs OpenAIChatModel, OllamaChatModel, AnthropicChatModel, GroqChatModel, ...), that value never reached the provider:

• supported_params for OpenAI contains stop, never stop_sequences, so include_keys(...) filtered it out of params;
• it was not in the exclusion set either, so it stayed in settings and was forwarded to acompletion() as the unknown keyword stop_sequences, which litellm.drop_params = True silently discards.

Verified against litellm at the current lockfile version:

get_supported_openai_params(model="gpt-4o", custom_llm_provider="openai") -> [..., 'n', 'seed', 'stop', ...]
_transform_input(ChatModelInput(messages=[...], stop_sequences=["q", "quit"]))
  -> {'api_key': ..., 'max_retries': 0, 'model': ..., 'stop_sequences': ['q', 'quit'], ...}   # no 'stop'

The LangChain adapter already performs this translation (parameters["stop"] / stop=input.stop_sequences), and the Transformers adapter honours input.stop_sequences directly, so the LiteLLM path was the odd one out.

Change

• _transform_input now translates stop_sequences to stop when the provider advertises stop in supported_params, keeping the precedence the existing parameter merge uses: per-run value, then the model's default parameters.
• stop_sequences is added to the exclusion set so the unknown keyword is no longer forwarded.

If a provider does not advertise stop, the value is dropped exactly as it is today.

Tests

Four unit tests in tests/backend/providers/test_litellm_chat.py (monkeypatched acompletion, no network): per-run value, default parameters value, per-run override, and no-value. Reverting the fix makes the first three fail.

Checklist

General

• I have read the appropriate contributor guide: Python
• I have signed off on my commit: Python instructions
• Commit messages and PR title follow conventional commits
• Appropriate label(s) added to PR: Python for Python changes, TypeScript for TypeScript changes

Code quality checks

• Code quality checks pass: mise check (mise fix to auto-fix)
  • ruff check and ruff format --check clean on both changed files

Testing

• Unit tests pass: mise test:unit
  • pytest tests/backend -q -> 94 passed (excluding tests/backend/providers/test_transformers.py, which needs the optional outlines/torch extras)
  • pytest tests -m unit -q -> 400 passed
• E2E tests pass: mise test:e2e (requires a running Ollama; not run here)
• Tests are included (for bug fixes or new features)

Documentation

• Documentation is updated - no doc change needed; the docs and examples already describe the behaviour this now matches.

---

Note: this change was prepared with AI assistance and reviewed, tested and signed off by the contributor above.

[Tomas2D]

Thanks for this. It's a real bug, and the fix is in the right place.

On main with litellm 1.89.2, the unknown stop_sequences keyword isn't actually dropped. litellm forwards it as a provider-specific param. OpenAI-compatible endpoints receive a literal "stop_sequences" field, watsonx fails outright with ValueError ... Param: stop_sequences, and Anthropic and Gemini only worked because their native field has the same name. Your change fixes all of these. I checked that Anthropic, Gemini, Vertex, Bedrock and watsonx get the right native field once litellm maps stop, and that models without stop support (e.g. gpt-5) now leave it out.

One behaviour change to be aware of (not blocking): litellm truncates stop to 4 items for every provider unless litellm.disable_stop_sequence_limit is set, so Anthropic and Gemini users with more than 4 sequences now get only the first 4.

Optional test additions: a provider without stop support sends neither field, and run(..., stop_sequences=[]) clears a non-empty default. Approving, and I'll merge once CI is green.