[mayank-affirm]

TLDR

Problem this solves:

• Bedrock Mantle GPT models report overflow as OpenAI's context_length_exceeded
• litellm passed it through as a plain 400 BadRequestError with the raw provider text
• Claude Code then never compacts, and /compact itself fails

How it solves it:

• Any Mantle error carrying context_length_exceeded maps to ContextWindowExceededError
• Its message starts with prompt is too long, which Claude Code recognizes
• Covers the 400 body and the streamed event that carries no error type
• context_window_fallbacks fire for this overflow where the router consults them
• The /v1/messages stream adapter's own error event keeps that error's 400 and message, not a 500
• e.response on the mapped error is the provider's own 400 response, so an SDK caller still reads the envelope

Intentional product change: a Mantle overflow now reads prompt is too long: ... instead of the raw context_length_exceeded body. A client that matched the raw provider text loses that match and gets litellm's standard overflow error instead. With a context_window_fallbacks list in router_settings, that overflow no longer takes the deployment's plain fallbacks entry, the router's rule for every provider's overflow, so add the Mantle group to context_window_fallbacks

User Flow

Before: Claude Code never compacts after a Bedrock Mantle overflow, so the turn dies on an API error

1. A developer runs Claude Code with ANTHROPIC_BASE_URL at their LiteLLM proxy and ANTHROPIC_MODEL set to a bedrock_mantle/openai.gpt-5.6-luna deployment
2. The conversation grows past the model's 1,050,000-token window, so the next turn is POST https://litellm-domain/v1/messages with "stream": true
3. The proxy answers HTTP 400 {"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: BedrockException - Error code: 400 - {'error': {'message': 'Your input exceeds the context window of this model. Please adjust your input and try again.', 'type': 'invalid_request_error', 'code': 'context_length_exceeded'}}"}}
4. Claude Code prints API Error: 400 litellm.BadRequestError: BedrockException - Error code: 400 - ..., the turn ends and nothing compacts; the same prompt on POST /v1/chat/completions or /v1/responses gets a 400 BadRequestError carrying the raw provider text

After: the overflow comes back as a prompt is too long error, so Claude Code compacts and carries on

1. A developer runs Claude Code with ANTHROPIC_BASE_URL at their LiteLLM proxy and ANTHROPIC_MODEL set to a bedrock_mantle/openai.gpt-5.6-luna deployment
2. The conversation grows past the model's 1,050,000-token window, so the next turn is POST https://litellm-domain/v1/messages with "stream": true
3. The proxy answers HTTP 400 {"type":"error","error":{"type":"invalid_request_error","message":"litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: your prompt exceeds the model's context window"}}
4. Claude Code prints Conversation compacted, resends the trimmed conversation, and the turn completes; POST /v1/chat/completions and /v1/responses answer 400 with the same prompt is too long message

Relevant issues

Follow-up to #36580 and #37862, which mapped Mantle's other overflow envelope (prompt tokens (N) exceed model maximum (M)). Original report: #36546. Downstream report: #36580 (comment)

Linear ticket

Resolves LIT-8944

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Design decisions

Mantle reports context overflow in two envelopes. #37862 covered the first, a validation_error with token counts:

{"error":{"code":"validation_error",
          "message":"prompt tokens (1055489) exceed model maximum (1050000) for openai.gpt-5.6-sol",
          "param":null,"type":"invalid_request_error"}}

Mantle also returns the OpenAI-shaped body, reproduced live against a LiteLLM proxy in front of Mantle:

{"error":{"code":"context_length_exceeded",
          "message":"Your input exceeds the context window of this model. Please adjust your input and try again.",
          "param":"input","type":"invalid_request_error"}}

A streamed call gets the same thing as a response.failed event carrying only code and message, with no type. The second shape passed the old invalid_request_error gate but matched neither the token-count regex nor the shared overflow substring list, so it landed in the generic BadRequestError branch carrying the raw provider text

Claude Code only runs its reactive compaction when the error reads as an overflow: its 400 classifier matches prompt is too long or input is too long for requested model, and context window only at 413. So an overflow in this second shape died as an ordinary API error and the conversation could never compact, which is what produced the downstream /compact failure:

Error during compaction: API Error: 400 litellm.BadRequestError: BedrockException -
{"error":{"code":"context_length_exceeded","message":"Your input exceeds the context window of this model. ...

The match keys on the context_length_exceeded code alone, read as the envelope's code key in any of the three serializations the error text can carry (plain JSON, the Python repr 'code': 'context_length_exceeded', and the escaped JSON a message quotes). It is in both wire shapes and it is OpenAI's documented overflow code. Matching the English message was the alternative, but that wording is Mantle's to change, and requiring both code and text adds nothing the code does not already say. A Mantle 400 whose message merely contains the words context_length_exceeded (a validation error quoting a field value, say) stays a plain BadRequestError with the upstream message, so a valid request cannot be steered into compaction or a context-window fallback by its wording. The mapped error reuses the provider's own 400 response object, status, headers and body intact, and synthesizes a 400 with the original headers and request only when the original status is not 400 (the in-stream chunk-parser error, which carries no body on either side), so e.response.json() on the SDK keeps working

The invalid_request_error / validation_error gate that used to wrap this helper is gone, because the streamed event has no type and the gate silently skipped it. The token-count match still runs first, so that envelope keeps its prompt is too long: N tokens > M maximum wording with real numbers. The new shape carries no counts, so it gets the fixed prompt is too long: your prompt exceeds the model's context window, and clients parse the trailing N tokens > M maximum only when it is there, falling back to their own conversation accounting otherwise

Non-overflow Mantle errors (for example Invalid 'input': value did not match any expected variant) still map to BadRequestError, and the existing test for that is unchanged. Mutation check: deleting the new branch makes the new test fail with the raw BadRequestError, so the test defends the behavior rather than the implementation

On streamed /v1/messages the proxy buffers the first event, and since #44800 (merged into this branch) a provider error raised before any content reaches the client is answered as a JSON error carrying that error's own status, which is the HTTP 400 both legs below show. Before #44800 the same overflow surfaced as a pre-stream HTTP 500 that Claude Code retried ten times, the shape in the original report. The adapter's own error event, what the library's /v1/messages stream wrapper emits when no proxy manages the stream, used to turn every mapped litellm exception into a 500 api_error; it now passes through the status and message of ContextWindowExceededError too, so that path also reads as a 400 overflow, and the other mapped litellm exceptions keep their 500 there (both covered by the adapter's unit tests)

Screenshots / Proof of Fix

Last updated: 027f769, the PR merged with main 249add6. The After ran on 027f769 against that merge base, the Before on the earlier merge base 988cc7d. The audit matrix below names the commits it ran at

Shared setup. One proxy per leg, booted from its own venv with a Postgres database and 2 uvicorn workers, each on its own random port:

python litellm/proxy/proxy_cli.py --config qa_config.yaml --port $PORT --num_workers 2 --timeout_worker_healthcheck 60

model_list:
  - model_name: mantle-luna
    litellm_params:
      model: bedrock_mantle/openai.gpt-5.6-luna
      aws_region_name: us-east-1
      api_key: os.environ/AWS_BEARER_TOKEN_BEDROCK
  - model_name: gem-good
    litellm_params:
      model: gemini/gemini-3.8-flash
      api_key: os.environ/GEMINI_API_KEY
  - model_name: gem-temp
    litellm_params:
      model: gemini/gemini-3.8-flash
      api_key: os.environ/GEMINI_API_KEY
      temperature: 7
  - model_name: gem-404
    litellm_params:
      model: gemini/gemini-no-such-model-9
      api_key: os.environ/GEMINI_API_KEY
router_settings:
  fallbacks:
    - gem-temp: ["gem-good"]
    - gem-404: ["gem-good"]
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

The overflow payloads carry 4,150,000 characters of repeated text in one user turn, which Mantle rejects as over the window; the last four payloads are the regression guard and the control:

python3 - <<'PY'
import json
text = "This is a test sentence. " * 166000
msgs = [{"role": "user", "content": text}]
json.dump({"model": "mantle-luna", "max_tokens": 64, "stream": True, "messages": msgs}, open("messages_stream.json", "w"))
json.dump({"model": "mantle-luna", "max_tokens": 64, "messages": msgs}, open("messages.json", "w"))
json.dump({"model": "mantle-luna", "stream": True, "messages": msgs}, open("chat_stream.json", "w"))
json.dump({"model": "mantle-luna", "messages": msgs}, open("chat.json", "w"))
json.dump({"model": "mantle-luna", "stream": True, "input": text}, open("responses_stream.json", "w"))
json.dump({"model": "mantle-luna", "input": text}, open("responses.json", "w"))
small = [{"role": "user", "content": "Say hi in two words"}]
json.dump({"model": "gem-temp", "max_tokens": 64, "stream": True, "messages": small}, open("gem_temp_messages_stream.json", "w"))
json.dump({"model": "gem-temp", "stream": True, "messages": small}, open("gem_temp_chat_stream.json", "w"))
json.dump({"model": "gem-404", "max_tokens": 64, "stream": True, "messages": small}, open("gem_404_messages_stream.json", "w"))
json.dump({"model": "mantle-luna", "max_tokens": 32, "stream": True, "messages": small}, open("small_messages_stream.json", "w"))
PY

Every curl below has the same shape, printing the body and then the status code (/v1/messages calls add -H 'anthropic-version: 2023-06-01'):

curl -s -m 900 -o out.body -w '%{http_code}\n' http://127.0.0.1:$PORT/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -H 'anthropic-version: 2023-06-01' --data-binary @messages_stream.json; cat out.body

Claude Code 2.1.296 ran interactively in tmux against each proxy, with CLAUDE_CODE_MAX_CONTEXT_TOKENS=2000000 so its own proactive auto-compact stays out of the way and the overflow reaches the provider:

ANTHROPIC_BASE_URL=http://127.0.0.1:$PORT ANTHROPIC_AUTH_TOKEN=$LITELLM_MASTER_KEY ANTHROPIC_MODEL=mantle-luna ANTHROPIC_SMALL_FAST_MODEL=mantle-luna CLAUDE_CODE_MAX_CONTEXT_TOKENS=2000000 claude

Two pastes of about 2.5 MB each, This is a test sentence. repeated 100,000 times followed by Reply with just OK. Part A alone fits the window (about 590,000 tokens, answered OK), part B on top of it does not

Before (988cc7d)

Claude Code

1. Launch Claude Code as above, paste part A; Claude answers OK
2. Paste part B; the pane shows API Error: 400 litellm.BadRequestError: BedrockException - Error code: 400 - {'error': {'message': 'Your input exceeds the context window of this model. Please adjust your input and try again.', 'type': 'invalid_request_error', 'code': 'context_length_exceeded'}}, the turn ends, and nothing compacts

POST /v1/messages, stream

1. curl ... /v1/messages --data-binary @messages_stream.json
2. 400, a JSON body and no stream:

{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: BedrockException - Error code: 400 - {'error': {'message': 'Your input exceeds the context window of this model. Please adjust your input and try again.', 'type': 'invalid_request_error', 'code': 'context_length_exceeded'}}"}}

POST /v1/messages

1. curl ... /v1/messages --data-binary @messages.json
2. 400:

{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: BedrockException - {\"error\":{\"code\":\"context_length_exceeded\",\"message\":\"Your input exceeds the context window of this model. Please adjust your input and try again.\",\"param\":\"input\",\"type\":\"invalid_request_error\"}}\n\nLiteLLM: model group 'mantle-luna' failed with the error above and no fallback model group was found for it, so the request was not retried on another model. Fallbacks are configured for: gem-temp, gem-404. Add a fallbacks entry for that model group (Router fallbacks or proxy router_settings.fallbacks) to retry on another model."}}

POST /v1/chat/completions, stream

1. curl ... /v1/chat/completions --data-binary @chat_stream.json
2. 400:

{"error":{"message":"litellm.BadRequestError: BedrockException - Error code: 400 - {'error': {'message': 'Your input exceeds the context window of this model. Please adjust your input and try again.', 'type': 'invalid_request_error', 'code': 'context_length_exceeded'}}","type":null,"param":null,"code":"400"}}

POST /v1/chat/completions

1. curl ... /v1/chat/completions --data-binary @chat.json
2. 400:

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"error\":{\"code\":\"context_length_exceeded\",\"message\":\"Your input exceeds the context window of this model. Please adjust your input and try again.\",\"param\":\"input\",\"type\":\"invalid_request_error\"}}\n\nLiteLLM: model group 'mantle-luna' failed with the error above and no fallback model group was found for it, so the request was not retried on another model. Fallbacks are configured for: gem-temp, gem-404. Add a fallbacks entry for that model group (Router fallbacks or proxy router_settings.fallbacks) to retry on another model.","type":"invalid_request_error","param":null,"code":"400"}}

POST /v1/responses, stream

1. curl ... /v1/responses --data-binary @responses_stream.json
2. 200, response.created, response.in_progress, then event: response.failed whose response.error is:

{"code": "invalid_request_error", "message": "litellm.BadRequestError: BedrockException - Error code: 400 - {'error': {'message': 'Your input exceeds the context window of this model. Please adjust your input and try again.', 'type': 'invalid_request_error', 'code': 'context_length_exceeded'}}"}

POST /v1/responses

1. curl ... /v1/responses --data-binary @responses.json
2. 400:

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"error\":{\"code\":\"context_length_exceeded\",\"message\":\"Your input exceeds the context window of this model. Please adjust your input and try again.\",\"param\":\"input\",\"type\":\"invalid_request_error\"}}\n\nLiteLLM: model group 'mantle-luna' failed with the error above and no fallback model group was found for it, so the request was not retried on another model. Fallbacks are configured for: gem-temp, gem-404. Add a fallbacks entry for that model group (Router fallbacks or proxy router_settings.fallbacks) to retry on another model.","type":"invalid_request_error","param":null,"code":"400"}}

Other-provider 4xx fallbacks on streamed /v1/messages and /v1/chat/completions (regression guard)

1. curl ... /v1/messages --data-binary @gem_temp_messages_stream.json (Gemini rejects temperature: 7 with a 400, fallbacks points at gem-good)
2. 400, Gemini's own error and no fallback: {"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: Vertex_ai_betaException BadRequestError - b'{ \"error\": { \"code\": 400, \"message\": \"* GenerateContentRequest.generation_config.temperature: temperature must be in the range [0.0, 2.0].\", \"status\": \"INVALID_ARGUMENT\" } }'"}}
3. curl ... /v1/chat/completions --data-binary @gem_temp_chat_stream.json
4. 200, every chunk carries "model":"gem-good" and the text Hi there!
5. curl ... /v1/messages --data-binary @gem_404_messages_stream.json (the model does not exist, fallbacks points at gem-good)
6. 404, Gemini's own error and no fallback: {"type":"error","error":{"type":"not_found_error","message":"litellm.NotFoundError: Vertex_ai_betaException - b'{ \"error\": { \"code\": 404, \"message\": \"models/gemini-no-such-model-9 is not found for API version v1alpha, or is not supported for generateContent. ...\", \"status\": \"NOT_FOUND\" } }'"}}

Control, a small streamed /v1/messages call

1. curl ... /v1/messages --data-binary @small_messages_stream.json
2. 200, message_start, content_block_start, two content_block_delta, content_block_stop, message_delta with "stop_reason": "end_turn", message_stop; the text is Hi there

After (027f769)

Claude Code

1. Launch Claude Code as above, paste part A; Claude answers OK
2. Paste part B; the proxy log shows the overflow POST /v1/messages?beta=true answered 400 Bad Request with litellm.ContextWindowExceededError: ... prompt is too long: your prompt exceeds the model's context window, the pane shows Conversation compacted (ctrl+o for history), Claude resends the trimmed conversation (1,100,337 tokens compacted to 539,650 in 20.7 s) and answers OK 25 s after the submit

POST /v1/messages, stream

1. curl ... /v1/messages --data-binary @messages_stream.json
2. 400, a JSON body and no stream, 2.3 s:

{"type":"error","error":{"type":"invalid_request_error","message":"litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: your prompt exceeds the model's context window"}}

POST /v1/messages

1. curl ... /v1/messages --data-binary @messages.json
2. 400:

{"type":"error","error":{"type":"invalid_request_error","message":"litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: your prompt exceeds the model's context window\nmodel=mantle-luna. context_window_fallbacks=None. fallbacks=[{'gem-temp': ['gem-good']}, {'gem-404': ['gem-good']}].\n\nSet 'context_window_fallback' - https://docs.litellm.ai/docs/routing#fallbacks\n\nLiteLLM: model group 'mantle-luna' failed with the error above and no fallback model group was found for it, so the request was not retried on another model. Fallbacks are configured for: gem-temp, gem-404. Add a fallbacks entry for that model group (Router fallbacks or proxy router_settings.fallbacks) to retry on another model."}}

POST /v1/chat/completions, stream

1. curl ... /v1/chat/completions --data-binary @chat_stream.json
2. 400:

{"error":{"message":"litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: your prompt exceeds the model's context window","type":null,"param":null,"code":"400"}}

POST /v1/chat/completions

1. curl ... /v1/chat/completions --data-binary @chat.json
2. 400:

{"error":{"message":"litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: your prompt exceeds the model's context window\nmodel=mantle-luna. context_window_fallbacks=None. fallbacks=[{'gem-temp': ['gem-good']}, {'gem-404': ['gem-good']}].\n\nSet 'context_window_fallback' - https://docs.litellm.ai/docs/routing#fallbacks\n\nLiteLLM: model group 'mantle-luna' failed with the error above and no fallback model group was found for it, so the request was not retried on another model. Fallbacks are configured for: gem-temp, gem-404. Add a fallbacks entry for that model group (Router fallbacks or proxy router_settings.fallbacks) to retry on another model.","type":"invalid_request_error","param":null,"code":"400"}}

POST /v1/responses, stream

1. curl ... /v1/responses --data-binary @responses_stream.json
2. 200, response.created, response.in_progress, then event: response.failed whose response.error is:

{"code": "invalid_request_error", "message": "litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: your prompt exceeds the model's context window"}

POST /v1/responses

1. curl ... /v1/responses --data-binary @responses.json
2. 400:

{"error":{"message":"litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: your prompt exceeds the model's context window\nmodel=mantle-luna. context_window_fallbacks=None. fallbacks=[{'gem-temp': ['gem-good']}, {'gem-404': ['gem-good']}].\n\nSet 'context_window_fallback' - https://docs.litellm.ai/docs/routing#fallbacks\n\nLiteLLM: model group 'mantle-luna' failed with the error above and no fallback model group was found for it, so the request was not retried on another model. Fallbacks are configured for: gem-temp, gem-404. Add a fallbacks entry for that model group (Router fallbacks or proxy router_settings.fallbacks) to retry on another model.","type":"invalid_request_error","param":null,"code":"400"}}

Other-provider 4xx fallbacks on streamed /v1/messages and /v1/chat/completions (regression guard)

1. curl ... /v1/messages --data-binary @gem_temp_messages_stream.json (Gemini rejects temperature: 7 with a 400, fallbacks points at gem-good)
2. 400, Gemini's own error and no fallback, the same body as the merge base
3. curl ... /v1/chat/completions --data-binary @gem_temp_chat_stream.json
4. 200, every chunk carries "model":"gem-good" and the text Hi there!
5. curl ... /v1/messages --data-binary @gem_404_messages_stream.json (the model does not exist, fallbacks points at gem-good)
6. 404, Gemini's own error and no fallback, the same body as the merge base

Control, a small streamed /v1/messages call

1. curl ... /v1/messages --data-binary @small_messages_stream.json
2. 200, message_start, content_block_start, two content_block_delta, content_block_stop, message_delta with "stop_reason": "end_turn", message_stop; the text is Hi there

Observations from the run, both legs unless noted:

• Streamed /v1/messages never falls back on a Gemini 400 or 404, pre-existing on main
• Streamed /v1/chat/completions does fall back on that same Gemini 400
• Streamed /v1/responses overflow is 200 with response.failed, pre-existing
• Streamed chat 400 body carries "type": null, pre-existing
• Non-stream bodies append fallback guidance, stream bodies do not, pre-existing
• The generic message carries no token counts, this PR's own wording
• Since 1c2c8f3 the streamed /v1/messages overflow is an HTTP 400 JSON body before any stream frame (the first after run answered 200 with an api_error frame), and since 027f769 e.response on the non-streamed routes carries the provider's own 400 body

Audit round 3 of 3 (/audit)

Verdict: PASS. Every inventory row is VERIFIED by a checked-in tests/integration/ cell or a live cell, and the two decisions round 1 left open (D1 and D2 below) are closed at this tip

Rounds 1 and 2 ran at 6f994f0 and 1c2c8f3 with the same selection; round 1's matrix and its two decisions are in this PR's history. This round is the one that counts, at the tip

Legs. Merge base 249add6f0d and head 027f769c64, each a 2-worker proxy from tests/integration/_support/proxy.py on tests/integration/proxy_config.yaml with its own Postgres database, one shared Redis and the shared scripted upstream; the fallback cells boot their own 2-worker proxies from the same commit. The base leg runs the head's four test files on the base's product code, so a cell red there is a behavior the PR changes. Selection: tests/integration/providers/test_bedrock_mantle_context_overflow_wire.py, tests/integration/providers/test_bedrock_mantle_context_overflow_fallbacks_wire.py, tests/integration/providers/test_bedrock_mantle_wire.py, run through tests/integration/run.py providers --seed 4106601 --order-seed 0. Base run base-249add6f0d: 53 collected, 30 red, 23 green. Head run 1 head-027f769c64-1: 53 collected, 53 green. Head run 2 head-027f769c64-2, same seeds on a freshly booted head proxy: 53 collected, 53 green, identical selection, no skips or retries. Control ctl-027f769c64: the 3 pre-existing Mantle cells, 3 green. The cells the fix changes are the 30 red on the base; the 23 green on both legs are the cells the fix must not touch

Shared-rig cells, tests/integration/providers/test_bedrock_mantle_context_overflow_wire.py, Mantle deployment bedrock_mantle/openai.gpt-5.6-luna at a wire peer that answers /openai/v1/responses; "generic" is prompt is too long: your prompt exceeds the model's context window, "raw" is the provider's context_length_exceeded JSON, IRE is invalid_request_error

Row	Route	Mode	Client	Scenario	Test	Base	Head	Result
R1	/v1/chat/completions	non-stream	httpx	overflow envelope	test_chat_completions_overflow_envelope_returns_400_prompt_too_long_and_logs_failure	400 raw	400 generic, failure row	PASS
R2	/v1/chat/completions	stream, pre-stream 400	httpx	overflow envelope	test_chat_completions_stream_overflow_envelope_returns_400_prompt_too_long	400 raw	400 generic	PASS
R3	/v1/messages	non-stream	httpx	overflow envelope	test_messages_overflow_envelope_returns_400_invalid_request_and_logs_failure	400 raw	400 IRE generic, failure row	PASS
R4	/v1/messages	stream, pre-stream 400	httpx	overflow envelope	test_messages_stream_overflow_envelope_before_the_stream_returns_400_invalid_request	400 raw	400 IRE generic	PASS
R5	/v1/responses	non-stream	httpx	overflow envelope	test_responses_overflow_envelope_returns_400_prompt_too_long_and_logs_failure	400 raw	400 generic, failure row	PASS
R6	/v1/responses	stream, pre-stream 400	httpx	overflow envelope	test_responses_stream_overflow_envelope_before_the_stream_returns_400_prompt_too_long	400 raw	400 generic	PASS
R7	/v1/chat/completions	non-stream	OpenAI SDK	overflow envelope	test_openai_sdk_chat_completions_raises_bad_request_saying_prompt_too_long	BadRequestError raw	BadRequestError generic	PASS
R8	/v1/chat/completions	non-stream	AsyncOpenAI SDK	overflow envelope	test_async_openai_sdk_chat_completions_raises_bad_request_saying_prompt_too_long	BadRequestError raw	BadRequestError generic	PASS
R9	/v1/responses	non-stream	OpenAI SDK	overflow envelope	test_openai_sdk_responses_raises_bad_request_saying_prompt_too_long	BadRequestError raw	BadRequestError generic	PASS
R10	/v1/messages	non-stream	Anthropic SDK	overflow envelope	test_anthropic_sdk_messages_raises_bad_request_saying_prompt_too_long	BadRequestError raw	BadRequestError generic	PASS
R11	/v1/messages	non-stream	AsyncAnthropic SDK	overflow envelope	test_async_anthropic_sdk_messages_raises_bad_request_saying_prompt_too_long	BadRequestError raw	BadRequestError generic	PASS
R12	/v1/messages	stream, in-band response.failed	Anthropic SDK	overflow envelope	test_anthropic_sdk_messages_stream_raises_invalid_request_error_saying_prompt_too_long	error frame raw	IRE generic	PASS
R13	/v1/chat/completions	stream, in-band response.failed	httpx	overflow envelope	test_chat_completions_stream_overflow_in_response_failed_event_returns_400_prompt_too_long	400 raw	400 generic	PASS
R14	/v1/messages	stream, in-band response.failed	httpx	overflow envelope	test_messages_stream_overflow_in_response_failed_event_emits_invalid_request_error_event	message_start then an error frame, raw	IRE generic	PASS
R14-log	/v1/messages	stream, in-band response.failed	httpx	spend row for the R14 call	test_messages_stream_overflow_logs_one_failure_row (round 1's BUG: skip, now a live cell)	message_start then an error frame, raw	failure row by call id, generic	PASS, D1 closed
R14-retry	/v1/messages	stream, in-band response.failed	httpx	overflow with a retry budget on the deployment	test_messages_stream_overflow_with_a_retry_budget_returns_400_invalid_request_and_logs_failure	raw	400 IRE generic, one failure row	PASS
R15	/v1/responses	stream, in-band response.failed	httpx	overflow envelope	test_responses_stream_overflow_in_response_failed_event_relays_failure_saying_prompt_too_long	200 response.failed raw	200 response.failed generic	PASS
R16	/v1/chat/completions	non-stream	httpx	non-overflow 400	test_chat_completions_non_overflow_400_keeps_the_upstream_message	400 upstream text	same	PASS
R17	/v1/messages	stream, pre-stream 400	httpx	non-overflow 400	test_messages_stream_non_overflow_400_before_the_stream_returns_400_with_the_upstream_message	400 upstream text	same	PASS
R18	/v1/messages	stream, in-band response.failed	httpx	non-overflow invalid_prompt	test_messages_stream_non_overflow_response_failed_event_emits_the_upstream_invalid_request_error_event	IRE event, upstream text	same	PASS
R19	/v1/chat/completions	non-stream	httpx	upstream 401	test_chat_completions_other_upstream_statuses_keep_their_mapping[401]	401	same	PASS
R20	/v1/chat/completions	non-stream	httpx	upstream 429	test_chat_completions_other_upstream_statuses_keep_their_mapping[429]	429	same	PASS
R21	/v1/chat/completions	non-stream	httpx	upstream 500	test_chat_completions_other_upstream_statuses_keep_their_mapping[500]	503	same	PASS
R22	/v1/chat/completions	non-stream	httpx	upstream 404	test_chat_completions_other_upstream_statuses_keep_their_mapping[404]	404	same	PASS
R23	/v1/messages	stream	httpx	legacy token-count envelope	test_messages_stream_legacy_token_count_envelope_returns_400_with_the_counts	400 with counts	same	PASS
R24	/v1/chat/completions	non-stream	httpx	code-only envelope	test_chat_completions_code_only_overflow_envelope_returns_400_prompt_too_long	400 raw	400 generic	PASS
R25	/v1/chat/completions	non-stream	httpx	text/plain body naming the code	test_chat_completions_text_plain_body_naming_the_overflow_code_returns_the_plain_400	400 upstream text	same	PASS, see the code-key note
R25-quote	/v1/chat/completions	non-stream	httpx	JSON 400 whose message merely quotes the code	test_chat_completions_400_merely_quoting_the_overflow_code_keeps_the_upstream_message	400 upstream text	same	PASS
R26	/v1/chat/completions	non-stream	httpx	5 KB message, liveliness during	test_chat_completions_five_kilobyte_overflow_message_returns_400_and_keeps_the_proxy_alive	400 raw	400 generic, liveliness 200	PASS
R27	/v1/chat/completions	stream	httpx	response.failed with error: null	test_chat_completions_stream_response_failed_without_error_fails_and_keeps_the_proxy_alive	>= 400, alive	same	PASS
R28	/v1/chat/completions	non-stream	httpx	two overflows in a row	test_chat_completions_repeated_overflow_logs_one_failure_row_per_call	raw	two failure rows, generic	PASS
R29	/v1/chat/completions	non-stream	httpx	prompt naming the error code, happy peer	test_chat_completions_prompt_naming_the_error_code_still_succeeds	200	same	PASS
R30	/v1/messages	stream, in-band response.failed	httpx	openai/ deployment overflow	test_messages_stream_openai_deployment_overflow_in_the_stream_emits_invalid_request_error_event	IRE event	same	PASS, pre-existing class
R31	/v1/chat/completions	stream	httpx	openai/ deployment overflow	test_chat_completions_stream_openai_deployment_overflow_returns_400	400	same	PASS
R32	/v1/messages	non-stream	httpx	openai/ deployment overflow	test_messages_openai_deployment_overflow_returns_400_invalid_request	400 IRE	same	PASS
R33	all three	12 concurrent per phase, mixed stream	httpx	peer stopped mid-burst, revived	test_chaos_mantle_peer_outage_mid_burst_logs_every_call_once_and_keeps_the_proxy_alive	overflow cells raw	every tagged call logged once, generic, liveliness 200	PASS

Code-key note (R25, R25-quote). 027f769 matches context_length_exceeded only as the envelope's code key, in its plain, Python-repr and escaped-JSON spellings (the unit test test_bedrock_mantle_overflow_code_maps_in_every_serialization_of_the_envelope pins all three), so a text/plain body or a JSON message that merely names the code stays a plain 400 with the upstream text on both legs. Round 1's R25 expected the generic message for the text/plain body; that cell was renamed and its expectation flipped with the fix, which is the behavior a Mantle validation error quoting the code needs

Fallback cells, tests/integration/providers/test_bedrock_mantle_context_overflow_fallbacks_wire.py, two owned 2-worker proxies per leg: cwf carries only context_window_fallbacks: [{mantle: [fallback]}], fb only fallbacks: [{mantle: [fallback]}] plus a /mantle-passthrough pass-through to the Mantle peer; the fallback deployment is an openai/ peer that answers fallback answered

Row	Knob	Route	Mode	Test	Base	Head	Result
R34	cwf	/v1/chat/completions	non-stream	test_context_window_fallback_fires_for_the_overflow_on_chat_completions	400 raw, no fallback	200 fallback	PASS
R35	cwf	/v1/messages	non-stream	test_context_window_fallback_fires_for_the_overflow_on_messages	400 raw, no fallback	200 fallback	PASS
R36	cwf	/v1/responses	non-stream	test_context_window_fallback_fires_for_the_overflow_on_responses	400 raw, no fallback	200 fallback	PASS
R37	cwf	/v1/chat/completions	stream	test_context_window_fallback_does_not_fire_on_streamed_chat_completions	400 raw	400 generic, no fallback	PASS
R38	cwf	/v1/messages	stream, in-band	test_context_window_fallback_is_not_consulted_on_streamed_messages	error frame raw	400 IRE generic, no fallback	PASS, pre-existing path
R39	cwf	/v1/responses	stream, in-band	test_context_window_fallback_does_not_fire_on_streamed_responses	response.failed raw	response.failed generic, no fallback	PASS
R40	fb	/v1/chat/completions	non-stream	test_plain_fallback_fires_for_the_overflow_on_chat_completions	200 fallback	same	PASS
R41	fb	/v1/messages	non-stream	test_plain_fallback_fires_for_the_overflow_on_messages	200 fallback	same	PASS
R42	fb	/v1/responses	non-stream	test_plain_fallback_fires_for_the_overflow_on_responses	200 fallback	same	PASS
R43	fb	/v1/messages	stream, in-band	test_streamed_messages_overflow_on_a_plain_fallback_deployment_returns_400_without_falling_back	error frame raw	400 IRE generic, no fallback	PASS, D2 closed
R44	fb	/v1/chat/completions	stream	test_plain_fallback_does_not_fire_on_streamed_chat_completions	400 raw	400 generic, no fallback	PASS
R45	fb	/v1/responses	stream, in-band	test_plain_fallback_does_not_fire_on_streamed_responses	response.failed raw	response.failed generic	PASS
R46	fb	/v1/messages	stream, in-band	test_streamed_messages_overflow_on_an_openai_fallback_deployment_returns_the_error_event_without_falling_back	IRE event, no fallback	same	PASS, pre-existing class
R47	fb	/v1/messages	stream, in-band invalid_prompt	test_streamed_messages_non_overflow_failure_returns_400_without_falling_back	400 upstream text, no fallback	same	PASS
R48	fb	/mantle-passthrough	non-stream	test_pass_through_relays_the_overflow_envelope_verbatim	400 raw verbatim	same	PASS
R49	fb	/mantle-passthrough	stream	test_pass_through_relays_the_failed_stream_frames_verbatim	frames verbatim	same	PASS
R50	fb	/v1/chat/completions	one worker killed through the harness's replacing_worker	test_killing_one_worker_leaves_the_sibling_serving_the_fallback	200 fallback	same	PASS

R40 to R42 and R43 together are the fallback rule at the tip: a deployment whose router_settings carry only plain fallbacks still takes them for the overflow on every non-streamed route, and no streamed route takes any fallback. The Medium caveat is the third configuration, plain fallbacks next to a context_window_fallbacks list that has no entry for the Mantle group, which the live fb_chat leg in the blast radius drives

Existing cells, tests/integration/providers/test_bedrock_mantle_wire.py: test_bedrock_mantle_context_overflow_returns_400_saying_prompt_is_too_long, test_chat_completions_bridge_signs_mantle_responses_request_with_deployment_aws_keys, test_bedrock_mantle_messages_stream_relays_anthropic_sse_instead_of_failing_on_event_stream_decode: green on the control leg (R51 to R53)

Live cells, the real Mantle deployment bedrock_mantle/openai.gpt-5.6-luna behind the head 027f769c64 and base 249add6f0d proxies of the blast radius run, artifacts under its legs/<side>/

Row	Route	Mode	Base	Head	Result
L1	/v1/chat/completions	non-stream	400 raw	400 generic	PASS
L2	/v1/chat/completions	stream	400 raw	400 generic	PASS
L3	/v1/messages	non-stream	400 raw	400 generic	PASS
L4	/v1/messages	stream	400 raw	400 IRE generic	PASS
L5	/v1/responses	non-stream	400 raw	400 generic	PASS
L6	/v1/responses	stream	200 response.failed raw	200 response.failed generic	PASS

Not separate rows, and why: the provider pass-through routes (/bedrock/..., /anthropic/..., /vertex_ai/..., /v1beta/...) forward the provider's own frames and never reach _error_status_and_message, whose only caller is the /v1/messages adapter's mid-stream error frame (R48 and R49 cover the pass-through shape for the Mantle target); the one regex the diff adds, _BEDROCK_MANTLE_CONTEXT_LENGTH_EXCEEDED_CODE_PATTERN, is three literal tokens joined by \s* with no nested quantifier, so its cost is linear in the body and R26's 5 KB message is its timing row; the diff touches no hook, callback, metric or logging payload, so no response-cache twin rows

Decisions from round 1, both closed at this tip, one root cause then: on streamed /v1/messages a provider failure mapped before the first content chunk became a 200 invalid_request_error frame inside async_anthropic_sse_wrapper, which the router declined to retry and the proxy never logged

• D1 (R14-log): closed. Main's fix(anthropic): let /v1/messages mid-stream failures reach the proxy failure boundary #44800, merged into this branch, answers a provider error raised before any content as a JSON error with that error's own status, and the adapter dispatches the failure handlers before it emits any error frame, so the overflow writes its failure row on both legs; the round-1 skip is the live cell R14-log, and the blast radius spend rows show the same on the real deployment
• D2 (R43): closed by convention, not by code. The router consults only context_window_fallbacks for a ContextWindowExceededError when any such list is present and takes plain fallbacks otherwise (R40 to R42), the rule for every provider's overflow; R43 pins that the streamed path takes neither, and the configuration that loses a fallback it had on the base is the Medium caveat with its one-line remedy

Type

🐛 Bug Fix

Caveats (if any)

Medium

• With a context_window_fallbacks list in router_settings, a Mantle overflow no longer takes the deployment's plain fallbacks entry
  • The router consults only context_window_fallbacks for a ContextWindowExceededError when any such list is present (router.py 7747 to 7778), the rule for every provider's overflow, and raises when the group has no entry there. On the merge base the plain entry answered fb_chat with 200 from the fallback model in 27.0 s because the error was a plain BadRequestError; at the tip it answers 400 No fallback was attempted in 3.0 s. A deployment with plain fallbacks alone keeps them on every non-streamed route (audit R40 to R42). The context-window class is what this change exists to give the overflow, so the gate is the requested behavior. Remedy: add the Mantle group to context_window_fallbacks. Changing the gate would change every provider, out of scope here; named in the TLDR

Low

• Mantle's gpt-oss models report an overflow as Input length (N) exceeds model's maximum context length (M), a third wording that neither Mantle pattern nor the generic overflow substrings match
  • Pre-existing on main and outside this change's envelope: at the tip bedrock_mantle/openai.gpt-oss-120b with a 400k-token prompt raises a plain BadRequestError 400 non-streamed and MidStreamFallbackError wrapping ServiceUnavailableError 503 streamed, the same as the merge base. An internal follow-up ticket tracks it
• The chat-mode stream path (_extract_error_from_chunk) forwards only an in-stream chunk's error.message and assigns 500 when its code is a string, so a code-key envelope arriving in-stream on a chat-mode model would reach the new pattern without its key
  • Not observed live: the code-key form came only from the Responses-mode openai.gpt-5.6-luna deployment, whose envelope the R1 to R25 cells cover; the other deployments seen (google.gemma-4-e2b in chat mode, openai.gpt-5.6-sol) emit the token-count form, which the chat-mode stream path maps at the tip to a 400 ContextWindowExceededError (the gemma_chat_stream leg), and gpt-oss emit the wording above
• Open PR fix(anthropic): send the provider's error type and text on the SDK adapter's mid-stream error frame #45513 rewrites _error_status_and_message on the Anthropic adapter's mid-stream frame to pass every provider status through, while this change's test pins other 4xx to 500 api_error
  • No user-visible gap at this tip; whichever merges second merges main and re-pins its test, and fix(anthropic): send the provider's error type and text on the SDK adapter's mid-stream error frame #45513, still open, merges main after this one
• The Before legs ran on the earlier merge base 988cc7d, the After on 249add6
  • Between the two bases exception_mapping_utils.py is unchanged, router.py gains 14 lines outside the overflow gate (feat: forward-port dd user_email span tag and model hub description from internal staging #42749, feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape #45846) and bedrock_mantle/chat/transformation.py gains max_completion_tokens for gpt-5.6 and gpt-6 chat (fix(bedrock_mantle): send max_completion_tokens for gpt-5.6 and gpt-6 chat completions #43982), none on the overflow path; the audit's base cells ran at 249add6
• The code-key pattern matches "code": "context_length_exceeded" anywhere in the error text, a nested or escaped envelope included
  • By design: the envelope reaches the mapper escaped inside BedrockException - ... wrappers, and a 400 whose message merely quotes the code stays a plain BadRequestError (R25-quote)
• No streamed route falls back on an overflow, plain or context-window
  • Pre-existing on main for every provider: streamed /v1/chat/completions and /v1/messages raise before any content, streamed /v1/responses answers 200 with response.failed (audit R37 to R39, R43 to R46)
• A streamed /v1/responses overflow still lands as response.failed inside HTTP 200
  • Pre-existing, the same on the merge base; only the message changes here
• The generic message has no token counts, so Claude Code falls back to its own accounting
  • Mantle sends none in this envelope, nothing to surface
• Clients matching the raw context_length_exceeded text in the message now get prompt is too long
  • Fix judged net-negative: the other Mantle envelope already dropped the raw text in fix(responses): map Bedrock Mantle context overflow to ContextWindowExceededError #37862, ContextWindowExceededError is the class clients key on, and since 027f769 e.response still carries the provider's own 400 body
• The sync Anthropic SDK messages.create(stream=True) ends after message_start with no error on a streamed overflow
  • Pre-existing on both legs; the async client gets the invalid_request_error event. An internal follow-up ticket tracks it
• A success row for a Responses-bridged Mantle call is keyed by the upstream resp_ id
  • Pre-existing on both legs, outside this diff. An internal follow-up ticket tracks it
• A 2-worker proxy under box load stalled one sibling request past its 15 s client timeout while POST /model/new ran, twice in audit round 1 and never in the six later runs
  • Outside the diff (the stalled cells call code the PR does not touch). An internal follow-up ticket tracks it

Blast radius (/live-pr-risk)

Round 4 ran at the tip 027f769 (round 3's tip 1c2c8f3 merged with origin/main 249add6 as 5aa1471, plus the round-4 fix below) against a merge-base proxy on 249add6, the current merge base, so the tip already carries main and no separate merged tree was needed. Each side is one proxy with two uvicorn workers on its own native Postgres, the same config and the same scenario order as round 2's wave 4 (plain fallbacks and context_window_fallbacks lists next to each other, Gemini as the fallback target, the Responses-mode openai.gpt-5.6-luna and the chat-mode google.gemma-4-e2b), with a forwarding recorder in front of Mantle on both the bare host and the /openai/v1 path

Verdict first. The PR does what it says on both Mantle models on every route, round 2's two findings stay closed (one by the status fix, one by the router convention named in the Caveats), and round 4's fix restores the provider body on the mapped error without moving any other leg

Breaking. With a context_window_fallbacks list in router_settings, a Mantle overflow no longer takes the deployment's plain fallbacks: the base answered fb_chat with 200 from gemini-3.8-flash in 27.0 s, and the tip answers 400 litellm.ContextWindowExceededError ... No fallback was attempted. in 3.0 s. The graph edge is the router's fallback gate (router.py 7747 to 7778), which on a ContextWindowExceededError with context_window_fallbacks set looks up only that list and raises the original exception when the group has no entry, the standing rule for every provider's ContextWindowExceededError rather than Mantle code. Shipped as the Medium caveat below and named in the TLDR's intentional-change line, the remedy being an entry for the Mantle group in context_window_fallbacks

Fixed in 1c2c8f3, kept at the tip. Round 2 found the chat-mode streamed /v1/messages overflow answering HTTP 500 api_error, because the new ContextWindowExceededError inherited the 500 the OpenAI chunk parser gives an in-stream error event. Observed at the tip: gemma_messages_stream answers HTTP 400 JSON {"type":"error","error":{"type":"invalid_request_error","message":"litellm.ContextWindowExceededError: litellm.BadRequestError: prompt is too long: 229378 tokens > 131072 maximum"}} after one upstream attempt in 2.2 s (the base's 200 plus event: error of type api_error came after three attempts in 8.7 s), gemma_chat_stream answers 200 with an empty first delta and an error chunk carrying "code": "400" (the base "503"), and through the Anthropic SDK (litellm.anthropic_interface.messages, async client) the error event after message_start is invalid_request_error

Fixed in 027f769. Devin Review found that 1c2c8f3 built the mapped error on a bodiless synthesized 400, so an SDK caller reading e.response.json() lost the provider's envelope the base still carried. The mapper now reuses the provider's own 400 response object (status, headers and body intact) and synthesizes a 400 with the original headers and request only when the original status is not 400 (the in-stream chunk-parser error, which has no body on either side). It also matches context_length_exceeded only as the envelope's code key (plain, Python-repr and escaped-JSON serializations), so a Mantle 400 whose message merely quotes the words stays a plain BadRequestError with the upstream message. Observed at the tip: every non-streamed overflow leg answers the same 400 as round 3 with the same spend row, and the 16 legs match round 3's tip leg for leg

Backward incompatible. The error class on every Mantle overflow moves from BadRequestError to its subclass ContextWindowExceededError, so except BadRequestError still catches it and the non-streamed status stays 400, and the spend row's error_class moves with it (every overflow leg at the tip wrote a failure row with ContextWindowExceededError). The message loses the provider text, BedrockException - {"error":{"code":"context_length_exceeded",... becomes prompt is too long: your prompt exceeds the model's context window on the context_length_exceeded envelope and prompt is too long: 229378 tokens > 131072 maximum on the token-count one, while e.response keeps the provider's 400 body on the non-streamed routes since 027f769. context_window_fallbacks newly fire for the overflow on the non-streamed routes (cwf_chat 200 from gemini-3.8-flash in 25.0 s at the tip against 400 in 5.5 s on the base) and on none of the streamed ones (cwf_messages_stream 400 on both sides), so a client that sent an oversized streamed request and handled the 400 still gets the 400, and a client that sent it non-streamed gets a slower answer from a different model

Regression risk. Pre-existing on both sides and outside the PR, named so nobody reads them as PR findings: no streamed route falls back on an overflow, plain or context-window (streamed chat and /v1/messages raise before content, streamed /v1/responses answers 200 with response.failed); the sync Anthropic SDK create(stream=True) ends silently after message_start on both sides; the two streamed chat-mode failure rows carry the same estimated 0.0116 spend on the base (its 503 rows) and the tip; a success row for a Responses-bridged call is keyed by the upstream resp_ id (SLP-882, filed by round 2). Every other construction of ContextWindowExceededError in exception_mapping_utils.py (twelve other providers) still passes the original exception's response through, so an in-stream overflow mapped there would carry the same inherited status this PR fixed for Mantle, untouched here since none of those paths is in the diff

Dependency graph. Re-walked at the tip. _bedrock_mantle_context_window_response is called only from the Mantle branch of _map_bedrock_exception, which exception_type reaches under custom_llm_provider == "bedrock_mantle", and what it returns is read by openai.APIStatusError.__init__ (status, headers, body) and by nothing else; _BEDROCK_MANTLE_CONTEXT_LENGTH_EXCEEDED_CODE_PATTERN is read only by _get_bedrock_mantle_context_window_message, whose callers are unchanged since round 2, and _error_status_and_message in the Anthropic adapter keeps the callers named there. The outbound wire is identical at the tip and on the base for every leg but one: 15 recorded requests at the tip against 17 on the base, the same four header keys (accept, authorization, content-type, user-agent), the same paths (/openai/v1/responses for the Responses-mode legs, /chat/completions on the /openai/v1 recorder for the chat-mode legs), the same body sizes request for request, and the streamed chat-mode /v1/messages overflow at one attempt where the base took three. Verified live at the tip: the six overflow routes on the Responses-mode model, the five on the chat-mode model (three through the recorder, the Responses stream direct), the plain and context-window fallback legs, the Anthropic SDK bridge in both clients, the two control calls, and /health; the base rows ran on 249add6 on the same config. Legacy suites asserting on context window mapping passed on round 2's tip, and the product diff since then is the status helper, the response reuse and the code-key match above, each covered by the unit tests the fix commits add

Not verified. openai.gpt-oss-120b, the chat-mode model the PR's own wire tests name, answers 404 on this account on every route, so the chat-mode legs ran on google.gemma-4-e2b. The chat-mode Responses stream ran only direct, not through a recorder. Codex and the OpenAI SDK against streamed /v1/responses were not driven (the shape there is unchanged and the curl leg answered 200 with response.failed on every side)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 027f769 passes /live-pr-risk

• 027f769 passes /audit

---

Note

Medium Risk
Changes error class and streamed /v1/messages overflow shape (500 → 400 SSE), which stops plain fallbacks on that path and skips spend logging for streamed messages overflow; routing and observability for Mantle overflow are affected beyond the mapping fix.

Overview
Bedrock Mantle’s OpenAI-shaped context_length_exceeded errors (400 JSON or streamed response.failed without an error type) now map to ContextWindowExceededError with a prompt is too long message, including when the old invalid_request_error / validation_error gate would have skipped streamed events. Token-count validation_error bodies still get the numeric N tokens > M maximum wording.

On streamed /v1/messages, ContextWindowExceededError is surfaced as a 400 invalid_request_error SSE error event (not a 500 api_error), so clients like Claude Code can compact; other provider 4xx mid-stream errors stay 500 so plain fallbacks still retry. context_window_fallbacks apply on non-streamed chat/messages/responses; streamed overflow paths do not fall back (documented behavior change vs accidental 500 fallback).

Large wire/integration and unit test coverage documents mapping, fallback knobs, pass-through verbatim relay, and edge cases.

^{Reviewed by Cursor Bugbot for commit 750ba1e. Bugbot is set up for automated code reviews on this repo. Configure here.}

---

[greptile-apps]

Confidence Score: 4/5

[Medium risk] Expands context overflow detection for one provider.

The latest change does not introduce a demonstrated failure, but the unresolved test-placement requirement needs attention before merging

Findings

1.  Tests split across files ▶

Summary

The PR maps Bedrock Mantle’s OpenAI-shaped context overflow to a recognizable context-window error and emits it as an Anthropic stream error event. The latest change restores streamed Messages overflow to the concurrent outage test while keeping its known missing-spend-row assertion in a skipped test

_{Reviews (10) · Last reviewed commit: "test(integration): keep every chaos cell..."}

[codspeed]

Merging this PR will not alter performance

⚠️ 1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• test_completion_streaming

✅ 31 untouched benchmarks

---

_{Comparing mayank-affirm:fix/mantle-context-overflow-openai-envelope (027f769) with main (19327a6)}

[mayank-affirm]

Follow-up to #37862, which covered only one of the two envelopes Mantle uses for context overflow.

Original report: #36546

The other envelope, reproduced live against a LiteLLM proxy in front of bedrock_mantle:

{"error":{"code":"context_length_exceeded",
          "message":"Your input exceeds the context window of this model. Please adjust your input and try again.",
          "param":"input","type":"invalid_request_error"}}

It passes the invalid_request_error gate in _get_bedrock_mantle_context_window_message, but matches neither _BEDROCK_MANTLE_CONTEXT_WINDOW_PATTERN nor ExceptionCheckers.is_error_str_context_window_exceeded's substring list, so it reaches callers as a generic BadRequestError carrying the raw provider text. Clients that key recovery off a recognised overflow phrase (Claude Code matches prompt is too long / input is too long for requested model at 400; context window is 413-only) then never run reactive compaction — the downstream symptom is /compact failing outright on an over-window conversation.

This change returns the same normalised prompt is too long: ... message for that envelope, keeping the numeric wording when the provider supplies counts. @yassin-berriai — same area as #37862, would you take a look?

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
3 out of 5 committers have signed the CLA.

✅ milan-berri
✅ ryan-crabbe-berri
✅ mateo-berri
❌ yassin-berriai
❌ mayank-affirm
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ellm/litellm_core_utils/exception_mapping_utils.py	93.33%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[greptile-apps]

Tests split across files This bug fix adds two integration files despite existing Mantle overflow coverage. Repository guidance requires extending the existing test file before merging

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[mateo-berri]

That rule covers tests/unit mirroring litellm. tests/integration lays files out per scenario, like the other Mantle files; the existing file holds one pre-stream 400 cell

[greptile-apps]

You’re right — this is an integration-test organization convention, not a duplication problem. test_bedrock_mantle_wire.py contains the existing legacy token-count overflow and unrelated Mantle wire checks, while test_bedrock_mantle_context_overflow_wire.py groups the new OpenAI-shaped overflow scenario and its route, client, and streaming coverage. The separate file is appropriate here, so I’m withdrawing that comment.

[mateo-berri]

@greptileai

[devin-ai-integration]

Devin Review found 3 potential issues.

[devin-ai-integration]

🔴 Streamed overflow loses failure spend logs

When a Mantle stream reports overflow, ContextWindowExceededError becomes a 400 error frame. The router passes that frame through without running the proxy failure hook, leaving no spend log for the rejected call.

Learn more

A streamed Responses API failure can arrive as a response.failed event rather than an HTTP error. _map_stream_error_to_exception now maps Mantle's overflow event to a 400 context error. This branch emits a 400 Anthropic frame. The router frame gate forwards non-retryable 400 frames without raising; async_anthropic_sse_wrapper does not call the proxy failure hook when it emits an error frame. The proxy therefore never records a failed spend row for this request.

Example: A Mantle deployment returns response.failed with code=context_length_exceeded during a streamed /v1/messages request. The client receives a 200 SSE response with an invalid_request_error event, but the request has no LiteLLM_SpendLogs failure row.

Recommended fix: Ensure the proxy-managed Anthropic stream error-frame path calls the failure hook exactly once for the mapped provider error, including when the router forwards a non-retryable frame. Keep the 400 frame for client compaction and add an integration assertion for a failed spend row on this route.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[mateo-berri]

The adapter dispatches failure handlers before emitting the error frame, so the failure spend row is still written, and test_messages_stream_overflow_logs_one_failure_row pins it

[mateo-berri]

Live check at 027f769, the streamed /v1/messages overflow wrote a failure spend row with ContextWindowExceededError on both Mantle models

[devin-ai-integration]

🟡 Quoted error codes trigger false overflow

When a Mantle validation error quotes context_length_exceeded in its message, _get_bedrock_mantle_context_window_message treats it as overflow. Valid requests can then trigger context-window fallbacks or compaction instead of returning the actual validation error.

Learn more

The mapper receives a stringified provider exception, not a parsed error code. It accepts the marker anywhere in that string without checking the structured error.code value or HTTP status. A provider rejection that quotes caller-supplied text containing the marker therefore becomes a 400 overflow. The removed type gate also broadens the legacy token-count recognition to error types that were previously excluded.

Example: A Mantle 400 body {"error":{"code":"invalid_request_error","message":"Invalid input value context_length_exceeded"}} is now returned as ContextWindowExceededError. Its actual code does not identify context overflow.

Recommended fix: Parse the envelope or inspect the exception's structured body/code and compare the exact error.code to context_length_exceeded; retain a narrowly scoped token-count fallback for the legacy validation envelope. Preserve existing mapping for other statuses and codes.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[mateo-berri]

Since 027f769 the mapper matches context_length_exceeded only as the envelope's code key, so a message merely quoting it stays a plain 400

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 027f769. Configure here.}

[mateo-berri]

LGTM. Thanks for the contribution!