Repository navigation
Description
Symptom
A conversation where a hosted OpenAI Responses model ran a server-side web search fails on every later request that a route sends to a Responses-compatible local server (llama.cpp). The local server answers 400 Cannot determine type of 'item', so the whole session is stuck once the history holds a web_search_call.
Reproduction
A route that mixes a hosted openai_responses target with a local one, e.g. an llm_classifier route whose judge and default target run on llama.cpp's /v1/responses, and whose stronger targets are hosted:
[llm_clients.local]
format = "openai_responses"
base_url = "http://127.0.0.1:8082/v1" # llama.cpp llama-server
[llm_clients.hosted]
format = "openai_responses"
base_url = "https://chatgpt.com/backend-api/codex"
forward_auth = true- Run Codex against the route. A turn goes to the hosted target, which runs a web search. Codex records it and replays it in every later request as an input item:
{"type": "web_search_call", "id": "ws_…", "status": "completed", "action": {"type": "search", "query": "…"}} - The next turn is classified by the local judge. Switchyard forwards
inputunchanged to llama.cpp.
The same thing reproduces with one request to POST /v1/responses whose input is [user message, web_search_call, assistant message, user message]. Without the web_search_call item the request succeeds.
Expected vs. actual
- Expected: the local leg gets a request it can parse. The search's findings are already in the assistant message that follows the item.
- Actual:
WARN switchyard_server::request: LLM request failed wire_format=openai_responses status=400 requested_model="<route>" selected_model="" error="upstream_error (HTTP 400 Bad Request)" {"error":{"message":"Cannot determine type of 'item'","type":"upstream_error","code":"upstream_error"}}selected_model="": the request fails in the classifier, before any target is chosen. Every later turn in that session fails the same way.
Environment
- Switchyard version (or commit SHA):
mainat 3cf60ce (also seen on fbabf51) - Python version: n/a (Rust server)
- OS / arch: Debian 13, x86_64
- Install path: source build (
cargo build --release -p switchyard-server) - Inbound format: Responses (Codex CLI 0.161.0)
- Backend: hosted OpenAI Responses (ChatGPT) plus llama.cpp
llama-server/v1/responses
Additional context
sanitize_openai_responses_provider_body runs for every openai_responses backend, hosted ones included, so dropping the item there would change what hosted providers see. A per-target opt-in like omit_body_fields, scoped to input item types, keeps hosted behaviour unchanged. A PR doing that follows.
Activity