Skip to content

[bug] Replayed web_search_call items 400 on Responses-compatible local servers #954

Open

Description

@mrPronin

Symptom

A conversation where a hosted OpenAI Responses model ran a server-side web search fails on every later request that a route sends to a Responses-compatible local server (llama.cpp). The local server answers 400 Cannot determine type of 'item', so the whole session is stuck once the history holds a web_search_call.

Reproduction

A route that mixes a hosted openai_responses target with a local one, e.g. an llm_classifier route whose judge and default target run on llama.cpp's /v1/responses, and whose stronger targets are hosted:

[llm_clients.local]
format = "openai_responses"
base_url = "http://127.0.0.1:8082/v1"   # llama.cpp llama-server

[llm_clients.hosted]
format = "openai_responses"
base_url = "https://chatgpt.com/backend-api/codex"
forward_auth = true
  1. Run Codex against the route. A turn goes to the hosted target, which runs a web search. Codex records it and replays it in every later request as an input item:
    {"type": "web_search_call", "id": "ws_…", "status": "completed",
     "action": {"type": "search", "query": "…"}}
  2. The next turn is classified by the local judge. Switchyard forwards input unchanged to llama.cpp.

The same thing reproduces with one request to POST /v1/responses whose input is [user message, web_search_call, assistant message, user message]. Without the web_search_call item the request succeeds.

Expected vs. actual

  • Expected: the local leg gets a request it can parse. The search's findings are already in the assistant message that follows the item.
  • Actual:
    WARN switchyard_server::request: LLM request failed wire_format=openai_responses status=400
         requested_model="<route>" selected_model="" error="upstream_error (HTTP 400 Bad Request)"
    {"error":{"message":"Cannot determine type of 'item'","type":"upstream_error","code":"upstream_error"}}
    
    selected_model="": the request fails in the classifier, before any target is chosen. Every later turn in that session fails the same way.

Environment

  • Switchyard version (or commit SHA): main at 3cf60ce (also seen on fbabf51)
  • Python version: n/a (Rust server)
  • OS / arch: Debian 13, x86_64
  • Install path: source build (cargo build --release -p switchyard-server)
  • Inbound format: Responses (Codex CLI 0.161.0)
  • Backend: hosted OpenAI Responses (ChatGPT) plus llama.cpp llama-server /v1/responses

Additional context

sanitize_openai_responses_provider_body runs for every openai_responses backend, hosted ones included, so dropping the item there would change what hosted providers see. A per-target opt-in like omit_body_fields, scoped to input item types, keeps hosted behaviour unchanged. A PR doing that follows.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions