Repository navigation
Description
Problem
When a routed upstream call ends in HTTP 429 or 503, Switchyard preserves the status and error message but drops the upstream's client-facing retry and correlation headers. A caller cannot use Retry-After to schedule its next request or use the upstream request ID to investigate the failure.
Reproduction
Use an openai_chat passthrough route to a loopback HTTP mock with max_retries = 0 and failure_cooldown_ms = 0. Send POST /v1/chat/completions with a valid route model. Have the mock return each of 200, 429, and 503 with these headers:
Retry-After: 15
X-RateLimit-Reset-Requests: 15s
X-Request-Id: req_test_1The 200 response forwards the allowed rate-limit and request-ID headers. The routed 429 and 503 responses retain their status and error body but lose all three headers. A raw-fallback 429 preserves them. The same routed failure occurs through /v1/messages with an anthropic_messages backend. This was reproduced with a running Switchyard server and local mock; no provider credentials or external calls were needed.
Expected behavior
A terminal routed upstream error should carry a small, explicit allowlist of safe retry, rate-limit, and correlation headers from the final failed attempt. Headers from earlier attempts must not leak after a retry or candidate fallback. Do not forward authorization, cookies, or hop-by-hop headers, and do not change the existing status or error-body format.
Likely cause
The upstream client retains Retry-After for its internal retry delay, then reduces a terminal HTTP failure to UpstreamHttp { status, body }. The server creates a new error response from those fields, with no upstream headers to forward. The existing response-header forwarding work in #571 covers successful responses, not this terminal error path.
Acceptance criteria
- Routed 429 and 503 responses preserve safe final-attempt retry/rate-limit/request-ID headers for OpenAI Chat and Anthropic Messages callers.
- Internal retries and candidate fallback never expose headers from an earlier failed attempt.
- Untrusted, credential, cookie, and hop-by-hop headers are not forwarded.
- Status, provider-appropriate error body, and existing retry/fallback behavior remain unchanged.
- Focused tests use a loopback mock and include a successful control.
Activity
gschaetz commentedon Oct 16, 2026
A concrete downstream use case for this, in case it helps prioritize #904.
I'm building humpyard, a gateway that embeds
switchyard-llm-client/libsyas a library and fails over across free and paid OpenAI-compatible endpoints. It keeps a per-endpoint circuit breaker, and for rate-limited free tiers the right cooldown is whatever the provider says (Retry-After), not a guessed backoff. TodayLlmClientError::UpstreamHttp { status, body }is all a library consumer gets, so the header thatretry_after_delayalready parses is invisible to us: it only paces Switchyard's own internal retries.The shape in #904 (the error carrying an allowlisted header map) is exactly what a library consumer needs, and the allowlist keeps credentials out. Until it lands I'm parsing the "try again in 7.6s" hints that some providers put in the error body, as a stopgap, and I'd switch to the header the moment a release has it.
Happy to test a branch of #904 against a real multi-provider setup and report back if that would help.