Skip to content

[bug] Routed upstream errors drop retry and correlation headers #903

Open
chethanuk/Switchyard
#11

Description

@cpakkamisaac-sae

Problem

When a routed upstream call ends in HTTP 429 or 503, Switchyard preserves the status and error message but drops the upstream's client-facing retry and correlation headers. A caller cannot use Retry-After to schedule its next request or use the upstream request ID to investigate the failure.

Reproduction

Use an openai_chat passthrough route to a loopback HTTP mock with max_retries = 0 and failure_cooldown_ms = 0. Send POST /v1/chat/completions with a valid route model. Have the mock return each of 200, 429, and 503 with these headers:

Retry-After: 15
X-RateLimit-Reset-Requests: 15s
X-Request-Id: req_test_1

The 200 response forwards the allowed rate-limit and request-ID headers. The routed 429 and 503 responses retain their status and error body but lose all three headers. A raw-fallback 429 preserves them. The same routed failure occurs through /v1/messages with an anthropic_messages backend. This was reproduced with a running Switchyard server and local mock; no provider credentials or external calls were needed.

Expected behavior

A terminal routed upstream error should carry a small, explicit allowlist of safe retry, rate-limit, and correlation headers from the final failed attempt. Headers from earlier attempts must not leak after a retry or candidate fallback. Do not forward authorization, cookies, or hop-by-hop headers, and do not change the existing status or error-body format.

Likely cause

The upstream client retains Retry-After for its internal retry delay, then reduces a terminal HTTP failure to UpstreamHttp { status, body }. The server creates a new error response from those fields, with no upstream headers to forward. The existing response-header forwarding work in #571 covers successful responses, not this terminal error path.

Acceptance criteria

  • Routed 429 and 503 responses preserve safe final-attempt retry/rate-limit/request-ID headers for OpenAI Chat and Anthropic Messages callers.
  • Internal retries and candidate fallback never expose headers from an earlier failed attempt.
  • Untrusted, credential, cookie, and hop-by-hop headers are not forwarded.
  • Status, provider-appropriate error body, and existing retry/fallback behavior remain unchanged.
  • Focused tests use a loopback mock and include a successful control.

Activity

  1. gschaetz commented on Oct 16, 2026

    @gschaetz

    A concrete downstream use case for this, in case it helps prioritize #904.

    I'm building humpyard, a gateway that embeds switchyard-llm-client / libsy as a library and fails over across free and paid OpenAI-compatible endpoints. It keeps a per-endpoint circuit breaker, and for rate-limited free tiers the right cooldown is whatever the provider says (Retry-After), not a guessed backoff. Today LlmClientError::UpstreamHttp { status, body } is all a library consumer gets, so the header that retry_after_delay already parses is invisible to us: it only paces Switchyard's own internal retries.

    The shape in #904 (the error carrying an allowlisted header map) is exactly what a library consumer needs, and the allowlist keeps credentials out. Until it lands I'm parsing the "try again in 7.6s" hints that some providers put in the error body, as a stopgap, and I'd switch to the header the moment a release has it.

    Happy to test a branch of #904 against a real multi-provider setup and report back if that would help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions