You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Feature: Flush response headers immediately for streaming requests #40
When a Chat Completions provider has high time-to-first-token (e.g., GLM-5.2 with large context can take 20-90s before first token), codex-relay does not flush the HTTP response headers until the first SSE event arrives.
This causes client-side idle timeouts ("stream disconnected before completion: idle timeout waiting for SSE") because the client sees no HTTP response at all during the TTFT period.
Current workaround
Proxy scripts must:
Send HTTP 200 + headers to the client immediately (before forwarding to codex-relay)
Inject SSE keepalive comments (": keepalive\n\n") every 5 seconds during TTFT
Stop keepalives once real output begins
This is ~40 lines of keepalive management that would be unnecessary if codex-relay flushed headers early.
Proposed solution
codex-relay could:
Send HTTP 200 + Content-Type: text/event-stream headers immediately when accepting a streaming request
Optionally send SSE keepalive comments while waiting for the upstream's first token
Environment
codex-relay 0.5.1
GLM-5.2 via Chat Completions (TTFT 5-90s for large context)
Thanks for the report. The streaming path already yields response.created before opening the upstream request, so the relay can flush 200 + text/event-stream without waiting for the provider's response headers. It also had Axum SSE keepalive enabled, but was using Axum's 15-second default interval.
I have tightened the keepalive interval to 5 seconds and made the comment explicit as : keepalive. I also added an end-to-end regression test with a mock upstream that waits 7 seconds before returning its HTTP headers. The test verifies that:
the relay response headers arrive within 1 second
response.created arrives immediately
a keepalive comment arrives at 5 seconds, before the upstream headers
The full test suite passes (168 non-ignored tests; 12 live-provider tests ignored).
Activity
mountain commentedon Jul 11, 2026
Thanks for the report. The streaming path already yields
response.createdbefore opening the upstream request, so the relay can flush200+text/event-streamwithout waiting for the provider's response headers. It also had Axum SSE keepalive enabled, but was using Axum's 15-second default interval.I have tightened the keepalive interval to 5 seconds and made the comment explicit as
: keepalive. I also added an end-to-end regression test with a mock upstream that waits 7 seconds before returning its HTTP headers. The test verifies that:response.createdarrives immediatelyThe full test suite passes (168 non-ignored tests; 12 live-provider tests ignored).
Keep streaming responses alive during upstream TTFT (#41)