Define a graceful drain policy for in-flight turns during shutdown

[0disoft]

Context

Current dev handles process shutdown in src/cli.ts:

• SIGINT / SIGTERM call a shutdown handler;
• the handler calls server.stop(true), removes the pid file, restores native Codex when not running as OCX_SERVICE, then exits.

The request paths already have good client-cancel handling in several places (abortSignal, upstream aborts, WebSocket close aborts, etc.). This issue is specifically about the process shutdown path, not normal client disconnects.

Concern

When the proxy is stopped while a long streaming /v1/responses or WebSocket turn is active, it is not obvious what contract users should expect:

• should opencodex wait briefly for active turns to finish?
• should it reject new requests but let existing streams drain up to a deadline?
• should it actively abort in-flight upstream requests and emit/record an incomplete/aborted terminal outcome?
• should ocx stop behave differently from Ctrl+C or service stop?

Right now the code appears to stop and exit immediately from the CLI shutdown handler. That is simple, but for Codex users a mid-turn stop can look like a random reconnect/stall rather than an intentional proxy shutdown.

Possible approach

A small, bounded policy might be enough:

1. Track active HTTP/WebSocket turns in the server.
2. On shutdown, stop accepting new work.
3. Wait up to a short configurable deadline, e.g. 5-10 seconds, for active streams to finish.
4. After the deadline, abort remaining upstream controllers and record/log a terminal shutdown/incomplete outcome where possible.
5. Keep ocx stop --force or service-manager hard kill behavior available for cases where the process must exit immediately.

This does not need to be a big reliability framework; the main value is making shutdown behavior explicit and preventing in-flight turns from being cut off without any diagnostic signal.

Question

Do you think opencodex should support a bounded graceful drain on shutdown, or is immediate shutdown the intended behavior? If you agree with adding a small drain policy, would you prefer to implement it yourself, or should I prepare a focused PR after the current dev merge settles?

[lidge-jun]

Thanks for the thorough write-up. I agree this is worth addressing — here's where things stand on dev and my take on a bounded approach.

Current shutdown behavior

The shutdown path (src/cli.ts:203-214) is immediate:

const shutdown = () => {
  server.stop(true);   // Bun's immediate close
  removePid();
  if (!process.env.OCX_SERVICE) restoreNativeCodex();
  process.exit(0);
};
process.on("SIGINT", shutdown);
process.on("SIGTERM", shutdown);

There's no tracking of in-flight turns at the server level. Individual request paths do handle client-side disconnects well — bridge.ts has cancel() → clientCancelled → upstream abort, and ws-bridge.ts cancels the reader on WebSocket close — but those are reactive (client goes away), not proactive (server initiating shutdown while streams are live).

What I think makes sense

Your "possible approach" is close to what I'd want. A few specifics:

1. Track active turns — a Set<AbortController> (or similar) at the server level, incremented on /v1/responses entry and decremented on stream close/abort. Lightweight, no per-request overhead beyond set add/delete.

2. Shutdown sequence:

• SIGINT/SIGTERM → set a "draining" flag → reject new /v1/responses with 503 + Retry-After: 5
• Wait up to shutdownTimeoutMs (default ~5s, configurable in config.json) for the active set to empty
• After deadline: abort remaining upstream controllers via the tracked set, log which turns were interrupted
• Then proceed with current cleanup (pid, restore, exit)

3. ocx stop --force — skip the drain, same as today's behavior. Service manager SIGKILL after the configured timeout also acts as a hard kill.

4. Diagnostic signal — when a turn is aborted by shutdown, emit a response.failed SSE event with last_error: { type: "server_shutdown", message: "..." } before closing the stream, so Codex sees a clean terminal event instead of a dropped connection.

What I'd skip

• No persistent queue, no turn replay, no checkpoint/resume. This is a proxy, not a message broker.
• No separate drain behavior for WebSocket vs HTTP/SSE — both use the same timeout.
• No ocx stop vs Ctrl+C distinction — both should drain. Only --force or SIGKILL should skip.

Sizing

This is maybe 80-120 lines of change across server.ts (active set + reject-when-draining) and cli.ts (timed wait loop). The bridge already emits response.failed so we'd reuse that path.

If you want to prepare a PR for this, go for it — happy to review. The scope is small enough that it shouldn't conflict with other work. Main thing to watch: Bun.serve().stop() behavior with closeActiveConnections: true vs false and how it interacts with a drain loop.

[lidge-jun]

Implemented in 1fb4fdb on dev:

• Active turn tracking: Set<AbortController> tracks all in-flight HTTP streams and WebSocket pumps at stream-lifetime granularity (not handleResponses return)
• Graceful drain: SIGINT/SIGTERM//api/stop → set draining flag → reject new /v1/responses with 503 + Retry-After: 5 → wait up to shutdownTimeoutMs (default 5s, configurable in config.json) → abort remaining turns → clean exit
• trackStreamLifetime() wraps Response bodies to register/unregister turns on stream close/cancel/error
• WebSocket turns register at pump start, unregister in finally
• New shutdownTimeoutMs config field (optional, default 5000)
• Tests: 4 new tests in tests/shutdown-drain.test.ts

Full suite: 360/360 pass, tsc clean.