Repository navigation
[Bug]: Codex subagent child threads and stuck-active root threads are never unsubscribed, so their MCP servers live until the app-server exits #21354
Description
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
Related, not duplicates: #15979 (closed, fixed by #16917) covered idle root/delegated threads. This report is what is still left after that fix on a build that includes it.
Observed on a long-lived shared codex app-server (macOS, T3 Code Nightly 0.0.46-nightly.20261014.3118, codex-cli 0.163.0, several stdio MCP servers configured, Codex threads that spawn their own subagents, plus a few long-running root threads):
- Run the above for about a day.
- Count the Codex thread files the app-server holds open (
lsof -p <app-server pid>, rollout.jsonlfiles) and the app-server's descendant processes. - Compare with T3's own record of those threads (
statev2.sqlite) and with thethread/unsubscribecalls in T3's provider log.
Model-free check of the Codex side (no model calls): start codex app-server with a stub stdio MCP server, thread/start, then thread/unsubscribe. The stub exits only after Codex emits thread/closed, about 30 minutes later with the default delay. Skip the unsubscribe and the stub lives as long as the app-server does. So the cleanup has to come from T3, as #16917 found.
Expected behavior
- Codex-native subagent child threads (
source: subagent) are unsubscribed on the same idle rule as root threads, so their MCP servers stop. - A root thread that T3 marks
activebut that has produced no events for a long time eventually times out or is reconciled, so it does not hold its MCP servers forever.
Actual behavior
After about 24 hours, one app-server held 100 loaded Codex threads, 393 descendant processes and about 1.5 GB. T3's database knew 98 of the 100:
| Threads | Count | T3 status | Idle for |
|---|---|---|---|
Codex subagent children (source: subagent) |
97 (95 found in T3) | idle |
5.7 h to 24 h |
| T3 root threads | 3 | active, no pending background work |
16 h to 19 h |
- Root threads that went idle follow the fix(server): unload Codex threads left idle on the shared app-server #16917 path correctly:
thread/unsubscribe, thenthread/closedabout 30 minutes later, and their MCP servers are gone. T3's retained log shows 76 unsubscribe calls; none is for any of the 100 open threads. The log is rotated and partial, so this is evidence, not proof. - From reading
ProviderSessionManager.tsonmain(not instrumented): the idle-unload timer is keyed by the attached T3 thread (idleThreadUnloads, set inholdThreadLoaded) and tracks one provider thread per T3 thread. A Codex-native child thread is never attached as a T3 thread, so no timer exists for it. - Likewise from source reading:
unloadIdleThreadskips a thread with a busy turn and defers whilehasPendingBackgroundWorkForThreadis true, so a thread that T3 believes is running is never unloaded. That may explain the 3activeroots; I did not confirm why they areactive. - 32 MCP sets (one
node_repleach) were alive, started 5.7 h to 21.8 h ago and none younger. Three start in the same second as the three root threads' files; eleven start in the same second as a subagent thread file; fourteen match no thread-file creation time (a thread resumed later would not). Thread count and set count are not one-to-one, so this shows who holds some sets, not all.
Not proven: which of the 100 threads owns each remaining set, and whether the 3 active roots are truly stuck or are long turns that T3 mis-tracks. No thread was opened or changed to gather this.
Impact
Major degradation or frequent failure
Version or commit
Desktop app bundle 0.0.46-nightly.20261014.3118 (includes #16917 / eba0521). Source references are main as read on 2026-10-18.
Environment
macOS, T3 Code Nightly desktop, Codex app-server codex-cli 0.163.0, shared Codex provider session.
Logs or stack traces
No raw logs attached. Counts above come from lsof on the app-server, ps for descendants, a read-only read of statev2.sqlite, and T3's provider log.
Workaround
Restarting the T3 server (and so the codex app-server) releases everything. In a stub-server check, archiving a thread freed its MCP servers at once.
Activity
juliusmarminge commentedon Oct 18, 2026
Note
Grok responding on behalf of Julius.
Accepting the subagent half as a bug. On main, the Codex adapter's
unloadThreadsendsthread/unsubscribeonly for the root native thread, and the idle unload timer inProviderSessionManageris keyed by attached T3 threads, so Codex-native subagent children are never unsubscribed. Open PR #20048 makesunloadThreadunsubscribe the whole subagent subtree, which covers this half.The three roots stuck
activeare a separate question. Unloading is deliberately skipped while a turn is busy, and #20048 notes that a wedged turn whose interrupt times out stays active natively. The report doesn't show why those turns never completed, so we can't act on that part yet. If it happens again, these would help:codex-interrupt-timeout.1kuna commentedon Oct 24, 2026
Title: Additional retention measurement and owner-aware disposal visibility for delegated MCP stacks
Problem: A passive census found 20 MCP/helper stacks with 275 descendants and 13.50 GiB summed RSS under the T3-owned Codex app-server. Of 19 time-matched stacks, 10 aligned with completed delegated tasks. The inspected projection did not expose a definitive process-to-provider-thread ownership map. These are retained stacks, not proven orphaned processes: a final worker answer can intentionally leave background work running, and a reusable provider thread may remain loaded after a turn ends. The absence of ownership and disposal status prevents distinguishing legitimate retention from missed unloads.
Reproduction or observation: Use a stub stdio MCP server and a delegated task with no background work. Record server births, child completion, subscription/unsubscribe events,
thread/closed, and descendant exit. Repeat for a provider-native child and for a child with intentionally live background work. Observe T3's state projection separately from provider terminal evidence. The audit itself used process-tree and lifecycle-time comparisons; it did not run this controlled test or terminate processes.Proposed behavior: Expose the owning task/provider thread, MCP server identity, process tree, last actual tool call, in-flight work, subscription state, retention reason, and pending/completed disposal state. On task completion, release idle task-owned resources through the provider's supported unsubscribe/unload lifecycle. Keep resources needed by live commands, requests, or background work until that work finishes, then release them. Reconcile terminal provider events with T3's projected state so a stale Running flag cannot retain resources indefinitely. Disposing a session must preserve durable transcript/resume state.
Why native: T3 knows delegated-task ownership and Codex owns its transports; they can coordinate safe unloads. Process-name/age-based cleanup cannot tell an idle server from a live background job, and polling agents cannot supply the missing ownership map.
braden808 commentedon Oct 27, 2026
Additional macOS incident on T3 Code Beta 0.1.2 / Codex 0.164.0, October 26, 2026.
A retained 21:37 MDT process snapshot has 782 direct children of the T3-owned Codex app-server, including 774 helpers: 259
cua_node, 259node_repl, and 256cua-driverMCP processes. Their summed RSS was 4,474.8 MiB (summed RSS, not unique physical memory). The app-server had been running since approximately 18:10 MDT. A subsequent read verified the T3 → Node launcher → Codex app-server parent chain; no processes were terminated to collect this evidence.The host operator reported approximately 30 GB swap, Data-volume free space declining from 111 GB to 61 GB, and load above 200. Those are the operator's incident measurements, not new measurements by this source investigation. The raw snapshot contains unrelated private command arguments and is retained locally rather than attached.
Current
main(f6a9315d19284422e2f2718e487c16acda4ba24e) still has root-onlythread/unsubscribeinCodexAdapterV2.unloadThread; its registered native subagents are not included. This matches the accepted native-subagent source defect here. I found the existing fix in #20048 and am preparing that fix locally rather than opening a duplicate implementation PR.The process census does not establish ownership of each helper or prove that every turn leaks. The separate documented T3/Codex idle delays, live background work, and roots still marked active need to remain distinct. The fix must preserve active work, independently loaded children, shared app-server lifetime, and durable resume history. This incident does not establish actual macOS teardown of the proposed fix; that still needs verification on an isolated runtime before deployment.