Repository navigation
Description
Status (quality-loop audit, 2026-09-25)
Landed: #729 made retention a single scan.
Remaining: codex-headless responsesProxy.ts:408-432 still mirrors every Responses event synchronously (appendFileSync + base64, no cap), and it is on for every Codex session.
Acceptance: Chunks are no longer mirrored synchronously by default; proxy, feed-debug and perf logs have per-run byte caps and dropped counters. Evidence: a real 100–280 MiB proxy-events.jsonl.
Original report
Problem
The heap-crash investigation found a separate pressure amplifier from #369: packages/codex-headless/src/proxy/responsesProxy.ts mirrors every proxy event synchronously, and response-chunk events serialize raw Buffers through JSON/base64 before normal listeners run. This creates large transient strings, blocks the event loop, and writes unbounded active proxy-events.jsonl files.
Local evidence from the investigation found proxy storage near 965 MiB, with individual proxy-events.jsonl files over 100-280 MiB. Feed-debug and performance logging have similar per-file cap gaps.
Findings
ResponsesProxy.emitmirrors beforesuper.emit, so slow/string-heavy mirror writes delay the stream adapter.response-chunkmirroring can hold raw bytes, base64 text, JSON text, and append buffers at once.- Request capture buffers and parses full bodies for shape extraction and may base64 encode request bodies.
- Retention eventually prunes disk, but active writers have no per-run byte cap or drop mode.
- Feed-debug renderer memory is count-capped, but disk files are not per-file byte-capped.
- Retention sweeps are cleanup, not pressure control, and can themselves be expensive during pressure.
Proposed Fix
- Stop raw/base64 mirroring of every
response-chunkby default. Store metadata plus bounded samples/tails; gate full raw capture behind an explicit forensic flag. - Add per-run/per-file byte caps and dropped counters for proxy mirrors, feed-debug, and performance logs.
- Move mirror writes off the hot synchronous path into a bounded async queue/drop policy.
- Add lightweight counters: chunk bytes, serialized bytes, mirror write ms, cap drops, queue depth, per-run bytes.
- Make retention cheaper and visible: one scan per prune, protected bytes, active-grace skips, removals, and scan duration.
Acceptance
- Large Responses streams no longer synchronously base64/JSON mirror every chunk by default.
- Active proxy/feed/perf log files have hard caps and record dropped counters.
- A stream stress test demonstrates bounded proxy mirror memory/disk behavior.
- Debug/retention metrics make it clear when diagnostics are being dropped because of caps.
Activity
Juliusolsson05 commentedon Sep 25, 2026
Status after #1317 (merged via batch #1318, 2026-09-25).
Shipped: codex-headless#53. The proxy mirror now writes response chunks as base64 (
{ _buffer_b64 }), as documented, instead of decimal byte arrays:JSON.stringifyranBuffer#toJSONbefore the replacer. That was 3.65× the payload, 2,266 MiB of chunk lines for 621 MiB in one recorded 3.19 GB session. The same file's chunk lines drop to about 830 MiB, with identical bytes.Still open: per-file byte caps with dropped counters, request-body mirroring (775 MiB in that file), and the synchronous write.
fix(debug): bundle the rotated Codex mirror's tail after a rotation; …
fix(debug): one rotation-safe proxy-events reader for Codex and Claude
fix(debug): #1332 review — find a run mid-rotation, retry a split ope…
test(debug): pin the exhausted-retry header; bump codex-headless to #56…
chore(codex): bump codex-headless to the codex-headless#56 merge (f10…
Juliusolsson05 commentedon Sep 27, 2026
Merged in #1332. Its three defaults (rotate, 64 MiB / 16 MiB, no /v1/models dedup) are pending the owner's confirmation; see #1332's comment.