Repository navigation
fix(proxy): bound the events mirror and take it off the synchronous path - #56
Conversation
The mirror was one appendFileSync per emitted event, before any listener ran, with no cap. A real 3 h Codex session wrote 103.5 MB (91% chunk lines), and the recorded median chunk benchmarked at a worst single append of 74 ms on the main process. With ~15 sessions live the proxy directory grew by gigabytes a day. EventsMirror writes through one ordered stream, drops and counts lines past a 16 MiB queue (marking the gap in the file), and rotates to <name>.1.jsonl at eventsFileMaxBytes (default 64 MiB), so a run holds at most two files and always keeps its newest traffic. Rotation, not Claude's omit-past-budget, because on Codex the chunks are the bulk and a debug bundle tails the newest 5 MiB. stop() flushes and closes the mirror. Refs Juliusolsson05/agent-code#372 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tream accounting Review A/B round 1: - A1: a rotated-out stream's error zeroed the shared in-flight counter; the new stream's callbacks drove it negative and flush()/stop() hung. Writes are now accounted per stream, and only the current file failing turns the mirror off. - A2: the queue bound was per stream, so a rotation doubled it. It is now on the total unwritten bytes across every open stream. - A3/A4/B2: the per-file cap leaked through a pending async open (restart at the cap, startup burst), uncounted markers, and oversized lines. The file is opened synchronously (one syscall per file) and sized from that fd; a write is marker plus line and must fit; anything that cannot fit even a fresh file is dropped and counted. - B1: drops at the end of a run are marked on close. - B3/B4: tests pin the 64 MiB default, close-not-flush on stop, the marker's rotatedBytes, and that a failed mirror stays off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…count against the queue - C1: append() rotated first and only then found the line could not fit even a fresh file, so rejected lines evicted accepted events (a one-line cap left only markers). The rotation marker is deterministic, so the fresh file's first write is measured before anything is renamed, and a line that fits no file is dropped without rotating. - A2: the queue check now includes the drop or rotation marker's bytes. - A1/A3: tests pin per-stream accounting (the failure is delivered while the live write is certainly pending) and the marker in the cap check. - The separate oversized-line queue guard was redundant with the aggregate check and is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review disposition (final head
|
…ker; a light 64 MiB default test - A/C round 3: close() wrote its drop marker onto a possibly full queue (a marker-only write has no line bytes to check), so one marker could sit over the queue bound. close() now drains first; on an empty queue the marker is the only thing queued. - The 64 MiB default test wrote ~70 MB of real events and timed out under load. It now pre-sizes the file sparsely to just under the default, as a restart finds it, and checks that one recorded event rotates it and one event less does not. Fixed after the final verification pass (step-7 cap): not re-verified by the reviewers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
close() drains before its drop marker, but a marker-only append skipped the queue check (bytes === 0), so a configured cap smaller than one marker line was still exceeded. Every write, markers included, now fits the queue or is not written; a line is counted as dropped, a marker that cannot fit leaves the drops in stats(). Pinned with a 1-byte cap; the bytes>0 exception mutant is killed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Final addendum (manager). Round 3 was the last review pass. Reviewers A and C each found one minor issue: the close-time drop marker could exceed |
Refs Juliusolsson05/agent-code#372. App side: Juliusolsson05/agent-code#1332 (pointer bump plus the shared, rotation-safe debug-bundle reader).
Plan: agent-code
docs/plans/2026-09-26-codex-proxy-mirror-bound.md.Problem
ResponsesProxy.emitranJSON.stringifyand thenappendFileSyncfor everyevent, before the listeners, with no cap and no drop policy.response-chunklines were 91.3% of it.proxy/was at 2.3 GB, and nothing rotated.Change
src/proxy/eventsMirror.ts,EventsMirror:fs.WriteStreamper file;emitonly serialises and queues.openSyncand sized withfstatSyncon that fd, so a stream is never pending and a rotation never waits for an open. Earlier, deferring the rotation let a startup burst or a restart at the cap push the file past its cap (review A3).eventsFileMaxBytes(default 64 MiB). The rotation marker is deterministic, so fit is measured first, and a line that fits no file never rotates away kept events. When it would not fit, the file is renamed to<name>.1.jsonland a fresh file starts with{kind:'mirror-rotated', rotatedBytes, rotations, droppedEvents, droppedBytes}. A line that cannot fit even in a fresh file is dropped and counted. A run holds at most two files.mirror-droppedmarker precedes the next written line, or is written at close.flush()orstop(). Only the current file failing turns the mirror off, and it stays off. Theerrorlistener means a failure never crashes the main process.stop()closes the mirror after the server teardown.flushMirror()is added. API.md is updated.Tests
responsesProxy.mirrorBound.test.ts(18 tests), on the recorded Codex 0.157 chunk (testing/fixtures/proxy-mirror/models-chunk-2865.json):emitdoes not write on the emitting call;rotatedBytesequals.1's size;flush()andclose(), and gives back only its own share of pending writes;stop()closes (a later event is not written).responsesProxy.mirror.test.tsis ported toawait flushMirror(), with its assertions unchanged.Review
7575ae3.3e7e35e.ceefb9c, and steering q62's tiny-cap edge in6b8ca32. Both came after the final pass and are not re-verified.The dispositions are in the PR comment.
Verification
tsc --noEmitis clean.src/proxy: 59 tests.tsc --noEmitis clean. The mutant setting the default to 65 MiB is killed. The full package suite is run in the background at each head;src/transcript/*tests time out only under heavy machine load. Those files are untouched here, and they pass on CI.Residual
/v1/models(about 523 KB, every ~6.5 min). Rotation bounds it, but it shortens the kept window.🤖 Generated with Claude Code