[nwparker]

OOM hardening slice 13/29 — bound daemon admission & PTY delivery credits

Part of a 29-PR stack re-landing #10179 ("bound OOM-prone accumulators across Orca"), which was reverted in #10255 for being too large (~1,600 files). This is slice 13/29 (base: oom/12-b-ssh). Merge in numeric order. The full stack's end-state is byte-for-byte the commit that passed #10179's complete CI matrix (36k+ tests, multi-OS).

Summary

This slice re-lands OOM ceilings across the terminal daemon and PTY layer: it caps daemon transport sockets/clients/requests/response bytes, bounds NDJSON/binary-frame line and payload sizes, admits and byte-limits every daemon request field, streams overlay-tree removal and config-overlay mirroring under entry/depth caps, and bounds terminal-history logs, pre-ready stdin, pending output, shell-startup rc reads, and terminal size (500x500).

ELI5

Imagine the app's terminal engine as a mailroom that used to accept letters of any size, any number, from anyone, and would pile them up until the building collapsed. This change adds sensible rules: letters can't be huge, only so many can be in flight, envelopes must be addressed correctly, and cleanup crews stop after a reasonable amount of work instead of running forever. Almost nobody in normal use ever bumps these limits; they only kick in when something is broken or hostile.

Proof of OOM fix

• daemon-admission-limits.ts: sockets=128, control clients=32, stream attachments=24, handshake timeout=10s, active requests 256 (128/client), active request bytes 32MB (16MB/client), control buffer 32MB; per-field caps: clientId/requestId=1KB, type=256B, sessionId=4KB, cwd=256KB, command=4MB, historySeed=12MB, env 4096 entries/4MB with 1MB/value. Tests: daemon-admission-limits.test.ts, daemon-server-admission.test.ts, daemon-client-outbound-admission.test.ts
• daemon-response-admission.ts: response=16MB, active response 128MB (64MB/client), control process buffer 64MB; reservation only for createOrAttach/getSnapshot/listSessions/takePendingOutput. Test: daemon-server-admission.test.ts
• session.ts: PENDING_OUTPUT_MAX_RECORDS=4096, PRE_READY_STDIN 16MB code-units / 4096 segments. Tests: session-pending-output.test.ts, session-pre-ready-input-memory.test.ts, session.test.ts
• terminal-history-file-limits (external): LOG=5MB, checkpoint=16MB, meta=64KB, enforced via encodeLogBatchWithinLimit/stringifyJsonWithinByteLimit in history-manager.ts. Tests: history-manager-memory.test.ts, terminal-history-log.test.ts
• ndjson.ts: line=16MB, handshake line=64KB, structural tokens=1,000,000, nesting depth=128. Test: ndjson.test.ts
• binary-frame.ts: decode now rejects payloadLength > FRAME_MAX_PAYLOAD (1MB) and clears the buffer. Test: binary-frame.test.ts
• overlay-mirror.ts safeRemoveTree: OVERLAY_REMOVE_MAX_ENTRIES=100,000, OVERLAY_REMOVE_MAX_DEPTH=256, streamed via opendirSync (no unbounded readdir array / recursion). Test: overlay-mirror.test.ts
• config-overlay-mirroring.ts: 4096 source entries, 4KB/entry-name, 1MB retained-name bytes. Test: config-overlay-mirroring.test.ts
• shell-startup-env.ts: rc file read cap=1MB, LRU cache 128 entries/8MB, env value cap=64KB, name<=256, path<=16KB. Tests: shell-startup-env-memory.test.ts, shell-startup-env.test.ts
• daemon-control-file-reader.ts=64KB and daemon-login-session-probe-verdict.ts=64B bounded reads replace unbounded readFileSync across daemon-spawner/health/init/host-relocation/entry/client
• daemon-host-relocation.ts: MAX_PINNED_DAEMON_VERSIONS=1024 with fail-closed prune (skips pruning if pin scan saturates), streamed opendirSync

Regressions

Normal-path (ordinary use, below the new limits): none — behavior is unchanged below the ceilings.

Overload-only (extreme input / sustained backpressure) — intended, documented degradations:

• Daemon rejects a control client past 32, a stream attachment past 24, or a transport socket past 128 (retryable hello error); far above 1-few ordinary app instances
• Daemon replies 'Daemon request capacity exceeded; retry' once >256 in-flight requests / 32MB request bytes / 128MB reserved response bytes accumulate (or per-client 128 / 16MB / 64MB)
• A single response serializing over 16MB (e.g. an enormous getSnapshot/takePendingOutput) is converted to an error reply instead of being sent
• A peer that never sends a newline, sends a >16MB request line (>64KB during handshake), or a binary frame claiming >1MB payload gets its socket destroyed
• overlay-tree removal past 100k entries or 256 depth is left partially removed for a later launch instead of walking unbounded
• config overlay mirroring throws ConfigOverlayCapacityError once a source/plugins dir exceeds 4096 entries or 1MB of retained names
• Pre-ready stdin queue past 16MB or 4096 segments throws before shell readiness; pending-output past 4096 non-coalesced records drops to overflow snapshot
• Client that fails hello within 10s handshake timeout is disconnected
• Terminal size is now hard-capped at 500 cols x 500 rows (shared terminal-size-limits). A resize request beyond that is rejected outright by daemon admission (writeRequestError), and headless-emulator normalizes out-of-range dims to the 80x24 fallback instead of honoring them. Rows=500 is unreachable, but cols~500 is conceivably reachable on an ultra-wide monitor with a very small font. (only when: Ultra-wide display + tiny font pushing a terminal past 500 columns; ordinary terminals (80-300 cols) are unaffected.)
• Agent config-overlay mirroring (used by mimo/opencode hook services) fails the spawn with ConfigOverlayCapacityError if a user's config/plugins source directory exceeds 4096 entries or 1MB of retained entry-name bytes. (only when: Power users with an unusually large agent config or plugins directory (>4096 files); typical config dirs are well under the cap.)
• Shell-startup env probing now reads at most 1MB of an rc file and ignores env-var values longer than 64KB, so an assignment living past the first 1MB of a very large rc file (or a >64KB value) is no longer picked up. (only when: User rc file (.zshrc/.bashrc/.profile) larger than 1MB or a targeted env var whose value exceeds 64KB; normal rc files are far smaller.)
• A single Orca instance can fail to restore or create its fifth concurrently mounting terminal. createOrAttach reserves 16 MiB against a 64 MiB per-client response budget, so requests 1-4 are admitted and request 5 receives "Daemon request capacity exceeded; retry"; the adapter propagates that error instead of retrying. Five split/restored panes is an ordinary workspace size, and the code explicitly documents simultaneous pane mounts. (only when: Five createOrAttach requests from one app overlap before any of the first four finishes, such as concurrent split-pane mounts or startup restore while PTY preparation is pending.)

Build / CI

• Part of the Main services + CLI group; this group's cumulative tree is interlinked, so it becomes typecheck-green at its checkpoint PR Add dismiss button to update notifications #25 (whole-repo interconnection — see stack README). Content is exact production code.
• Touches 1 file(s) also changed on main since the revert; git auto-merges them (no conflict): daemon-server.ts.

✅ Inherited defect FIXED in this slice

• ✅ FIXED [medium, overload-path only] src/main/daemon/daemon-response-admission.ts — Each create reserves the full 16 MiB response max against a 64 MiB per-client budget, so 4 pending terminal creates can exhaust it and reject the 5th even though real replies are tiny; DaemonPtyAdapter retries daemon-loss but not the capacity error.
  • Fix applied: Added a bounded, pre-dispatch-safe retry (6 attempts, ~25-250ms backoff) in DaemonClient.request keyed to the 'request capacity exceeded' rejection; a >=5-pane restore fan-out now waits for an in-flight response to release its 16 MiB reservation instead of failing. Verified: typecheck + daemon client/admission suites pass.

Defects inherited from the original #10179 code (not introduced by slicing; not present in today's unbounded main). Fixed items were patched in-slice and re-verified (typecheck + affected suites); remaining flagged items are overload-only and noted for a fast-follow.

Adversarial review

• 3 skeptic lens(es) incl. a frontier-model (GPT) cross-check. Sign-off: FLAGGED — see above. Residual risk: medium.
• Adversarial (skeptic) pass on PR Add Orca logo and polish README header #13 B-daemon-pty. No CONFIRMED normal-path regression. The prior reviewer's summary is accurate; I verified actual numeric ceilings and reachability. Two items warrant human sign-off (residualRisk=medium, not signoff-blocking since this is production-shipped code that passed full CI).

CONCERN 1 - Terminal size cap 500x500, HARD-rejected with no clamp (src/shared/terminal-size-limits.ts, wired via daemon-admission-limits.ts + daemon-pty-size.ts). Confirmed the renderer does NOT import terminal-size-limits (git grep over src/renderer is empty), so it does not pre-clamp. createOrAttach with cols>500 -> createOrAttachPayloadError -> spawn rejected outright (NOT clamped to fallback). resize runs terminalSizeAdmissionError with NO allowMissing, so cols>500 -> writeRequestError, PTY keeps old size (renderer/PTY mismatch). Reachability: a maximized full-width terminal on a 49-inch super-ultrawide (5120px) at a ~10px cell font computes 500-512 cols - a real if uncommon dev config, not purely abusive. Rows=500 unreachable; cols500 is the edge. Normal panes/displays (80-300 cols) unaffected. Low-medium; human should confirm Orca clamps fitted dims to 500 in renderer/adapter (out of this slice).

CONCERN 2 - Response-reservation model (daemon-response-admission.ts + daemon-server.ts dispatchRequest). createOrAttach/getSnapshot/listSessions/takePendingOutput each reserve a FIXED 16MB regardless of actual size; per-client cap 64MB => only 4 concurrent per app instance (test daemon-server-admission.test.ts:217 confirms 64MB/16MB=4; 5th throws 'request capacity exceeded'); global 128MB => 8. Reservation held for full preparePtySpawn + host.createOrAttach duration, released in .finally(). If the renderer fires 5+ createOrAttach concurrently during multi-terminal restore, requests 5+ get 'Daemon request capacity exceeded; retry'. Multi-terminal restore is a normal power-user scenario. Could NOT confirm it triggers: restore orchestration/retry lives in the renderer/adapter (daemon-pty-adapter.ts is OUT of this slice and unchanged), and the 'retry' error text plus full-CI shipping strongly imply restore serializes or retries. Human should confirm restore path serializes createOrAttach or handles 'capacity exceeded; retry'.

VERIFIED OVERLOAD-ONLY (ceilings far above ordinary use): pending-output record cap 4096 (session.ts) dominated by pre-existing 2MB byte cap and only matters while detached; pre-ready stdin 16MB/4096 segments (throw is caught by write RPC handler, daemon survives); history log cap unchanged at 5MB, checkpoint 16MB (JsonStringifyByteLimitError caught in checkpoint try/catch); shell-startup rc 1MB read + 64KB env-value cap; config-overlay mirroring 4096 entries/1MB names only affects mimo/opencode/relay agent hook services (callers confirmed: src/main/mimo, src/main/opencode, src/relay only), never ordinary spawns; overlay-tree removal 100k entries/256 depth; daemon 128 sockets/32 control clients/24 stream attachments; NDJSON 16MB line/128 depth/1M tokens; 10s handshake timeout.

CLEANUP/CORRECTNESS: No leak. dispatchRequest increments then decrements in .finally() (runs on throw/cancel); admission failures return BEFORE incrementing. writeRequestError early-returns for null/notify ids. DAEMON_MAX_STREAM_ATTACHMENTS(24) < DAEMON_MAX_CONTROL_CLIENTS(32) asymmetry is intentional, only reachable with >24 concurrent app instances.

OOM EVIDENCE: strong. Concrete caps in daemon-admission-limits.ts, daemon-response-admission.ts, terminal-history-file-limits.ts, shell-startup-env.ts, config-overlay-mirroring.ts, overlay-mirror.ts, plus regression tests daemon-server-admission.test.ts, history-manager-memory.test.ts, session-pending-output.test.ts, session-pre-ready-input-memory.test.ts, headless-emulator-size-limits.test.ts, shell-startup-env-memory.test.ts, config-overlay-mirroring.test.ts, overlay-mirror.test.ts.

• Correctness notes: /Users/nwparker/orca/workspaces/orca/oom-followup/src/main/daemon/daemon-response-admission.ts: The large-response reservation is too coarse for an ordinary restore burst: every createOrAttach reserves the full 16 MiB response maximum, while the per-client reservation ceiling is 64 MiB. Consequently, four pending terminal creates consume the entire budget and the fifth is rejected even though normal create replies are tiny. The existing admission test proves this exact rejection, and DaemonPtyAdapter only retries daemon-loss errors, not the returned capacity error. Request counters, byte reservations, pending spawn preparations, sockets, timers, and directory handles otherwise release on the reviewed success/error/disconnect paths; exact-limit tests did not reveal an inclusive/exclusive error.

Files

48 files changed (+2716 / −358).

---

🤖 Decomposed, reviewed & assembled by Claude Code from the reverted #10179. Draft — do not merge out of order.

[nwparker]

Superseded by #10548, which re-scopes this work to the bound-at-ingress doctrine (see #10533 and src/shared/memory-safety/README.md).

An audit found the original #10179 scope was ~50x the size of the fix for the crash it targeted, and that bounds had landed ahead of their callers. Every file in these slices was re-adjudicated individually: ingress bounds (transport frames, PTY flow control, on-disk transcripts, parsing, enumeration, media decode, user-keyed accumulators) were kept; interior ceilings behind an already-bounded input were dropped. 1,882 files → 1,155, full suite green, and no main feature regressed.

No memory protection was lost — only speculation.