[nwparker]

OOM hardening slice 17/29 — bound native chat transcript & speech input

Part of a 29-PR stack re-landing #10179 ("bound OOM-prone accumulators across Orca"), which was reverted in #10255 for being too large (~1,600 files). This is slice 17/29 (base: oom/16-b-codex). Merge in numeric order. The full stack's end-state is byte-for-byte the commit that passed #10179's complete CI matrix (36k+ tests, multi-OS).

Summary

Re-introduces OOM ceilings across native-chat transcript reading (per-record/page/retention byte caps, message-count cap, and a process-wide concurrency+byte admission gate for transcript reads) and speech (cloud-audio sample cap, credential/response/stderr size caps, and bounded model-cache migration + directory scans). The most significant user-visible shift is that native-chat transcript reads are no longer "entire file, no cap" — they now return the newest bounded window with pagination.

ELI5

Imagine a chat log that could grow to any size. Before, the app tried to load the whole thing into memory at once, which could crash it on huge logs. Now it loads the newest chunk (still very large) and lets you scroll back for older parts, and it only lets a couple of these big loads happen at the same time so memory stays safe. The speech-to-text side gets similar seatbelts: caps on how much audio, error text, and model-folder scanning it will hold. Ordinary users won't notice; only enormous logs or extreme conditions hit the limits.

Proof of OOM fix

• Transcript retention: MAX_NATIVE_CHAT_TRANSCRIPT_MESSAGES=50,000 and MAX_NATIVE_CHAT_TRANSCRIPT_RETAINED_BYTES=64 MiB (retained-byte estimate = sourceBytes*2+256) in transcript-message-retention.ts; tested in transcript-message-retention.test.ts.
• Per-record cap: MAX_NATIVE_CHAT_TRANSCRIPT_RECORD_BYTES=2 MiB via TranscriptRecordBuffer; tested in transcript-record-buffer.test.ts and transcript-stream-lines.test.ts (oversized record dropped, next line preserved, UTF-8 split across 1-byte chunks intact).
• Per-page cap: MAX_NATIVE_CHAT_TRANSCRIPT_PAGE_RETAINED_BYTES=16 MiB in transcript-tail-reader.ts (sets pageBudgetReached, forces hasMore=true); tested in transcript-reader.test.ts.
• Incremental drain: APPEND_BATCH_RETAINED_BYTE_LIMIT=8 MiB, INCREMENTAL_DRAIN_RETAINED_BYTE_LIMIT=32 MiB in transcript-incremental-reader.ts; tested in transcript-incremental-reader.test.ts (byte-flush before count limit, pause/resume at record boundary without gaps, always advances one record even when budget < record).
• Read admission: MAX_...READ_CONCURRENCY=8, MAX...READ_WAITERS=256, MAX..._PROCESS_RETAINED_BYTES=128 MiB; each watch drain reserves PAGE(16 MiB)+DRAIN(32 MiB)=48 MiB (transcript-read-admission.ts); tested in transcript-read-admission.test.ts and transcript-watch-drain-admission.test.ts (concurrency+byte budget, aborted/unsubscribed waiter removed and slot reused, idempotent release, listener detach).
• Speech cloud audio: MAX_CLOUD_AUDIO_SAMPLES=16000*600 (10 min) with a single growing Float32Array in openai-transcription-client.ts; tested in openai-transcription-client.test.ts (100k one-sample feeds keep one buffer).
• Speech model-cache migration: 16,384 entries / depth 16 / 16 MiB visited-path bytes in model-cache-path.ts; tested in model-cache-path-bounds.test.ts.
• Speech directory scan: SPEECH_MODEL_ARCHIVE_SCAN_MAX_ENTRIES=32,768, retained-name 4 MiB, SPEECH_MODEL_VOCAB_SCAN_MAX_ENTRIES=4,096 in speech-model-directory-scanner.ts; tested in speech-model-directory-scanner.test.ts.
• tar stderr: SPEECH_MODEL_EXTRACTION_STDERR_MAX_RETAINED_BYTES=64 KiB (4 KiB prefix + ring tail, 500 chars/end evidence) in speech-model-extraction-stderr.ts; tested in speech-model-extraction-stderr.test.ts.
• Credential/response caps: openai-api-key-store.ts uses readIntegrationCredentialFileSync (rejects >1 MiB key file, tested in openai-api-key-store.test.ts); openai-transcription-client.ts uses readFetchResponseJsonWithinLimit for the provider response body.

Regressions

Normal-path (ordinary use, below the new limits): none — behavior is unchanged below the ceilings.

Overload-only (extreme input / sustained backpressure) — intended, documented degradations:

• Native-chat transcript reads truncate to the newest 50,000 messages / 64 MiB retained (~32 MB source JSONL); older history reachable only via pagination (beforeOffset).
• A single per-page tail read caps at 16 MiB retained and returns hasMore=true; UI must page for the rest.
• Records >2 MiB are dropped (surfaced as a fallback-id placeholder message).
• Admission throws 'Too many queued native chat transcript reads' only when >256 transcript reads are simultaneously queued process-wide.
• Speech model-cache migration aborts best-effort (logs a warning, leaves a partial copy) only past 16,384 entries / depth 16 / 16 MiB path bytes.
• BPE vocab discovery scans at most 4,096 entries; a .vocab beyond that falls back to greedy_search (loses hotwords).
• Cloud dictation still hard-caps at 10 minutes of audio (unchanged limit, now sample-based).
• tar extraction error messages bounded to 4 KiB prefix + 64 KiB tail with an 'N stderr bytes omitted' marker.
• readNativeChatTranscript changed from 'entire transcript, NO message cap' to a bounded newest-window (50,000 messages / 64 MiB retained). For very heavy sessions, oldest messages are no longer in the canonical parse and are only reachable via pagination (the read-cache doc comment explicitly dropped 'canonical, unwindowed'). Confirm all surfaces page correctly. (only when: Opening a native-chat (Claude/Codex/Grok) session whose transcript exceeds ~32 MB source JSONL or 50k messages; reachable in ordinary heavy coding sessions, which the code comments describe as 'routinely tens of MB'.)
• Live transcript watcher drains now pass through a shared 128 MiB / 8-concurrency admission gate, each reserving a fixed 48 MiB, so at most ~2 watch drains run concurrently process-wide (shared with page reads from other call sites). With many native-chat views open, live updates serialize behind each other, adding refresh latency. (only when: Several native-chat sessions actively updating at once (multiple worktrees / paired web/mobile clients). Correctness preserved (FIFO admit, no loss); only latency increases.)
• readNativeChatTranscript / the read-cache now return a bounded newest window (50,000 messages / 64 MiB retained, ~32 MB source JSONL) instead of the whole file. Below those ceilings the parse is byte-for-byte identical to BASE; above them the oldest history is dropped from the canonical parse and only reachable via pagination (beforeOffset). This is the intended OOM fix, but it IS user-visible for very heavy sessions and warrants explicit human signoff. (only when: Opening a native-chat session whose transcript exceeds ~32 MB / 50k messages; the code comments describe heavy Claude/Codex sessions as 'routinely tens of MB'.)
• A tail page can return hasMore=true with fewer than limit messages once the 16 MiB per-page retained-byte budget is hit, requiring the UI to page for the rest. Only reachable when records approach the 2 MiB per-record cap (tens of near-max records); ordinary KB-sized messages keep tens of thousands within 16 MiB. (only when: Paging a transcript composed of many very large (near-2 MiB) records.)

Build / CI

• Part of the Main services + CLI group; this group's cumulative tree is interlinked, so it becomes typecheck-green at its checkpoint PR Add dismiss button to update notifications #25 (whole-repo interconnection — see stack README). Content is exact production code.

Adversarial review

• 2 skeptic lens(es). Sign-off: FLAGGED — see above. Residual risk: low.
• Adversarial regression pass on PR I attached a github issue with this worktree, but I have no way of seeing it #17 (native-chat transcript + speech). I tried hard to refute 'ordinary users see no change' and could not confirm a normal-path regression.\n\nKEY REFUTATION OF THE PRIOR REVIEWER'S MEDIUM FINDING: the claimed 'readNativeChatTranscript went from entire-file to bounded window, older msgs only via pagination' is largely moot. grep -rn shows readNativeChatTranscript / readNativeChatTranscriptCached / decodeTranscriptStream have no wired user-facing caller. All live surfaces use readNativeChatTranscriptTail(File) which ALREADY did newest-first message-count windowing with hasMore/beforeOffset before this PR (src/main/ipc/native-chat.ts default window, MAX 2000; RPC default 40, MAX 2000). The renderer already lazy-pages ('starts at the default window and raises limit to page in older history as it scrolls up'). So the paginated window is pre-existing behavior, not introduced here.\n\nWHAT THIS PR ACTUALLY ADDS on the user path: a per-page byte ceiling (MAX_NATIVE_CHAT_TRANSCRIPT_PAGE_RETAINED_BYTES = 16 MiB retained, ~8 MB source) on the tail reader, and the 48 MiB-reservation admission gate on watch drains. The page cap only makes a heavy page smaller and sets hasMore=true; the existing pagination UI continues loading. transcript-reader.test.ts verifies [...older, ...newest] reconstructs the full ordered list, and that a single record can never single-handedly trip the page budget (records are hard-capped at 2 MiB source => ~4 MiB retained, well under the 16 MiB page budget => >=4 records/page, no starvation).\n\nCEILINGS VERIFIED (all far above ordinary use): retention window 50,000 msgs / 64 MiB retained (estimate = sourceBytes2+256, ~32 MB source); page cap 16 MiB retained; admission process budget 128 MiB / concurrency 8 / 256 waiters (throws only >256 simultaneous queued reads); cloud dictation 10 min unchanged (now sample-based, MAX_CLOUD_AUDIO_SAMPLES = 16000600); model-cache migration 16,384 entries / depth 16 / 16 MiB path bytes (a model has a handful of files); BPE vocab scan 4,096 entries; tar stderr 64 KiB (4 KiB prefix + 60 KiB tail); key file 1 MiB.\n\nCORRECTNESS CHECKS (no defect found): incremental drain-budget break resets state.offset to pendingStart and clears pendingRecord so the next drain re-reads that record with no gap; the drainRetainedBytes>0 guard forces one oversized record to advance instead of stalling (both verified by transcript-incremental-reader.test.ts). Admission abort removes queued waiter + detaches listener + re-admits; release is idempotent (transcript-read-admission.test.ts). OfflineAudioChunker findQuietSplitIndex returns Math.max(1, bestIndex) so push() cannot infinite-loop; 100k one-sample test reconstructs exactly. No deadlock in the admission gate since every reservation (16/48 MiB) <= 128 MiB budget.\n\nOnly residual user-visible effect is the low-severity multi-session watch-drain serialization latency noted in confirmedRegressions. Safe to open as a draft PR for human review.

Files

32 files changed (+1703 / −166).

---

🤖 Decomposed, reviewed & assembled by Claude Code from the reverted #10179. Draft — do not merge out of order.

Made with Orca 🐋

[nwparker]

Superseded by #10548, which re-scopes this work to the bound-at-ingress doctrine (see #10533 and src/shared/memory-safety/README.md).

An audit found the original #10179 scope was ~50x the size of the fix for the crash it targeted, and that bounds had landed ahead of their callers. Every file in these slices was re-adjudicated individually: ingress bounds (transport frames, PTY flow control, on-disk transcripts, parsing, enumeration, media decode, user-keyed accumulators) were kept; interior ceilings behind an already-bounded input were dropped. 1,882 files → 1,155, full suite green, and no main feature regressed.

No memory protection was lost — only speculation.