Skip to content

fix(memory): bound OOM-prone accumulators across Orca - #10179

Merged
nwparker merged 1 commit into
mainfrom
nwparker/fix-bounded-memory-accumulators
Jul 23, 2026
Merged

nwparker merged 1 commit into
mainfrom
nwparker/fix-bounded-memory-accumulators

Conversation

@nwparker

@nwparker nwparker commented Jul 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #9936: a repository-wide audit found the same unbounded-memory failure class in other long-lived maps, caches, queues, streams, parsers, scans, and fan-outs.

This change adds explicit count, byte, depth, concurrency, and retention ceilings across desktop, renderer/preload, daemon, relay, SSH/WSL, web, mobile, CLI, browser/emulator, filesystem, providers, notifications, speech, native chat, and session discovery.

The pattern is consistent:

  • Validate untrusted sizes before copying, parsing, decoding, or retaining them.
  • Bound aggregate memory as well as individual entries.
  • Release accounting on success, failure, abort, disconnect, unsubscribe, and owner teardown.
  • Evict, truncate, coalesce, retry, resync, or fail closed only after a documented safety ceiling.
  • Preserve ordering and normal behavior below those ceilings.

Final scope: 1,626 files changed, including 686 test files and 303 newly added test files. There are no dependency, lockfile, or native-binary changes.

ELI5

Orca had many containers that were safe during ordinary use but could keep growing forever if a host, repository, provider, terminal, or client supplied enough data.

This change gives those containers labeled maximum sizes. Normal data still goes through unchanged. If a container becomes abnormally full, Orca now drops an old/reconstructible item, pauses or rejects new work, returns a bounded partial result, or asks the caller to retry instead of continuing until the process runs out of memory.

OOM hazards addressed

Area Protection added
Files, responses, and structured input Bounded file/response reads, growth-safe sparse/proc-file handling, JSON depth/token limits, bounded stringify, YAML alias limits, and CSV/IPYNB/media admission
Streams and transport Geometrically growing buffers, fragment caps, frame/body limits, process-wide byte ledgers, queue backpressure, ACK/credit delivery, and fail-closed synchronous send errors
Long-lived state Entry, key-byte, and aggregate-byte caps for maps, sets, caches, listeners, waiters, ownership records, notifications, and status-routing metadata
Queues and fan-out Global/per-owner queue limits, fixed concurrency, retryable overload errors, FIFO draining, coalescing, and cleanup after abort/failure
Enumeration and pagination Page, item, byte, depth, and retained-path budgets for integrations, filesystems, quick open, AI Vault, Linux procfs, repositories, and provider scans
PTY and terminal data Bounded pending input/output/history, delivery credits, remote/relay flow control, terminal WebView queues, and reconstructible-output shedding
Browser and emulator Screenshot/PDF admission, CDP command/event caps, bounded H.264 replay, subscriber/session limits, and one-frame-at-a-time delivery
Persistence and logs Bounded serialized state, migrations, histories, local logs, relay log rotation, cache persistence, and oversized-record rejection
Cross-runtime paths Equivalent admission and cleanup across native, web, mobile, daemon, relay, SSH, and WSL execution

Intentional overload-only behavior

There is no intentional normal-path behavior change for workloads below the new ceilings. These safety degradations can be user-visible only under extreme input or sustained backpressure:

Overload case New behavior
Oversized file, JSON, image, screenshot, PDF, request, or frame Rejected before full allocation/decoding, or returned as a bounded fallback
Very large provider result or filesystem/session scan Stops at the page/item/byte/depth budget and returns a bounded partial result or omission notice
Saturated queue, watcher, subscription, or request pool Rejects with a retryable error, defers, coalesces, or resynchronizes after capacity is released
Frozen/reloading PTY renderer May shed reconstructible live bytes after the pending cap; the pane heals from the retained snapshot and dropped-output sentinel
Stalled emulator video consumer Replay may shrink; an overloaded H.264 stream resumes at the next keyframe
Cache churn Old or least-recently-used values can be evicted and fetched again
Extremely large Linux procfs source Port/process results may be partial rather than materializing the complete source
Old relay without a bounded listing protocol Requires reconnect/update instead of falling back to an unbounded remote enumeration
Oversized notifications, transcript history, logs, or speech input Rejects, truncates, rotates, or omits older data according to the documented subsystem policy
Oversized CSV table Shows a localized safety fallback with the table limits and directs the user to source mode

Screenshots

No normal-path visual change.

The only new visual state is the overload-only oversized-CSV fallback, localized in English, Spanish, Japanese, Korean, and Chinese. No screenshot was captured because it requires a synthetic CSV beyond the safety ceiling.

Evidence of the fix

Representative hard boundaries and regression tests include:

Protection Production boundary Evidence
Integration pagination 100 pages, 10,000 items, 32 MiB retained Exact boundaries, +1 rejection, and no capacity consumption after rejection
Integration fan-out Four concurrent loads using the same item/byte budget Ordering, concurrency, exact/+1 limits, and settled failures
Orchestration waits 1,024 global, 64 per handle, bounded handle/filter bytes Per-handle isolation, global admission, cancellation, and cleanup
Orchestration queries 256 rows, 2 MiB/row, 8 MiB/page; writes capped at 512 KiB and 4,096 items Successive paging, exact counts, oversized legacy-row skipping, DAG correctness, and no starvation
Daemon admission 128 transports, 32 control clients, 24 streams; 256 active requests and 32 MiB process-wide Pre-auth, authenticated raw-protocol, per-client, response, disconnect, and idle-retirement cleanup
File accumulation Caller-defined limit checked before allocation and during growth Initial oversize, mid-read growth, proc-style zero sizes, descriptor cleanup, and no oversized allocation
Fragmented buffers One geometric allocation instead of retained fragment arrays 100,000 one-byte fragments with bounded suffix consumption
Remote runtime 4 MiB outbound JSON, 8 MiB binary frames, 256 subscriptions/requests, aggregate retained-byte limits Exact/+1 admission and release after serialization/encryption failure
WebSocket backpressure Bounded frames/bytes plus fail-closed send handling Immediate and timer-drain send exceptions cannot escape or strand bytes/timers; overflow signals once
Mobile transport 64 MiB/frame, 96 MiB per inbound queue, 64 buffered frames; process-wide inbound/outbound ledgers Multi-socket accounting, per-process isolation, overflow, disposal, and idempotent release
PTY delivery 4,096 retained states, 32 MiB pending characters, 16,384 credit spans Normal ordering, unique-ID floods, exactly-once settlement, renderer teardown, and recovery
Emulator replay 120 GOP frames, 32 MiB/device, 128 MiB total, 16 devices, 8 subscribers Delta shedding, keyframe recovery, global eviction, reentrant cleanup, and live delivery
Filesystem and procfs 100,000 directory entries/12 MiB names; proc tables and metadata capped at 8 MiB Streaming, exact/+1 limits, caller ceilings, and bounded partial scans
AI Vault 100,000 discovery entries, depth 64, 32 MiB paths/list results; bounded JSONL/whole-JSON readers and caches Newest-session preservation, omission reporting, exact UTF-8 limits, oversized-record recovery, FIFO readers, and LRU eviction
Renderer state/cache churn Transient status routing capped at 1,024 keys/1 MiB; repo slug and work-item caches have entry and byte limits Fail-closed clearing, host isolation, LRU behavior, invalidation, aborted waiters, and oversized-value bypass
CSV parsing 50 MiB source/cell text, 250,000 rows, 1,024 columns/row, 1,000,000 cells Every exact boundary, +1 fail-closed behavior, and unchanged output at the limits
Notifications and chat Mobile events 256 KiB, 4 MiB/256 active; native transcript 50,000 messages/64 MiB Exact/+1 limits, newest retention, identifier bounds, and released capacity
Browser capture 32,768 px dimensions, 32 MiB effective pixels/encoded bytes, two concurrent captures/prints Ordinary HiDPI use, exact/+1 bytes, process-wide admission, and failure cleanup

Testing

  • pnpm lint
  • pnpm typecheck
  • pnpm test
  • pnpm build
  • Added or updated high-quality regression tests

Detailed results on final commit 301d5999a5:

  • Root tests: 3,555 files and 36,473 tests passed; 7 files/58 tests skipped.
  • Mobile tests: 340 files and 2,497 tests passed; 2 tests skipped.
  • Final GitHub Actions matrix passed at 301d5999a5: root and mobile verification, Wayland terminal input, Windows terminal restart, packaged crash survival, Linux/macOS golden E2E, and Linux/Windows native-smoke.
  • Root and mobile typechecks passed.
  • Full production desktop/web/native build passed.
  • Full lint chain passed.
  • Mobile lint and formatting passed.
  • Changed-file formatting passed.
  • Localization catalog, parity, and coverage checks passed.
  • Focused WebSocket backpressure run: 16 tests passed.
  • CI build-order reproduction: build:cli → build:electron-vite → verify:cli-runtime passed.
  • Final focused CSV/daemon run: 27 tests passed.
  • git diff --check passed.
  • All 34 rebase conflicts received an independent semantic audit.
  • No dependency or lockfile changes.
  • No new max-lines bypasses.

Local lint note: the primary clone contains unrelated local version tags that confuse the skill snapshot verifier. The full lint chain passed against an origin-only Git tag namespace; clean CI clones should not need that workaround.

AI Review Report

AI-assisted review explicitly checked:

  • Individual and aggregate memory accounting.
  • Exact-boundary and boundary-plus-one behavior.
  • Release after success, throw, rejection, abort, disconnect, renderer destruction, unsubscribe, and reentrant callbacks.
  • Ordering, FIFO draining, stale cleanup, partial completion, double release, and starvation.
  • Per-client, per-renderer, per-host, and process-wide authority isolation.
  • macOS, Linux, Windows, WSL, and SSH paths, including Windows log-handle fallback, Linux procfs behavior, path handling, shell/subprocess behavior, and remote-relay compatibility.
  • GitHub, GitLab, and provider-neutral review/integration paths.
  • Every rebase conflict for lost upstream behavior or lost memory-safety intent.

The independent final review found one medium-severity issue before publication: a synchronous WebSocket send exception could escape a timer callback and strand the remaining queue with stale timer state. The shared queue now catches both immediate and drain-send failures, releases the backlog, clears the timer, and signals reconnect/resync exactly once. Two focused regression tests were added. The first CI run then exposed a cross-build module-boundary regression: Electron output replaced a CLI dependency emitted under out/main. The dependency now lives in shared output, and production plus computer-use workflows permanently boot-check the compiled CLI after Electron builds. No other high- or medium-severity findings remain.

Residual review risk is primarily the breadth of the change. The boundary tests demonstrate bounded state and cleanup, but this PR does not include a repository-wide before/after heap or RSS benchmark. The final GitHub Actions matrix passed on Linux, macOS, Windows, mobile, Wayland, packaged crash-survival, terminal-restart, and native-smoke paths. SSH/WSL and Git 2.25 behavior remain covered by focused tests and compatibility checks rather than live remote-host CI.

Security Audit

The primary security benefit is reducing memory-exhaustion denial-of-service risk from untrusted or unexpectedly large local, remote, IPC, WebSocket, provider, filesystem, and persisted inputs.

Reviewed areas include:

  • Size and structure checks before parsing, base64 decoding, copying, encryption, native capture, or long-term retention.
  • Authenticated daemon admission plus pre-auth connection limits.
  • Sender/renderer/client ownership, stale-release protection, and cleanup on disconnect.
  • Filesystem roots, path-byte/depth limits, symlink-sensitive scans, and bounded remote listings.
  • Subprocess output bounds and argument-array handling on native, SSH, and WSL paths.
  • Credential/token size limits, endpoint redaction, private log permissions, and avoidance of secret-bearing overload output.
  • IPC schema whitelisting so unknown fields are not retained.
  • No new third-party dependencies, lockfile changes, or bundled native binaries.

No unresolved input-handling, command-execution, path, authentication, secret, dependency, or IPC vulnerability was identified by the completed reviews. Multi-platform CI remains the final confirmation before merge.

Notes

  • Normal workloads below the documented limits are intended to remain behaviorally unchanged.
  • Overload-only truncation, eviction, retry, reconnect, resync, and fallback behavior is documented above.
  • Contributor: @nparker2450

@nwparker
nwparker force-pushed the nwparker/fix-bounded-memory-accumulators branch from be3240a to 4d038c2 Compare July 23, 2026 12:50
@nwparker
nwparker force-pushed the nwparker/fix-bounded-memory-accumulators branch from 4d038c2 to 301d599 Compare July 23, 2026 13:04
@nwparker
nwparker merged commit 8f40ddf into main Jul 23, 2026
12 checks passed
brennanb2025 added a commit that referenced this pull request Jul 23, 2026
…ory file limits

Semantic resolutions where #10179 bounded the same reads this PR made
asynchronous:
- add async variants of the bounded terminal-history readers and use them
  for checkpoint.json, output.log, and legacy scrollback.bin so the reads
  are both off-thread and size/JSON-structure capped
- keep the header-only stat bypass ahead of the replay semaphore; the
  bounded log read now rejects oversized logs inside the slot
- daemon-pty-adapter keeps the shared ColdRestorePayload type from
  cold-restore-payload-cache alongside main's PtyProcessListAdmission
- await the now-async detectColdRestore in main's history-reader-memory
  tests
- extract the legacy scrollback restore path into
  terminal-history-legacy-scrollback-restore.ts to stay within max-lines
5hseok added a commit to 5hseok/orca that referenced this pull request Jul 24, 2026
Resolve conflicts from stablyai#10179 (bound OOM-prone accumulators), which moved
the parse-cache map into session-parse-cache-retention.ts and rewrote the
list caches around boundAiVaultListResult:

- session-parse-cache: re-export invalidateSessionParseCacheEntry from the
  retention module (new home of the cache map) instead of deleting from a
  now-removed local map.
- discovery: keep upstream's bounded openclaw scan (retainNewestFile) while
  calling the shared openClawAgentsRootDir / isOpenClawSessionFilePath helpers
  the deletion validator also consumes.
- cached-session-list & ipc/ai-vault: keep the cacheGeneration guard on top of
  upstream's boundAiVaultListResult so a delete landing mid-scan still can't
  repopulate the cache with a pre-delete result.
- ipc/ai-vault imports: drop the inline-subagent-handler imports upstream kept;
  this branch serves that handler from ai-vault-subagent-list.ts, which calls
  the same (now memory-bounded) scanner.
nwparker added a commit that referenced this pull request Jul 24, 2026
This reverts commit 8f40ddf (#10179).

Rebased on current main. The revert applied cleanly across the vast majority
of the 1,626-file PR; the remaining work was preserving PRs that had been built
on top of #10179 after it merged. All are preserved:

- #9990  perf(terminal): cold-restore replay
- #10213 fix(codex): TUI settings promotion
- #10135 fix(mobile): images in native chat
- #9387  fix(status-bar): Resource Manager badge + concurrent listSessions
- #9853  fix(app): bound wake/quit paths (phone-session-ended freeze)
- #10237 fix(claude-accounts): quote claude path for Windows (kept shell:false)

Because those PRs were written against helper modules/APIs that #10179
introduced, the minimal, typecheck-verified closure they depend on is kept so
the tree builds and every later-PR test passes:

- bounded file/JSON readers used by #10213/#9990 (agent-state-file-reader,
  node-bounded-file-reader, json-text-structure-limit, terminal-history limits)
- claimed-agent-pty-owner cap API + pty:listSessions admission (#9990/#9387)
- the mobile-E2EE outbound-memory-budget subsystem #9853 builds on
  (e2ee-channel, mobile-e2ee-*, ws-outbound-backpressure-queue,
  remote-runtime-request-frames/-memory-limits, map-with-concurrency, e2ee-crypto)

Everything else #10179 added (~555 files) is deleted; every file it modified
elsewhere is restored to its pre-#10179 state.

Verified: root typecheck (node+cli+web), mobile typecheck, full root test suite,
full mobile test suite, oxlint, localization catalog + coverage, reliability
gates, max-lines ratchet, and the CLI emit build all pass. Two checks fail only
locally and fail identically on clean origin/main (this clone's git-tag
namespace for verify:skill-bundle-manifest; an inherited GIT_CONFIG_* env for
one relay test) — both pass on a clean CI environment.

Co-authored-by: Orca <help@stably.ai>
nwparker added a commit that referenced this pull request Jul 24, 2026
5hseok added a commit to 5hseok/orca that referenced this pull request Jul 24, 2026
Upstream reverted "bound OOM-prone accumulators (stablyai#10179)" (stablyai#10255), which
deleted the parse-cache/list retention modules my earlier merge had adapted to.
Re-resolve back to the pre-stablyai#10179 structure while keeping this branch's work:

- session-scanner-parse-cache: the cache Map is local again; move
  invalidateSessionParseCacheEntry back here (from the deleted retention module)
  as a plain cache.delete, keeping ai-vault-delete's import path unchanged.
- session-scanner-discovery: upstream's openclaw scan restored, but calling the
  shared openClawAgentsRootDir / isOpenClawSessionFilePath helpers the deletion
  validator also consumes.
- cached-session-list & ipc/ai-vault: keep the cacheGeneration guard on top of
  upstream's inline (unbounded) scan + direct-result caching, so a delete
  landing mid-scan still can't repopulate the cache with a pre-delete result.
- ipc/ai-vault imports: keep serving the subagent handler from
  ai-vault-subagent-list.ts; use upstream's mergeAiVaultListResults.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant