[nwparker]

OOM hardening slice 09/29 — bound git/worktree/persistence/repo/usage state

Part of a 29-PR stack re-landing #10179 ("bound OOM-prone accumulators across Orca"), which was reverted in #10255 for being too large (~1,600 files). This is slice 09/29 (base: oom/08-b-mobile-rest). Merge in numeric order. The full stack's end-state is byte-for-byte the commit that passed #10179's complete CI matrix (36k+ tests, multi-OS).

Summary

This slice re-introduces ~40 independent count/byte/depth/concurrency ceilings across git worktree persistence, terminal-history GC, crash reporting, cloud-profile session stores, usage-history scanning, and process/persistence state files. The behaviorally significant change is a new 10MB default output cap on ALL local git commands (previously unbounded unless a caller opted in); everything else sits far above ordinary use.

ELI5

Imagine dozens of buckets in the app that could previously fill up forever and eventually crash it by running out of memory. This change puts a lid on each bucket at a size much bigger than anyone normally uses. The one lid you can actually bump into during normal work is on git: if a single git command spits out more than 10MB of text (a giant diff or log), it now stops with an error instead of risking a crash.

Proof of OOM fix

• git/runner.ts: DEFAULT_GIT_MAX_BUFFER = 10MB now applied to every git exec (was only enforced when a caller passed maxBuffer); tested in runner-command-exec.test.ts
• persistence.ts: state file > ORCA_PERSISTED_STATE_MAX_BYTES (64MB) -> load defaults + writesFrozen=true; persistence-state-bounds.test.ts
• attribution/terminal-attribution.ts: ATTRIBUTION_COMMIT_MESSAGE_MAX_BYTES=1MB, ATTRIBUTION_COMMAND_OUTPUT_MAX_BYTES=1MB, WRITTEN_ROOT_MAX_ENTRIES=64 (evicted via while size>max), SHIM_VERSION_MAX_BYTES=1KB; terminal-attribution-bounds/retention.test.ts
• git/status.ts: MAX_SUBMODULE_PATHS_PER_REPO=10_000, MAX_SUBMODULE_PATH_CODE_UNITS=64KB, MAX_SUBMODULE_PATHS_CACHE_CODE_UNITS=16MB, MAX_EFFECTIVE_UPSTREAM_CACHE_KEY_BYTES=64KB, MAX_GIT_POINTER_FILE_BYTES=64KB; status-submodule/upstream/pointer-bounds tests
• worktree-orphan-gitdir-proof.ts / local-worktree-filesystem.ts / worktree-removal-safety.ts: MAX_WORKTREE_GIT_POINTER_BYTES=64KB, oversized .git pointer -> null
• terminal-history-gc.ts: MAX_DISCOVERY_ENTRIES=100_000, MAX_WSL_DISTROS=256, MAX_FILES_PER_WORKTREE=64, META_MAX_BYTES=64KB; capacity error is caught, remaining scanned next run
• crash-report-store.ts: MAX_CRASH_REPORT_FILE_BYTES=4MB, JSON_STRUCTURAL_TOKENS=1M, JSON_NESTING_DEPTH=128; crash-report-store.test.ts
• orca-profiles: MAX_ORCA_CLOUD_MEMORY_SESSIONS=128, MAX_ORCA_CLOUD_MEMORY_SESSION_BYTES=16MB, MAX_CLOUD_SESSION_TOMBSTONES=512 (FIFO shift()), DEV_ORG_ROSTER_MAX_ENTRIES=64, DEV_ORG_PENDING_INVITE_MAX_ENTRIES=256, MAX_ORCA_PROFILE_INDEX_FILE_BYTES=1MB
• usage-history: MAX_FILES=200_000, MAX_DISCOVERY_ENTRIES=1M, MAX_RECORDS=100_000, MAX_OWNERSHIP_KEYS=100_000, MAX_RETAINED_BYTES=256MB, MAX_JSONL_LINE_BYTES=4MB, MAX_USAGE_PROJECTION_STATE_FILE_BYTES=64MB
• git/git-capability-state.ts: LOCAL_GIT_CAPABILITY_HOST_MAX_ENTRIES=64 (LRU-ish while size>max), HOST_KEY_MAX_BYTES=4KB
• repo enrichment: REPO_IDENTITY_NEGATIVE_CACHE_MAX_ENTRIES=512, REPO_GIT_USERNAME_ATTEMPT_MAX_ENTRIES=512 (both evict via while size>max)
• diagnostics/main-thread-churn-probe.ts: SUBPROCESS_SPAWN_STATS_MAX_ENTRIES=128 (127 real keys + 1 overflow bucket), diagnostics-gated only
• startup markers: X_DISPLAY_LOCK_MAX_BYTES=64, MAX_GPU_FALLBACK_MARKER_FILE_BYTES=8KB, MAX_SHELL_PATH_STDOUT_BYTES=1MB, WINDOWS_ACL_GRANT_MARKER_MAX_BYTES=8KB
• misc file readers: MAX_GIT_MARKER_FILE_BYTES=64KB, GENERATED_NODE_MANAGED_FILE_MAX_BYTES=64KB, MAX_ACTIVE_VIEW_PREFERENCE_FILE_BYTES=4KB, MAX_REPO_ICON_SOURCE_BYTES=256KB, MAX_BUNDLE_SOURCE_FILE_BYTES=50MB, local-downloaded-folder-promotion depth/byte budget

Regressions

Normal-path (ordinary use, below the new limits): none — behavior is unchanged below the ceilings.

Overload-only (extreme input / sustained backpressure) — intended, documented degradations:

• persistence.ts freezes ALL state writes and loads defaults only when the on-disk state file exceeds 64MB (ordinary state is KB-few MB) — extreme/corruption only
• Worktree .git pointer treated as invalid (null) only when the pointer file exceeds 64KB; real pointers are a single 'gitdir:' line
• Submodule enumeration stops at 10,000 submodules / 64KB per path / 16MB cache — normal repos have a handful
• terminal-history GC throws a caught capacity error at 100k discovery entries / 256 WSL distros and simply resumes next run; no data loss
• Attribution wrappers refuse commit-message / gh-capture files over 1MB (exit code 74) — normal commit messages are tiny
• Cloud memory session eviction at 128 sessions / 16MB and tombstone FIFO at 512 — far above interactive use
• usage-history scan caps (200k files, 1M dir entries, 256MB retained) only reached by pathological ~/.orca trees
• Diagnostics spawn-stats map caps at 128 keys and only when main-thread diagnostics are explicitly enabled
• git/runner.ts now applies a 10MB DEFAULT_GIT_MAX_BUFFER to every local git command that does not pass its own maxBuffer. Previously the cap was only enforced when a caller explicitly set maxBuffer (if (options.maxBuffer && ...)), so unbounded output was allowed. Git commands whose combined stdout/stderr exceed 10MB now fail with 'stdout exceeded maxBuffer.' instead of returning the output. This is reachable on large monorepos (huge git diff, long git log, git status with very many untracked files). (only when: Any git operation not passing its own maxBuffer that produces >10MB of output on a large repo (native, WSL, or SSH). Intended anti-OOM cap and kept in sync with relay MAX_GIT_BUFFER, but a human should confirm 10MB is comfortably above real large-repo command output.)
• persistence.ts: when the state file exceeds 64MB the app loads default state AND sets writesFrozen=true, so the session silently stops persisting (windows/tabs/repos not saved) to avoid clobbering recoverable bytes. Correct by design, but the user gets a defaults-looking session with no persistence until the oversized file is resolved. (only when: Corrupted or pathologically large (>64MB) persisted state file on startup; not reachable in ordinary use.)
• spawnCommandCapture now applies the 10MB DEFAULT_GIT_MAX_BUFFER when no explicit maxBuffer is passed (BASE used if (options.maxBuffer && ...), leaving this path unbounded). CORRECTION to the prior reviewer: this is NOT a change to 'every local git command'. The mainline execFileCapture path ALREADY applied options.maxBuffer ?? DEFAULT_GIT_MAX_BUFFER (10MB) in BASE (runner.ts:442, constant at :253). spawnCommandCapture is reached ONLY for Windows batch-script shims (isWindowsBatchScript at runner.ts:879 and the Windows .cmd retry at :896) — e.g. gh.cmd/glab.cmd. So the newly-capped surface is Windows-only + batch-shim-only. On macOS/Linux and the normal execFile git path, output was already 10MB-capped in BASE; no change. (only when: On Windows only, a git/gh/glab command that resolves to a batch-script shim, does not pass its own maxBuffer, and emits >10MB combined stdout/stderr. Now fails with 'stdout/stderr exceeded maxBuffer' instead of returning unbounded output.)

Build / CI

• Part of the Main services + CLI group; this group's cumulative tree is interlinked, so it becomes typecheck-green at its checkpoint PR Add dismiss button to update notifications #25 (whole-repo interconnection — see stack README). Content is exact production code.
• Touches 1 file(s) also changed on main since the revert; git auto-merges them (no conflict): workspace-space-analysis.ts.

Adversarial review

• 2 skeptic lens(es). Sign-off: FLAGGED — see above. Residual risk: low.
• HEADLINE CLAIM REFUTED (in the direction of LESS risk, so signoff stands). The prior reviewer's medium-severity "new 10MB default cap on ALL local git commands" is OVERSTATED and largely inaccurate relative to THIS PR's diff.

Evidence (git -C ... diff aab1129 6eb70d8 -- src/main/git/runner.ts):

• DEFAULT_GIT_MAX_BUFFER (1010241024) is NOT introduced by this slice. It already exists in BASE (post-revert) at src/main/git/runner.ts:253, and BASE already applies it to the two PRIMARY paths:
  • execFileCapture at line 442: maxBuffer: options.maxBuffer ?? DEFAULT_GIT_MAX_BUFFER (present in BASE, UNCHANGED by the diff). This is the path used by repo-scoped runGit (huge git diff/git log/git status).
  • gitStreamStdout at line 957: const maxBuffer = options.maxBuffer ?? DEFAULT_GIT_MAX_BUFFER (present in BASE; the diff only swaps the accumulation mechanism to GrowingByteBuffer, not the cap).
• The ONLY place this diff newly applies the default is spawnCommandCapture (line 498: const maxBuffer = options.maxBuffer ?? DEFAULT_GIT_MAX_BUFFER), replacing BASE's if (options.maxBuffer && stdoutBytes > options.maxBuffer). spawnCommandCapture is reached ONLY from commandExecFileAsync for (a) Windows batch scripts (isWindowsBatchScript) and (b) the Windows command-shim retry (shouldRetryWindowsCommandShim). Normal git.exe invocations and the reviewer's cited "large diff/log/status" cases go through execFileCapture, which was ALREADY capped in BASE. So the primary and streaming git paths are behaviorally UNCHANGED by this PR; no new normal-path git-output regression.

Correctness of the GrowingByteBuffer swap: bounds check stdout.byteLength + chunk.byteLength > maxBuffer rejects the overflowing chunk before appending — equivalent boundary to BASE (no off-by-one). On overflow it calls killSpawnedCommandTree(child) then finish() (same child-cleanup semantics as BASE). finish() clears both GrowingByteBuffers, releasing memory. Decoding moved from per-chunk chunk.toString(encoding) (BASE, which could corrupt multibyte UTF-8 split across chunks) to a single decode-at-end via toString() — this is a CORRECTNESS IMPROVEMENT, not a loss. spawnCommandCapture only returns strings, so the 'buffer' encoding concern does not apply.

Other caps spot-checked and confirmed far above ordinary use (overload/corruption-only):

• persistence.ts writesFrozen on oversized state file (ordinary state is KB-few MB; MAX_AGENT_STATE_FILE_BYTES=4MB is the adjacent agent-state cap).
• status.ts: git pointer 64KB (real pointers are one gitdir: line), submodules 10,000 / 64KB per path / 16MB cache.
• terminal-history-gc.ts: 100,000 discovery entries / 256 WSL distros / 64KB meta; MAX_FILES_PER_WORKTREE=64 is a per-run GC throughput bound (cleanup resumes next run, no data loss/user impact).
• attribution: commit-message 1MB, gh-capture output 1MB, exit 74 — normal commit messages/captures are tiny.
• filesystem-provider-bounded-text.ts is a new helper whose limits are PARAMETERIZED (caller-supplied maxBytes/maxCodeUnits); no hardcoded low default in this slice (caller wiring lives in another slice).

Residual: the Windows batch-script / command-shim path now caps arbitrary command output at 10MB where BASE was unbounded — theoretically a >10MB batch/shim command would now fail with 'stdout exceeded maxBuffer'. This is a narrow, non-git, Windows-only path and matches the anti-OOM intent kept in sync with relay MAX_GIT_BUFFER; not a normal-path regression. Signoff=true.

Files

109 files changed (+4987 / −923).

---

🤖 Decomposed, reviewed & assembled by Claude Code from the reverted #10179. Draft — do not merge out of order.

Made with Orca 🐋

[nwparker]

Superseded by #10548, which re-scopes this work to the bound-at-ingress doctrine (see #10533 and src/shared/memory-safety/README.md).

An audit found the original #10179 scope was ~50x the size of the fix for the crash it targeted, and that bounds had landed ahead of their callers. Every file in these slices was re-adjudicated individually: ingress bounds (transport frames, PTY flow control, on-disk transcripts, parsing, enumeration, media decode, user-keyed accumulators) were kept; interior ceilings behind an already-bounded input were dropped. 1,882 files → 1,155, full suite green, and no main feature regressed.

No memory protection was lost — only speculation.