[bug] Renderer V8 OOM (exitCode 5) on opening a workspace with a large un-gitignored dir (e.g. node_modules) — Files View eagerly stats the entire tree #5320

Summary

Switching to a workspace tab whose project directory contains a large directory that is not gitignored (in my case nested node_modules) reliably crashes the renderer process with a V8 JavaScript heap OOM (exitCode 5). The main process then restarts the renderer, it reloads the same tab, and crashes again — an effectively permanent crash loop for that workspace. The app is fine when idle; the crash only fires on activating the offending tab.

Environment

• Superset 1.12.5 (stable channel)
• Electron 40.8.5
• macOS, Apple Silicon
• System memory was ~80% free with zero swap at crash time, so this is a per-renderer V8 heap limit, not OS memory pressure.

Symptoms

~/Library/Logs/Superset/main.log:

[error] [main-window] Renderer process gone { reason: 'crashed', exitCode: 5 }

No Crashpad dump is written (consistent with a JS-heap OOM rather than a native fault).

Root cause (captured)

Launching the binary with --enable-logging=stderr --v=1 captured the real fatal error in the renderer:

<--- Last few GCs --->
... Scavenge (during sweeping) 3965.6 (4016.0) -> 3962.3 (4023.5) MB ... allocation failure;
... Incremental Mark-Compact (reduce) 3969.5 (4024.3) -> 3966.3 (4001.8) MB ...
ERROR:third_party/blink/renderer/bindings/core/v8/v8_initializer.cc:920] V8 javascript OOM (Reached heap limit).

The renderer JS heap climbs to the ~4 GB V8 ceiling and dies. I observed RSS climbing to 5.5–10 GB with CPU pegged at 700–780% (multiple cores) in the seconds before the crash.

The tRPC traffic immediately before the runaway (also from stderr) shows the trigger:

trpc/workspace.get?input={... id: <workspace> ...}
trpc/filesystem.statPath?batch=1     <-- repeated in a storm
... heap climbs to 4 GB ... OOM

On activating the workspace, the Files View / filesystem layer eagerly stats and materializes the entire directory tree. In my case the tree was:

186,614 total entries
164,000 of them inside node_modules (3 nested frontend packages, each with its own node_modules)
 22,679 entries excluding node_modules

node_modules was not in that repo's .gitignore, so it was enumerated. Materializing ~186k file nodes in the renderer exhausts the heap.

Reproduction

1. Open a workspace whose directory contains a large directory that is not gitignored (e.g. a repo/monorepo where node_modules/ is present but not in .gitignore, ~150k+ files).
2. Switch to that workspace's tab.
3. Renderer heap climbs to ~4 GB and crashes (exitCode 5); the app restarts it and it re-crashes on reload → loop.

Workaround (confirmed)

Adding node_modules/ to the project's .gitignore:

• git status untracked count: ~164,000 → 7
• tree the renderer walks: 186k → ~22k entries
• crash stops; renderer stays bounded (~450–500 MB) when opening the same tab.

So the Files View does honor .gitignore for enumeration. The bug is that it eagerly enumerates everything that isn't ignored, with no lazy-loading or entry cap — so any large non-ignored directory (very common: node_modules is often present without being gitignored, plus build output, data dirs, caches, etc.) OOMs the renderer.

Suggested fixes

• Lazy-load the Files View tree (enumerate only expanded directories) instead of eagerly walking the whole tree on workspace activation.
• Default-exclude well-known heavy directories (node_modules, .git, dist, build, .next, target, venv, .cache, …) from enumeration/watching even when not gitignored.
• Cap the number of entries / nodes held in memory, with virtualization or a "show more" fallback.
• Guard the renderer against unbounded filesystem.statPath fan-out.

Related: #4293 (gray out gitignored files in Files View — implies ignored content is enumerated), #4868 (host-service V8 ~4 GB OOM, sibling process).