Repository navigation
Fourth unbounded git ls-files site: removeStaleWorkspaceFiles overflows on node_modules with a bare ENOBUFS (#323 missed it) #691
Description
Summary
This is a fourth call site of the class #323 closed. #323 enumerated three unbounded git ls-files
sites (:850, :919, :3584) and was closed COMPLETED against the criterion "all three sites bound
their output explicitly". removeStaleWorkspaceFiles was not in that list, and it is the most exposed of
the four because it is the only one that passes --ignored — which deliberately surfaces node_modules.
removeStaleWorkspaceFiles in the generated workflow enumerates untracked and ignored paths with
execFileSync and no maxBuffer. Any isolated worktree that has run a package install therefore has
node_modules in the listing, the output exceeds Node's 1 MiB default, and the node dies with:
SystemError: spawnSync git ENOBUFS (stdout or stderr buffer reached maxBuffer size limit)
Measured on a real worktree:
git ls-files --others --ignored --exclude-standard -z
-> 1,716,025 bytes across 30,682 paths
30,681 of those paths are under node_modules/
Node default maxBuffer = 1,048,576
The failure is deterministic once dependencies are installed, and it happens in ~2 s during the node's
pre-agent phase, so no agent work is even attempted.
The code
for (const args of [
["ls-files", "--others", "--exclude-standard", "-z"],
["ls-files", "--others", "--ignored", "--exclude-standard", "-z"]
]) {
for (const entry of execFileSync("git", args, { cwd: workspaceRoot, encoding: "utf8" }).split("\0")) {
if (entry) candidates.add(entry);
}
}
for (const relativePath of candidates) {
if (expected.has(relativePath) || isWorkspaceRuntimePath(relativePath)) continue;
...
}with
function isWorkspaceRuntimePath(relativePath: string): boolean {
const root = relativePath.split("/")[0];
return [".ultrafuzz", ".smithers", "node_modules", "artifacts"].includes(root);
}So the loop enumerates ~30k node_modules entries only to discard every one of them on the very next
statement. The overflow is incurred entirely for paths the function has already decided to ignore.
Why this matters beyond the crash
The bare SystemError names no subcommand, no size, no path and no cause. #311 established that an
oversized git capture must arrive as an actionable sentence, and both the handoff diff capture and
invariantSuiteGitPaths honour that. This call site does not, so the same class of failure regressed to
the pre-#311 experience: I had to trace every git invocation with a shim to find which call overran.
It is also worth noting the interaction with [retry] same_agent_attempts = 1: the node had already
succeeded once before dependencies were installed, and every retry afterwards failed here, so a node
whose real work was complete became permanently unrunnable.
Fix applied locally, offered as the shape of the real one
Excluding the four runtime roots in the pathspec is behaviour-identical to the isWorkspaceRuntimePath
skip, strictly faster, and removes the overflow:
const staleExclusions = [".ultrafuzz", ".smithers", "node_modules", "artifacts"].map(
(root) => `:(exclude)${root}/**`
);
for (const args of [
["ls-files", "--others", "--exclude-standard", "-z", "--", ".", ...staleExclusions],
["ls-files", "--others", "--ignored", "--exclude-standard", "-z", "--", ".", ...staleExclusions]
]) {
for (const entry of execFileSync("git", args, {
cwd: workspaceRoot,
encoding: "utf8",
maxBuffer: 64 * 1024 * 1024
}).split("\0")) {Verified: the node that failed twice in ~2 s then ran to completion.
Suggested acceptance criteria
- The stale-file enumeration excludes the runtime roots in the pathspec rather than filtering them
after the fact. - Every
execFileSync/spawnSyncgit call in the generated workflow states an explicitmaxBuffer
— enumerated exhaustively this time, rather than by the three paths Three git ls-files call sites still use Node's 1 MB default maxBuffer #323 happened to list. - No git capture can surface as a bare
SystemError: wrap ENOBUFS with the subcommand, the bound and
the cwd, per fix(runtime): name the largest paths when a git capture overflows its buffer #311. - Regression test: a worktree containing a large ignored directory (e.g. a populated
node_modules)
completes stale-file cleanup without error.
Activity
Discover git capture sites whitespace-tolerantly in the #691 exhausti…
Bound removeStaleWorkspaceFiles git enumeration and exclude runtime r…
Discover git capture sites whitespace-tolerantly in the #691 exhausti…
Bound removeStaleWorkspaceFiles git enumeration and exclude runtime r…
Reconcile #727 with the merged #691 scan: reword the prose that quote…
Bounded wait and stale-lock recovery in restoreWorkspacePatchPreparat…