Generated-corpus exclusion is an exact-name list, so an agent-chosen variant like recon-corpus-deep blows the capture buffer again

What happened

R53 (aave-v4-v0012-main-issue357fix-r53-invariant-only, image from 2d47e73) failed
stateful-invariant-handlers after ~15 minutes with:

git diff produced more than the 33554432-byte capture buffer (a handed-off patch must also stay under
16777216 bytes); the workspace is too large to hand off. Largest contributors within the first 33902592
bytes git wrote; git emits in path order, so anything past that cutoff is not visible here:
recon-corpus-deep (>=33865139 diff bytes in 1 file), echidna-deep (>=37453 diff bytes in 15 files)

Node ran 22:14:47Z -> 22:29:41Z, retry_count 0. The run went to recovery generation 1 at 22:30:58Z.

Why this is a NEW defect and not a regression of #305

#305 fixed exactly this class by excluding generated corpus directories from the workspace capture:

const WORKSPACE_GENERATED_ROOTS = ["recon-corpus", "echidna", "magic"] as const;

That is an exact-name list. The agent did not use those names — it created recon-corpus-deep
and echidna-deep, which are not in the list, so they were captured. One file in
recon-corpus-deep alone contributed ≥33.8 MB, over the 32 MiB buffer by itself.

Grepping .ultrafuzz/prompts/strategies/invariants/ for -deep returns nothing, so these names are
not prompt-specified. The agent chose them, presumably for a deeper fuzzing pass alongside the
prompt-specified --corpus-dir echidna --recon-corpus-dir recon-corpus. Any other variant
(recon-corpus-2, corpus-deep, …) would escape the same way.

So the exclusion mechanism assumes it can enumerate every generated directory by name, and nothing
constrains the agent to those names.

The #311 fix is what made this diagnosable

Worth noting because it validates that change: the message names the actual contributors with byte
counts and states its own visibility limit ("anything past that cutoff is not visible here"). Before
#311 this was an anonymous ENOBUFS. The diagnosis took one read.

Candidate fixes, not presuming which

Prefix/glob matching on the existing roots (recon-corpus*, echidna*). Smallest change, but it
is still a name-based guess and corpus-deep would escape it.
Exclude by what the directory IS rather than what it is called — e.g. any root the target
gitignores, or any untracked root above a size threshold. Robust against naming, but changes what a
legitimate large artifact means and needs care so real work is never silently dropped.
Constrain the agent to write corpora only under a single reserved root, and treat anything else
as a contract violation. Makes the invariant explicit rather than inferred, but it is a prompt change
and only as reliable as the agent's compliance.

Option 2 is the only one that does not depend on predicting a name, but it is also the one that could
silently exclude something a node legitimately needs to hand off, so it needs the most care. I am not
choosing here.

Acceptance criteria

A generated corpus directory with an agent-chosen name does not enter the workspace capture.
A regression test covers a variant name, not just the three known ones.
The failure mode, if it recurs, still names the contributing paths (do not regress fix(runtime): name the largest paths when a git capture overflows its buffer #311).

I prototyped the size ceiling (candidate 2) under TDD and hit a design problem worth recording before
anyone implements it, because the obvious version is worse than it looks.

The prototype works for the reported defect. A test that creates an untracked recon-corpus-deep/
above the ceiling now captures cleanly, with authored source still handed off:

excludes an oversized untracked root whose name no list could have predicted   PASS
keeps a small untracked root with an unfamiliar name                           PASS

The second test matters as much as the first: the ceiling must not become a licence to drop authored
work, and a small untracked directory with an unfamiliar name is exactly what a node legitimately hands
off.

But it breaks two existing #311 tests, and the reason is the design, not the tests:

attributes a capture overflow to what fed the diff, not to bulk that contributed nothing   FAIL
ranks the real capture by real bytes, through git and the code that runs it                FAIL

Both construct a single large untracked root and assert a loud ENOBUFS failure naming it. With a
ceiling, that root is silently excluded instead and no overflow occurs. So a naive size ceiling
converts a loud failure into silent data loss — precisely what the comments around
stageWorkspaceTree warn about for tracked paths:

Excluding a tracked path here would be silent data loss ... every check still passes and the
downstream node simply sees stale content.

The same hazard applies to untracked authored content above the ceiling. Today it fails loudly; a naive
ceiling would drop it and let every downstream check pass against stale content.

What a correct version needs, on top of the ceiling:

Record the exclusions in the manifest (e.g. an optional excluded_roots: {path, bytes}[]), so an
omission is auditable rather than invisible. validateManifest ignores unknown fields today, so this
is additive — but it needs plumbing from stageWorkspaceTree (which returns pathspecs) up through
captureWorkspaceTree/captureWorkspacePatch.
Re-aim the two fix(runtime): name the largest paths when a git capture overflows its buffer #311 tests at aggregate overflow — several roots each under the ceiling — so the
contributor ranking stays genuinely covered for the case the ceiling does not neutralise. Deleting or
weakening them would discard the diagnostic that made THIS issue diagnosable in one read.

One thing the prototype did settle: capture/apply symmetry holds. applyWorkspacePatch recomputes
trees on the dependent worktree, where an excluded root was never delivered and so does not exist, and
excluding an absent root is a no-op. A filesystem-sensitive rule does not break the tree comparison.

I am leaving this unshipped rather than merging the naive version. A change to the workspace-capture
core whose main effect is to hide data loss is worse than the failure it replaces, and the run it would
unblock is not worth that trade.

R53 has now failed terminally on this, and the retry reproduced it identically — so the agent's choice of
-deep names is deterministic, not a one-off.

state.json at 22:53Z: run: failed, 23 succeeded / 8 pending / 1 failed.

attempt	window	outcome
1	22:14:47Z → 22:29:41Z (~15 min)	`recon-corpus-deep (>=33865139 diff bytes in 1 file)`, `echidna-deep (>=37453 in 15 files)`
2	22:34:17Z → 22:41:35Z (~7 min)	`recon-corpus-deep (>=33871011 diff bytes in 2 files)`, `echidna-deep (>=47965 in 19 files)`

Same node, same two roots, same message, retry_count: 1 and no attempts left. Attempt 2 got there in
half the time and wrote more corpus, so nothing about the first failure discouraged the second: the
recovery generation restored the dependency baseline, the agent ran the same deep fuzzing pass into the
same self-chosen directories, and the handoff blew the same buffer.

Two things this settles:

This is the whole run blocker. Every one of the 8 pending nodes — including final-report — is
downstream of stateful-invariant-handlers. Nothing else failed.
Retry is not a mitigation. The name choice is stable across attempts, so raising max_attempts
would only buy more 7-minute failures.

The #311 diagnostic did its job again on both attempts: two reads, no guessing, and the byte counts even
show the corpus growing between attempts.

Reviewed #372 against this — the rule looks right, but I reproduced a blocking gap in it end to end and
left the measurement plus a patch there rather than duplicating it here.

Closing this needs two PRs, not one, and only one of them existed:

Exclude oversized untracked roots by size, and record what was left out #372 (runtime) — drops an oversized untracked root and records it in the manifest.
fix(artifacts): let a workspace-patch manifest record what the capture left out #374 (artifacts) — makes that record legal at the artifact gate.

Without #374, #372 does not unblock a run: workspacePatchSchema is a z.strictObject and
validateArtifactContract routes ultrafuzz/workspace-patch@1 into it, which artifact-gates.ts calls on
every declared node output — and stateful-invariant-handlers declares workspace-patch.json with exactly
that contract. So the node trades an ENOBUFS for WORKSPACE_PATCH_SCHEMA_INVALID: Unrecognized key: "excluded_roots" at the same node.

Measured on both packages against one capture of the R53 shape:

gate	result
`origin/main` artifacts	`false` — `Unrecognized key: "excluded_roots"`
#374	`true`

Against the acceptance criteria:

A generated corpus directory with an agent-chosen name does not enter the workspace capture — Exclude oversized untracked roots by size, and record what was left out #372,
confirmed end to end: excluded_roots: [{"path":"recon-corpus-deep","bytes":35651584}], with
files: [{"path":"test/Handlers.t.sol"}] still handed off.
A regression test covers a variant name, not just the three known ones — Exclude oversized untracked roots by size, and record what was left out #372's ceiling tests, and
fix(artifacts): let a workspace-patch manifest record what the capture left out #374 uses recon-corpus-deep as its fixture rather than a synthetic name.
The failure mode, if it recurs, still names the contributing paths — Exclude oversized untracked roots by size, and record what was left out #372 re-aimed the two fix(runtime): name the largest paths when a git capture overflows its buffer #311 tests
at aggregate overflow (six roots each under the ceiling) rather than deleting them, so contributor
ranking stays covered for the case a ceiling deliberately does not catch.

The general lesson, worth stating because it is what let this reach a live run: the runtime's own
validateManifest ignores unknown keys, so a manifest-field change stays green through the entire runtime
suite and fails only on Modal. Any new field needs all three of the zod strictObject, the JSON Schema
constant, and the checked-in packages/artifacts/schema/*.schema.json snapshot.

PR #372 is closed unmerged. It was wrong in a way worth recording so the next attempt does not repeat
it, and the correct design is below.

Why the size-ceiling approach failed

It silently destroyed the deliverable of this very node. The exclusion keyed on the FIRST path
segment, and in a Foundry target the node's authored Solidity lands under test/. Reproduced:

tracked test/ + authored test/recon/Handlers.t.sol, test/recon/TargetFunctions.sol
            + 9 MiB test/recon/corpus-deep/seed.bin

files handed off : []
excluded_roots   : [{"path":"test","bytes":9437224}]

A corpus written anywhere under test/ takes the whole root with it. That is not a tweak to the
approach; it follows from choosing path-prefix as the exclusion unit.

It would not have unblocked the run anyway. workspacePatchSchema
(packages/artifacts/src/workspace-patch.ts:66) is a z.strictObject, and artifact-contracts.ts:311
routes ultrafuzz/workspace-patch@1 into it — so the new excluded_roots field fails the contract the
moment the ceiling fires. ENOBUFS traded for WORKSPACE_PATCH_SCHEMA_INVALID.

The metric was wrong, and this repo rejected it once already. PR #306 was closed for measuring the
workspace instead of the diff. Scoping to untracked fixes #306's largest defect but keeps on-disk bytes
as the proxy, and measurement shows error in both directions: 7 MiB of NULs deflates to ~41 KB of patch
(over-exclusion by ~200x), while five 7 MiB incompressible roots still overflow (under-exclusion).

excluded_roots was a fig leaf. Nothing in the repo reads it. A manifest field with no consumer
documents data loss rather than mitigating it.

The design that should be built instead

Capture, and only on a MEASURED overflow, retry with the largest contributors excluded — ranked by real
diff bytes.

git-capture-diagnostics.ts already ranks roots by actual diff bytes for #311, which is the metric that
determines whether the capture breaks. So:

it keys on the thing that actually fails, not on a proxy that is wrong in both directions;
it fires only when a capture would genuinely fail, so the common case is untouched and nothing is
dropped speculatively;
it covers the aggregate case a per-root ceiling cannot;
it needs no magic constant and no name guessing.

Costs: a second diff on the overflow path, and #311's ranking is a prefix floor (git emits in path order,
so contributors past the buffer cutoff are invisible) — so the retry may need more than one round.

Whatever excludes anything must ALSO be visible, and that means extending the strict schema deliberately
plus giving the field a consumer or a gate — not adding an optional key and calling it recorded.

Cheap complements, neither sufficient alone

A prefix rule (recon-corpus*, echidna*) would have caught the observed incident at near-zero risk.
Not durable, but strictly better than today's exact-name list.
Reserving a scratch root under an existing exclusion and pointing the invariant prompts at it would
have made R53 pass. Prompt discipline cannot be the guarantee — both R53 attempts independently
invented recon-corpus-deep/ — but it is worth doing alongside.

fix(runtime): narrow the prefix to the two roots agents actually rena…

R53 is now terminally failed — the retry budget was spent on four byte-identical attempts

Follow-up evidence from the same run this issue was filed against
(ultrafuzz-aave-v4-v0012-main-issue357fix-r53-invar-a7b16bc8b56d). The run did not just hit this once
at 22:14 — it exhausted its retries and died.

Run state (state.json, read 23:34Z):

status            failed
started_at        2026-08-07T20:15:01.098Z
finished_at       2026-08-07T23:25:05.897Z   (3h10m)
nodes             23 succeeded / 1 failed / 8 pending

The 8 pending nodes never started — all wait_reason: dependency behind the failed one:
stateful-invariant-coverage, stateful-invariant-implement-properties, stateful-invariant-campaign,
dedupe-findings, triage, severity-classification, aggregate-test-files, final-report.

stateful-invariant-handlers: retry_count: 3, four attempts, all the same failure
(attempts.jsonl, all outcome: failed / executor-error):

attempt	window	duration
`attempt-d980c278`	22:14:47 → 22:29:41	14m54s
`attempt-41801aac`	22:34:17 → 22:41:35	7m18s
`attempt-76d9483c`	22:53:43 → 22:59:59	6m15s
`attempt-b9cfca3e`	23:17:15 → 23:24:14	6m59s

All four carry the same input_sha256: d3dbd5010519a9c9fd14151b7b3c645591ec661361003a8f30c93c1310bbab62.

Two things this adds to the diagnosis

1. recon-corpus-deep + echidna-deep is not a one-off dice roll — it reproduced 4/4.

stream.ndjson carries the identical contributor line for the first three attempts:

recon-corpus-deep (>=33865139 diff bytes in 1 file), echidna-deep (>=37453 diff bytes in 15 files)

and the terminal last_error in state.json shows the fourth attempt went further, not less far:

Largest contributors within the first 33914880 bytes git wrote; ...
recon-corpus-deep (>=33864319 diff bytes in 1 file), echidna-deep (>=50561 diff bytes in 20 files)

echidna-deep grew 37453 B / 15 files → 50561 B / 20 files. The agent is not backing off across retries;
it re-derives the same two directory names and writes more into them.

This settles the "cheap complements" question at the bottom of the issue in one direction: prompt
discipline cannot be the guarantee, and it is worse than the issue text suggests. It is not that "both
R53 attempts independently invented recon-corpus-deep/" — all four did, deterministically. A name
list or a prompt nudge is fighting a stable attractor in the model's naming, not a random variant.

Correspondingly it strengthens the case for the measured-overflow design you laid out over #375's prefix
rule: prefix matching happens to cover recon-corpus-deep/echidna-deep and would have saved this exact
run, but the reproducibility here shows the agent picks names with conviction, so the next escaping name
(corpus-deep, fuzz-corpus/) will also arrive four times in a row rather than once.

2. The retry budget bought nothing, and it structurally could not have.

The four attempts were not four chances — the retried agent's prompt is byte-for-byte the string attempt 1
got. The prompt is rendered once at plan time (packages/runtime/src/plan-run.ts:209 →
packages/prompts/src/render.ts:229, written to prompt.rendered.md), and execution-time assembly at
packages/runtime/src/templates/smithers/workflows/workflow.tsx:4795 concatenates only constants and that
plan-time file. SUPPORTED_TEMPLATE_VARIABLES (packages/prompts/src/render.ts:8-31) has no error-shaped
variable, and render.ts:194-197 hard-errors on unknown ones, so a prompt author cannot reference the
prior failure even deliberately. resetTaskArtifactsForRetry (workflow.tsx:250-298) additionally wipes
the previous attempt's artifact dir, so the agent cannot infer it either.

So ~35 minutes of node execution across four attempts was spent re-running one identical decision. I have
filed that as a separate defect since it is not specific to corpus capture — it makes every
agent-failure retry in the system a replay rather than a retry. Linking from here because R53 is its
cleanest reproduction, and because it is the reason this issue's blast radius was a dead 3-hour run rather
than one lost attempt.

Not touching #374/#375 — flagging the evidence only.

Correction to the durable design I recorded above, before anyone builds it. A reviewer of PR #375
raised this as a hypothesis; I verified it, and it is right.

I wrote that the durable fix is to "retry on MEASURED overflow, excluding the largest contributors ranked
by real diff bytes, which git-capture-diagnostics.ts already computes". The metric is right. The
exclusion unit is not, and taken naively that design reproduces both defects that closed #372.

git-capture-diagnostics.ts:243 derives each ranked entry as:

pending = { root: (header[1] ?? "").split("/")[0] ?? "", start: header.index };

First path segment. So:

It would drop test/ wholesale. A corpus at test/recon/corpus-deep/ ranks under root test, and
a retry that excludes the top-ranked root discards the node's entire authored suite. That is Exclude oversized untracked roots by size, and record what was left out #372's
headline failure — verified there as files handed off: [] — arriving again with a better metric.
The close comment on that PR says the granularity problem "is not a tweak to this approach, it is a
consequence of choosing path-prefix as the exclusion unit", and the ranking is path-prefix.
It would drop tracked edits. The ranking parses the WHOLE diff, which includes tracked
modifications, so a top-ranked root can carry authored tracked changes. Exclude oversized untracked roots by size, and record what was left out #372's other fatal defect.

Two constraints the durable design must carry, which my earlier note omitted:

Exclude at file granularity, or at worst the deepest directory that is itself the contributor —
never the first path segment. The ranking as it stands is a diagnostic aggregation for humans; it is
not an exclusion unit, and reusing it as one is the trap.
Intersect any exclusion set with the untracked listing, so a tracked edit can never be dropped
regardless of how the diff bytes rank.

Both are cheap to state and expensive to discover, which is why they belong here rather than in whoever
picks this up. Note that the ranking is also a documented prefix floor — git emits in path order, so
contributors past the buffer cutoff are invisible — so a retry may need more than one round, and each
round must re-measure rather than trusting the first ranking.

Match generated corpus roots by prefix, not exact name (#375)

Keep Foundry out/ out of workspace capture and make patch-overflow at…

Navigation Menu

Repository navigation

Generated-corpus exclusion is an exact-name list, so an agent-chosen variant like recon-corpus-deep blows the capture buffer again #368

What happened

Why this is a NEW defect and not a regression of #305

The #311 fix is what made this diagnosable

Candidate fixes, not presuming which

Acceptance criteria

Why the size-ceiling approach failed

The design that should be built instead

Cheap complements, neither sufficient alone

R53 is now terminally failed — the retry budget was spent on four byte-identical attempts

Two things this adds to the diagnosis

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Generated-corpus exclusion is an exact-name list, so an agent-chosen variant like recon-corpus-deep blows the capture buffer again #368

Description

What happened

Why this is a NEW defect and not a regression of #305

The #311 fix is what made this diagnosable

Candidate fixes, not presuming which

Acceptance criteria

Activity

aviggiano commented on Aug 7, 2026

aviggiano commented on Aug 7, 2026

aviggiano commented on Aug 7, 2026

aviggiano commented on Aug 7, 2026

Why the size-ceiling approach failed

The design that should be built instead

Cheap complements, neither sufficient alone

aviggiano commented on Aug 7, 2026

R53 is now terminally failed — the retry budget was spent on four byte-identical attempts

Two things this adds to the diagnosis

aviggiano commented on Aug 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions