The Recon campaign's three findings are discarded on a schema_version literal the contract never states

R52 (aave-v4-v0012-main-issue328fix-r52-invariant-only, image from merged main 3190f90) is the first run in this investigation to enter stateful-invariant-campaign — the Recon campaign — and the first to produce findings. It ran 18:37:09 → 18:52:35 and emitted three:

id	title	property
`failure-1`	Harness interest-rate assertion fails after one second of index growth	—
`failure-2`	Hub added assets exceed sum of listed Spoke converted assets by one unit	`property-49`
`failure-3`	Supplying after accrued debt decreases the added-share exchange ratio	`property-70`

All three were then discarded:

artifact-contract failure for findings.json (ultrafuzz/findings@1):
Invalid input: expected "1.0" at findings.json[0].schema_version,
findings.json[1].schema_version, findings.json[2].schema_version

The document carries schema_version: "ultrafuzz.finding.v1" on each entry.

Why the producer had no way to get this right

FINDINGS_SCHEMA_VERSION = "1.0" (packages/artifacts/src/findings.ts:6) is the odd one out. Every other artifact in the system uses the ultrafuzz.<name>.v1 form:

ultrafuzz.analysis-bundle.v1        ultrafuzz.properties.v1
ultrafuzz.implemented-properties.v1 ultrafuzz.property-campaign.v1
ultrafuzz.invariant-source-proof.v1 ultrafuzz.property-lens.v1
ultrafuzz.workspace-patch.v1        ultrafuzz.invariant-evidence-ledger.v1

And the contract description (packages/artifacts/src/artifact-contracts.ts:146) names the field without stating its value:

"A JSON array of findings. Every entry must satisfy the Ultrafuzz finding schema, including schema_version, id, title, status, severity_guess, confidence, and summary."

Sibling contracts do better — ultrafuzz/generated-tests@1 ships validEmptyExample: '{"schema_version":"1.0",...}' (:154), which tells a producer the literal. ultrafuzz/findings@1 has no such example.

So a producer that infers the convention from eight sibling schemas emits ultrafuzz.finding.v1, and nothing in the contract it was handed says otherwise.

Impact

This is the last node before final-report, which produces the report.md no run has ever produced. Three findings from a completed Recon campaign are being thrown away on a version string.

Note on the diagnostic

The error above names findings.json[0].schema_version, [1], [2] — exact field paths. That is #328's second half working on its first production run. The same failure one image earlier would have read Invalid input: expected "1.0" three times with nothing to locate it, and would have cost another round of volume archaeology. Recording that because the diagnostic fix earned its keep immediately.

Candidate fixes, not presuming which

Accept both literals and add a validEmptyExample naming one. Additive and backward compatible; unblocks the run.
State the required literal in the contract description and leave validation alone. Correct if "1.0" is deliberate, but leaves the convention inconsistent across the schema family.

Option 1 unblocks, but if "1.0" is a deliberate external contract then loosening it is wrong and the prompt-facing description is what should change. I am not picking the answer that unblocks fastest without that being decided.

implemented-properties rejects an empty OPTIONAL array, and the contract error discards the field path that would have said so #328 — the previous contract defect at the previous node, whose diagnostic half produced the field paths above.
applyWorkspacePatch base tree mismatch blocks stateful-invariant-implement-properties, with maxAttempts=1 so it kills the run #312, Invariant-suite handoff treats a superseded ancestor source as a conflict, because ancestors are ordered alphabetically rather than causally #315 — the two supersession defects that got a run this far.

Retracting this issue: I filed a mechanism I never reproduced.

I read the actual artifacts off the R52 volume. Both findings documents are literally [] (3 bytes):

artifacts/stateful-invariant-coverage/findings.json -> []
artifacts/stateful-invariant-implement-properties/findings.json -> []

There are zero findings in R52, so there were never "three findings" to discard. The workflow log
contains no unsupported schema_version string and no ultrafuzz/findings@1 rejection either — I
searched for the mechanism directly and it is absent.

The "three findings" claim was the same false-positive class as an earlier mistake of mine, where a
/report\.md$/ suffix match counted coverage-report.md as a report: I counted something that was
not a finding and then reasoned from the count.

For the record, the schema_version handling I described is not even reachable the way I claimed.
normalizeFinding does optionalString(value, "schema_version") ?? FINDINGS_SCHEMA_VERSION, so an
absent field defaults to "1.0" and passes. Only a present-and-different value is rejected. No
producer in this run emitted the field at all.

Closing as invalid. R52's real terminal causes are tracked in #285 (fan-in reference expectations)
and the new issue for the campaign node's workspace patch, filed separately.

Reproduced in R56 with the artifact on disk. Reopening.

This was closed as invalid on the grounds that the mechanism was never reached — "no producer in
this run emitted the field at all". That was correct about R52. It is not correct in general, and
R56 has now hit it. The retraction disproved the count of R52's findings; it did not disprove the
mechanism, and the mechanism is what killed this run.

Run: aave-v4-v0012-main-issue386fix-r56-invariant-only-gpt-5-5-…-trial-1
(volume vo-QBKtoUv70UhUj3z7yzxTc3), 26 nodes succeeded, stateful-invariant-campaign ran
09:39:17 → 10:00:15 UTC and failed on attempt 1 with retry_count: 0, taking the run to failed
with dedupe-findings, triage, severity-classification, aggregate-test-files and
final-report left pending / dependency.

"last_error": "artifact-contract failure for findings.json (ultrafuzz/findings@1):
  Invalid input: expected \"1.0\" at findings.json[0].schema_version, findings.json[1].schema_version"

The document the gate rejected is on the volume at
workspaces/stateful-invariant-campaign/artifacts/stateful-invariant-campaign/findings.json —
two entries, both "schema_version": "ultrafuzz.finding.v1", both otherwise complete:

id	schema_version	classification	title
`failure-1`	`ultrafuzz.finding.v1`	`harness-defect`	Harness drawn-rate sync assertion ignores elapsed-time precondition
`failure-2`	`ultrafuzz.finding.v1`	`harness-defect`	Harness collateral-status assertion is stronger than catalog collateral status

Each carries property_ids, backend_provenance with the Recon run-20260808T094655Z corpus
reproducer paths, severity_guess, confidence, summary, reproduction, and
deduplicated_failure_ids. Nothing about them is malformed except the version string. This is a
completed 21-minute Recon campaign discarded on a literal.

The prompt is where it goes wrong, and I can quote it

prompt.rendered.md for the node renders three JSON contracts back to back. Two of them state a
schema_version value; the third — the one being validated against a z.literal — states none:

  Contract: `ultrafuzz/property-campaign@1`
  Valid empty form: `{"schema_version":"ultrafuzz.property-campaign.v1","failures":[]}`

  Contract: `ultrafuzz/generated-tests@1`
  Valid empty form: `{"schema_version":"1.0","run_id":"<run-id>","node_id":"<node-id>","generated_tests":[]}`

  Contract: `ultrafuzz/findings@1`
  Schema: A JSON array of findings. Every entry must satisfy the Ultrafuzz finding schema,
          including schema_version, id, title, status, severity_guess, confidence, and summary.
  Valid empty form: `[]`

The description names the field and withholds its value. [] is a legal empty findings array, so
the example — the only place the other two contracts state their literal — teaches nothing here.
The two adjacent contracts disagree with each other on the convention, and six other artifacts in
the family (ultrafuzz.properties.v1, ultrafuzz.workspace-patch.v1,
ultrafuzz.implemented-properties.v1, ultrafuzz.invariant-evidence-ledger.v1,
ultrafuzz.property-lens.v1, ultrafuzz.invariant-source-proof.v1) use the ultrafuzz.<name>.v1
form. A producer generalizing from what it was handed writes ultrafuzz.finding.v1. That is the
observed behaviour, not a hypothesis.

Note this cannot be fixed by putting the literal in validEmptyExample: that string is written to
disk verbatim as the placeholder artifact
(workflow.tsx:1710, and findings@1 is the one primary contract allowed to be materialized), so it
has to stay []. The description is the only channel.

Why strictness buys nothing here

normalizeFinding (packages/artifacts/src/findings.ts:129) already defaults an absent
schema_version to "1.0" and accepts it. There is exactly one findings schema version, so the
field discriminates nothing — omitting it passes, and naming it in the house style is terminal.

Scope of the loss

stateful-invariant-campaign is the last node before the report chain. R56 is the furthest any run
in this investigation has reached, and the two findings it produced are the first real campaign
output. The run was resumed at 10:07:41 and the node is re-running now; on a fresh sample the model
may or may not pick the same string, which is the definition of a flaky terminal gate.

Fix in progress: state the literal in the contract description, and accept ultrafuzz.finding.v1
as an alias normalized to "1.0" so a run is never again lost to a version string that carries no
information. PR to follow.

Fix opened as #401 — the contract description now states the literal, and ultrafuzz.finding.v1 is accepted and normalized to "1.0". Both regression tests are red on origin/main.

The retry reproduced it exactly. R56 is dead; it is deterministic, not flaky.

I said above that on a fresh attempt the model "may or may not pick the same string". It picked the
same string. R56's final state.json:

run status: failed        2026-08-08T10:14:29.906Z
nodes: 26 succeeded, 1 failed, 5 pending
stateful-invariant-campaign: failed, retry_count 1,
  attempt 2: 10:07:41.847Z -> 10:13:35.367Z
  artifact-contract failure for findings.json (ultrafuzz/findings@1):
  Invalid input: expected "1.0" at findings.json[0].schema_version, findings.json[1].schema_version

Two independent attempts, byte-identical error, same two entries. The retry is a replay of the same
prompt, and the prompt is what is wrong — so the retry budget cannot spend its way out of this. R56
terminated with final-report never reached, and R57
(aave-v4-v0012-main-issue398fix-r57-invariant-only) launched at 10:11 UTC.

R57 carries an image built from main at #398/#399, which predates #401. If its campaign node
produces findings, it will hit this same wall at the same place. Worth folding #401 into the next
image before the run gets that far.

R56 has now failed on this three times in a row, and R57 is queued behind it

Live evidence from aave-v4-v0012-main-issue386fix-r56-invariant-only as of 2026-08-08T10:40Z. The
node is stateful-invariant-campaign; the run is still running only because it keeps retrying.

attempt	window (UTC)	outcome	`findings.json` `schema_version`
1	09:39:17 → 10:00:15 (21 min)	`failed / executor-error`	`ultrafuzz.finding.v1` ×2
2	10:07:41 → 10:13:35 (6 min)	`failed / executor-error`	unchanged on disk (mtime still 09:58)
3	10:25:09 → in flight	writing now	`ultrafuzz.finding.v1` ×1 (written 10:39:49)

Attempts 1 and 2 both carry the identical gate error:

artifact-contract failure for findings.json (ultrafuzz/findings@1):
Invalid input: expected "1.0" at findings.json[0].schema_version, findings.json[1].schema_version

Two things this adds beyond the original report:

It is not a one-shot producer slip. Attempt 3 re-ran the whole Recon campaign from scratch and
independently wrote "schema_version": "ultrafuzz.finding.v1" again. Three producer runs, three
times the same value — the contract is teaching it, exactly as #401 argues.
Retrying costs findings, it does not recover them. Attempt 1 produced two findings
(Harness drawn-rate sync assertion ignores elapsed-time precondition,
Harness collateral-status assertion is stronger than catalog collateral accounting rule).
Attempt 3 produced one — the collateral-status finding is gone. Each retry is a fresh
~20-minute campaign against a fresh corpus, so the retry loop is not converging on the same
result set; it is discarding real output and re-rolling. Retrying is not a workaround for this bug.

Also worth recording: attempt 2 failed on the gate without the producer having rewritten
findings.json at all (mtime stayed 09:58 across the whole 10:07→10:13 window), so it burned six
minutes re-validating attempt 1's artifact and re-reporting attempt 1's error.

#401 is MERGEABLE / CLEAN. R57 (…issue398fix-r57…) launched at 10:12 and is currently at
setup-foundry; on R56's timings it reaches the campaign in roughly two hours and, without #401,
hits this same wall.

R56 is now terminally failed. Attempt 3 finished at 10:42:02Z with the identical error and the retry budget is exhausted:

status: failed   last_transition_at: 2026-08-08T10:42:02.863Z
nodes: {succeeded: 26, failed: 1, pending: 5}
stateful-invariant-campaign: failed
  artifact-contract failure for findings.json (ultrafuzz/findings@1):
  Invalid input: expected "1.0" at findings.json[0].schema_version

dedupe-findings, triage, severity-classification, aggregate-test-files and final-report never ran. Three attempts, three producer runs, three times ultrafuzz.finding.v1 — 63 minutes of campaign time and the deepest run of this investigation, ended by a version string. R57 is behind it.

fix(artifacts): stop losing a campaign's findings to a version string (…

Navigation Menu

Repository navigation

The Recon campaign's three findings are discarded on a schema_version literal the contract never states #356

Why the producer had no way to get this right

Impact

Note on the diagnostic

Candidate fixes, not presuming which

Related

Reproduced in R56 with the artifact on disk. Reopening.

The prompt is where it goes wrong, and I can quote it

Why strictness buys nothing here

Scope of the loss

The retry reproduced it exactly. R56 is dead; it is deterministic, not flaky.

R56 has now failed on this three times in a row, and R57 is queued behind it

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

The Recon campaign's three findings are discarded on a schema_version literal the contract never states #356

Description

Why the producer had no way to get this right

Impact

Note on the diagnostic

Candidate fixes, not presuming which

Related

Activity

aviggiano commented on Aug 7, 2026

aviggiano commented on Aug 8, 2026

Reproduced in R56 with the artifact on disk. Reopening.

The prompt is where it goes wrong, and I can quote it

Why strictness buys nothing here

Scope of the loss

aviggiano commented on Aug 8, 2026

aviggiano commented on Aug 8, 2026

The retry reproduced it exactly. R56 is dead; it is deterministic, not flaky.

aviggiano commented on Aug 8, 2026

R56 has now failed on this three times in a row, and R57 is queued behind it

aviggiano commented on Aug 8, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions