Skip to content

Property lens expectations in camelCase are neither authorised nor stripped, so the node dies (third shape of #285) #293

Closed

Description

@aviggiano

Third live occurrence of the same underlying defect, under a third identifier shape. Found in Aave v4 run R45 at 2026-08-06T18:28:20Z.

The failure

property-specification-runtime-verification failed with:

Property lens expectation "testConvertToAssetsSharesDesirable" is not present in a supplied pinned-reference catalog;
Property lens expectation "testConvertToSharesAssetsDesirable" is not present in a supplied pinned-reference catalog;
Property lens expectation "testPreviewDepositZeroAmountReturnsZero" is not present in a supplied pinned-reference catalog;
Property lens expectation "testPreviewRedeemZeroAmountReturnsZero" is not present in a supplied pinned-reference catalog;
Property lens expectation "testPreviewWithdrawPositiveAmountReturnsGtZero" is not present in a supplied pinned-reference catalog

These are test-function names from that lens's own pinned reference — the ERC4626 property suite. The lens prompt tells the model to carry identifiers unchanged from the pinned-reference documents, and it did.

Mechanism, verified mechanically

sanitizeLensReferenceExpectationAuthority strips unauthorised expectations only when every unauthorised identifier looks like a catalogue label (artifact-gates.ts:1725):

if (unauthorized.some((expectationId) => !isUnsupportedExternalReferenceLabel(expectationId))) return property;

and the shape test (artifact-gates.ts:1792) is:

/^[A-Z][A-Z0-9]*(?:[-_][A-Z0-9]+)*[-_]\d+$/u
identifier matches catalogue shape
LEND-01 yes
LEND_ACC_01 yes (added by #284)
ERC4626-01 yes
A16Z_ERC4626_07 yes
testConvertToAssetsSharesDesirable no
testPreviewDepositZeroAmountReturnsZero no

So a camelCase identifier is neither authorised nor strippable, and the node fails.

History — this is whack-a-mole, and it should stop being played that way

  1. LEND-01, hyphenated: stripped correctly.
  2. LEND_ACC_01, underscored: killed R44 → Lens reference-expectation sanitizer only recognizes hyphenated labels, so underscore catalog IDs fail the node #283, fixed by fix(runtime): sanitize underscore-separated reference labels #284 widening the shape to accept _.
  3. testConvertToAssetsSharesDesirable, camelCase: kills R45's runtime-verification lens now.

A fourth widening would accept almost any token and would defeat the gate's purpose, so the shape test is the wrong axis. The real issue is the one #285 already states: the gate only honours identifiers from a structured ultrafuzz/reference-expectations@1 catalogue, and these runs never supply one, so every citation is technically unauthorised and survival depends on cosmetic identifier shape.

Suggested direction, not a patch

Distinguish two situations the gate currently conflates:

  • No catalogue supplied at all. No identifier could ever be authorised, so failing the node punishes the model for following its prompt. Strip and warn.
  • A catalogue is supplied and the identifier is absent from it. That is the forgery case, and it must keep failing closed.

This split looks compatible with the seven existing tests that pin byte-unchanged preservation of a forged benchmark mapping — worth verifying, because two earlier review suggestions (per-identifier stripping, and flipping the "unverifiable" policy) were rejected precisely for breaking them. Whoever picks this up should check whether those fixtures supply a catalogue; if they do, the split is safe and #285 can be closed with it.

Severity

Not run-fatal. R45's fan-in aggregated the seven surviving lenses and the pipeline continued to stateful-invariant-setup. The cost is the silent loss of one reference lens's entire contribution, which is a coverage regression that no report would show as missing.

Activity

  1. aviggiano commented on Aug 6, 2026

    @aviggiano
    CollaboratorAuthor

    A fourth shape, from a fourth run, on the image built immediately before the fix merged

    R46's property-specification-a16z failed at ~19:55Z:

    Property lens expectation "prop_asset" is not present in a supplied pinned-reference catalog
    

    prop_asset is lowercase with an underscore and no numeric suffix. The old shape matcher
    ^[A-Z][A-Z0-9]*(?:[-_][A-Z0-9]+)*[-_]\d+$ rejects it, so it was neither authorised nor strippable and the node died — the same mechanism, now under a fourth distinct identifier shape:

    run node identifier matched old shape
    — any LEND-01 yes → stripped
    R44 property-specification-0kn0t LEND_ACC_01 no → killed (#283)
    R45 property-specification-runtime-verification testConvertToAssetsSharesDesirable no → killed (this issue)
    R46 property-specification-a16z prop_asset no → killed

    Four shapes, four runs, one defect. This is the clearest possible argument that the shape pattern was the wrong axis, and that a fifth widening would only have deferred the next occurrence.

    Verified against the merged fix (980fe2b): prop_asset and prop_totalAssets both match COPIED_REFERENCE_CITATION, claim no reserved authority, and are therefore stripped — the node survives with a warning instead of dying.

    One more data point worth recording: R46's property-specification-runtime-verification succeeded this run, on the same image where R45's copy of that node died. So which lens dies is a shape lottery, not a property of the lens — exactly what you would expect if survival depends on the cosmetics of whatever identifiers the model happened to copy.

    Closing this: the mechanism is confirmed four times over, the fix is merged, and it demonstrably handles every observed shape. The remaining authority question — that no catalogue is ever supplied, so every citation is technically unauthorised — stays open as #285.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions