Skip to content

Bounded report preservation gate contradicts its own enrichment contract, failing every compliant final report #918

Closed

Description

@aviggiano

What happened

With #886/#892/#901/#909 in, smoke run 32898410734 (candidate a695b0a) executed the full pipeline for the first time — 2.5 hours of real agent work, all three workflows launched, every review stage ran — and all three targets failed at the very last node, final-report, on the report-severity-classification-preservation semantic gate:

Bounded report did not preserve dedupe field "lifecycle" at $.issues[0..3].lifecycle       (venus: every issue)
Bounded report did not preserve dedupe field "proof_of_concept"/"description" …            (very-liquid, stableswap)
Bounded report did not preserve dedupe field "triage_classification" at $.non_production…  (stableswap)

Root cause (provable from the contract itself)

The smoke lane's packaged topology omits severity classification, so the gate takes the bounded classification branch (reportBoundedDedupePreservationIssues). The final-report prompt for this mode instructs the agent to:

  • "enrich its matching dedupe lifecycle record only in the report object" (triage_classification, triage_reason, final_disposition, demotion_reason, canonical_severity), and
  • "choose exactly one triage_classification".

The gate honors that contract in its dedicated checks — lifecycleRecordPreservationIssues validates the report lifecycle against the authenticated ledger with BOUNDED_REPORT_LIFECYCLE_OWNED_FIELDS, and the row-level check requires row.triage_classification to equal the enriched lifecycle classification — but its generic loop then deep-equals every field of the dedupe finding against the report row, skipping only id/title for promoted rows. So whenever a dedupe row itself carries lifecycle or a preliminary triage_classification (schema-legal, and exactly what the venus and stableswap dedupe artifacts contained), a report that follows the prompt can never pass: venus failed on lifecycle for all four issues while preserving every prose field byte-for-byte — a maximally compliant report, rejected.

The gate landed in #558 (Aug 13) and was tightened in #621 (Aug 22), both after the last complete benchmark run (Aug 12), so the contradiction was never exercised end-to-end. Fifth latent post-Aug-12 regression surfaced by re-enabling the lane (#883, #889, #899, #908).

Fix

In the bounded branch's generic preservation loop, skip exactly the two fields that already have dedicated enrichment-aware validation: lifecycle and triage_classification. Everything else — including description, proof_of_concept, summary, evidence fields — keeps strict byte-preservation (venus demonstrates models comply). Regression test added: a dedupe row that embeds lifecycle + a preliminary classification passes with a compliant enriched report row, and tampering any non-owned field (e.g. summary) still fails.

The very-liquid/stableswap prose mismatches may additionally reflect model copy drift across retries; with the deterministic contradiction removed, retries can actually converge, and any residual prose flakiness will now be observable in isolation.

Activity

  1. added this to the v0.1.0 milestone on Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions