Epic: first-try proof rate on fresh estates -- trials, input generation, the attempts-to-proof histogram

[squid-protocol]

CardDemo and CBSA are development data now: the tickets, generator and harness were fixed against them, so a success there says little about the next estate. The number worth publishing is first-try success on estates the system has never seen.

Protocol

1. Freeze the system: the engine commit and the porting rules a trial runs under. "First try" always means first try at a named version.
2. Take a fresh estate (never scanned by any development corpus). Scan it, generate, and for each eligible program run the porting loop once and prove the port. That's attempt 1.
3. Iterate to identical output, or declare the program out of scope. Each attempt carries its cause:
   • model: retry, same system;
   • ticket/rules, generator, engine facts, harness: a fix on our side, with its commit;
   • out of scope: e.g. a CICS command the stub does not model.
4. The estate joins the development set for good. Its later numbers stay on the chart, but never count as first-try again.

Output

• A population histogram per program:
  • bins: proven at attempt 1, 2, 3, 4+, and not proven (split by cause);
  • coloured by estate, faceted by engine version.
• A second view: fixes per estate, split into ours and the model's.
• Caveats stated on the chart:
  • public repos may be in a model's training data (only private estates escape that);
  • eligibility is declared up front, so "out of scope" cannot hide failures.

Parts

• Input generation from record layouts: any estate becomes testable, with edge cases (it rarely ships data)
• Trial record + trial start / trial report + the histogram
• Two or three fresh public batch estates through the protocol

Builds on #3753 (porting loop), #3624 / #3754 (equivalence harness) and tests/cobol_mainframe/field_testing.json (the fresh-estate counters and stopping rule).

🤖 Generated with Claude Code