test(bench): define the complete S18-S26 qualification ladder

[DecisionNerd]

Objective

Define the complete Graph500 qualification ladder as declarative ReFrame/BenchExec profiles and a fail-closed controller, without executing provider scale runs from this issue.

Ladder

S18 -> S19 -> S20 -> S22 -> S24 -> S25 -> S26

• S18/S19 are lower qualification rungs, not developer-laptop benchmarks.
• S20 is projected only from adjacent completed S18/S19 evidence.
• S22 is separately gated by completed lower evidence.
• S26 is projected only from adjacent completed S24/S25 evidence.
• Every rung uses the same ordinary source -> ingest -> reopen/recount -> canonical queries -> export -> verify -> clean import -> reopen proof lifecycle.

Requirements

• Add distinct S18, S19, S20, S22, S24, S25, and S26 profiles.
• Pin generator identity, EF16, seed, public lifecycle phases, evidence schemas, limits, and headroom policy without provider resource IDs.
• Select only the first unexecuted rung; a failed or incomplete rung authorizes nothing larger.
• Project wall time, phase/process-tree RSS, retained/transient storage, logical/physical I/O, reader calls, and publication work from the latest required adjacent completed rungs.
• Treat sustained material RSS growth with edge count as an architectural refusal signal. Persistent size is expected to be disk-bound.
• Require measured provider throughput/capacity for physical I/O, reader calls, and publication work; missing metrics fail closed.
• Stage immutable executable copies and verify exact SHA-256 identities immediately before execution.
• Emit only closed, sanitized, schema-valid evidence and typed first-failure results.
• Keep profiles outside normal product CI. Tiny fixtures and dry-runs may run locally.
• The disk-constrained macOS controller host must not build or retain large Linux targets, Graph500 datasets, project copies, OCI layers, or scale evidence payloads.

Acceptance criteria

• Closed schemas and tiny fixtures validate all seven profiles, phase order, executable identities, selection, refusal, typed failures, and evidence sanitization.
• S24 and S25 are explicit executable profiles; S26 cannot be admitted without both.
• Conservative projection independently checks the four-hour rung limit, S20 4 GiB envelope, Fly 500 GB maximum with 15% storage reserve, measured I/O/reader/publication capacity, and bounded/plateauing RSS.
• The controller stages a selected profile and immutable executables, invokes BenchExec only after native-Linux admission, validates normalized output, and never provisions Fly itself.
• Missing ordinary-path metric receipts produce a typed refusal rather than inferred counters.
• No scale rung runs in normal CI or as issue-close evidence for this construction issue.

BDD completion scenario

Given a clean checkout, the complete seven-rung profile set, and fixture evidence
When the controller selects or projects the next rung
Then it chooses only the first authorized rung
And refuses absent metrics, non-adjacent projections, identity drift, resource-envelope violations, or prior failure.

Non-goals

Executing the real Fly ladder, provisioning provider resources, weakening canonical queries/durability/verification, benchmark-only engine hooks, laptop scale measurements, or claiming official Graph500 submission results.

Relationships

• Parent: epic(bench): externalize scale certification and add GDC benchmark suites #952
• Provider adapter: build(bench): provide the disposable Fly ladder adapter #958
• Real Fly ladder and S20/S26 evidence: test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900
• Ordinary staged-import repair: fix(api): route staged Parquet import through construction sessions #991

[DecisionNerd]

Qualification evidence diagnosis (2026-08-29)

The merged profile/schema/projection layer is not yet executable evidence for S18/S19:

• no current producer writes graphforge-progressive-qualification-rung-evidence metrics;
• ReFrame invokes the certification runner directly rather than placing the complete lifecycle under the admitted BenchExec process-tree authority;
• the runner discards ordinary gf stdout, so it cannot consume durable construction/storage/query receipts;
• reader_calls and publication_work_units have no current closed source-component definition;
• most importantly, the profiles ingest through gf import-session, whose current implementation decodes 8,192-row Parquet batches and calls scalar GraphWriter::create_node / create_edge for every row. That bypasses the 65,536-row append-only GraphConstructionSession repaired by fix(scale): use construction-budget Arrow batches for S20 ingest #979 and would recreate the wrong amplification at S18/S19.

Native blocker #991 now owns routing ordinary staged Parquet import through resumable one-generation construction and returning durable sanitized evidence. It is a sub-issue of #901 and blocks this issue.

After #991, this issue should assemble evidence from existing authorities rather than fabricate counts:

1. BenchExec: whole-process-tree wall time, peak RSS, and block I/O.
2. Durable construction receipt and ConstructionPhaseAttribution: application bytes/calls, artifacts, blocks, fsyncs, recovery, and transient allocation peak.
3. GraphForge::storage_attribution: reopened retained logical/allocated storage from authenticated inventories.
4. Ordinary query receipt: exact demand/reader counters emitted by the Rust execution path.

The validator must require every named component and retain the source vector used to derive aggregate reader/publication work. No S18/S19 execution is valid before #991 and this producer wiring are complete.

[DecisionNerd]

Completed as a construction issue, per this issue's explicit non-goal against using real scale runs as close evidence.

Authoritative merged evidence:

• PR test(bench): complete authoritative progressive rung assembly #994, squash merge c9dbf2e
• reviewed exact head 6eb7d97
• exact-head Repository Policy, Rust/Python/binding/native/Bazel/Concurrency Matrix, and CI Gate passed
• one final CodeRabbit review completed; all six findings independently dispositioned and all threads resolved
• closing references remained empty so provider-proof issues were not closed prematurely

Acceptance evidence:

• all seven S18/S19/S20/S22/S24/S25/S26 profiles validate
• S24/S25 are explicit profiles with distinct scale-specific identities
• full benchmark smoke passed: 54 Python controller tests, 12 certify tests, generator/smoke, formatting, and clippy
• controller stages immutable executables, requires native admission, consumes ordinary storage/query receipts, fails closed on missing/invalid metrics, and emits bounded sanitized typed evidence
• no real scale rung was executed for this construction close

Real provider execution remains tracked by #958/#900 and the open provider-proof issues.

[DecisionNerd]

Reopening after exact-tree launch audit found a construction acceptance gap at merged main c9dbf2e. The controller requires exactly one graphforge-lifecycle-storage/1 receipt in assemble_rung_evidence, but the contract appears only in benchmark documentation, the consumer/sanitizer, and synthetic tests. No checked-in profile action or ordinary gf command emits it. A real S18 would therefore terminate with ordinary_receipt_missing and cannot produce certifiable rung evidence. No scale run was started. The canonical construction issue remains the close gate until a production ordinary lifecycle path emits this closed receipt and an executable profile regression proves end-to-end assembly without fixture injection.