Performance: benchmark suite v2 — the benchmark-game set plus web-request and real-app workloads, per-commit history in JSON, a public chart page, and `bench-gate` extended to all of it (benchmarks/, script/bench-gate.php, docs/pages/bench.html)

[PurHur]

Category

Foundation: · performance program, measurement half · child of #36379

Why

benchmarks/ holds 7 programs from 2015 (fibo, Ack, mandelbrot, simple) and benchmarks/README.md's table is dated 2026-07-27 — it was not regenerated after wave 1 changed every number. script/bench-gate.sh (#36196) compares ratios for those 7 only. Nothing measures strings, arrays, objects, regex, JSON, templates, or a web request — the shapes the audit found 500x off. A compiler's performance claim is only as good as its benchmark suite.

Deliverable

• benchmarks/v2/: nbody, spectral-norm, fannkuch-redux, binary-trees, fasta, k-nucleotide, regex-redux, json-roundtrip (encode/decode 10 MB), template-render (string building, 100k rows), assoc-heavy (100k string keys, lookups, array_*), object-graph (1M method calls + property access), sort-mixed, str-builder (.=, implode, sprintf), closure-heavy, exceptions (10k throw/catch), plus web-request (MiniWebApp / and /api/status via phpc fcgi, 2,000 requests, req/s and p99 vs php-fpm in the same image).
• Each program deterministic, output-verified against Zend before timing; VM / JIT / AOT columns, ratio to Zend, best-of-N with a per-run cap (no hangs).
• benchmarks/history/<sha>.json appended by the nightly; docs/pages/bench.html renders the ratio history per program (sparkline); benchmarks/README.md table generated from the latest JSON only.
• bench-gate baseline = the JSON of the last release tag; regression threshold 20 % per program, 10 % on the geometric mean.

Done when

• 16 programs committed with verified outputs; PHP_8_2=$(command -v php) php script/bench.php --v2 regenerates the table in < 15 min
• Nightly history + chart page live; bench-gate covers v2 and is red on a deliberate 2x slowdown
• benchmarks/README.md no longer contains any hand-typed number

[PurHur]

claim: implementer-worker-lane-b — starting this run (lane B tie-break #36385 % 3 == 1; lane overflow: #36143–#36148 closed; #36387 claimed by lane A)

[PurHur]

Merged #36459 (Part of #36385)

Verification:

• ./script/bench-gate.sh --v2 → OK (5-case AOT subset, 20% ratio band)
• Deliberate ~2× baseline poison → exit 1 (call-heavy.ratio +177%)
• php script/bench.php --v2 → 16 programs, RESULTS.json + history JSON
• ./script/aot-smoke.sh → 9/9

Still open on #36385: web-request column, live chart, scrub legacy README prose numbers.

[PurHur]

claim: implementer-worker-lane-b — continuing this run (lane B tie-break #36385 % 3 == 1). Next: web-request column + chart page + scrub hand-typed README numbers after #36459.

[PurHur]

Merged #36462 (Part of #36385)

Landed:

• script/bench-web-request.php — MiniWebApp home + api/status req/s (Zend builtin vs phpc serve; AOT/php-fpm n/a with reasons)
• script/generate-bench-chart.php — live docs/pages/bench.html sparklines from benchmarks/history/*.json + RESULTS
• Scrubbed hand-typed speedup tables from benchmarks/README.md

Verify:

• BenchGateTest 8/8
• bench-gate.sh --v2 OK
• aot-smoke 9/9
• web-request @200: zend ~391 req/s, phpc_serve ~22 req/s

Still open on #36385: nightly history publisher, php-fpm column, 2000-req soak / live chart vs php-fpm in docs, full --v2 table refresh in <15 min on a quiet box.

[PurHur]

claim: implementer-worker-lane-b — starting this run (lane B tie-break #36385 % 3 == 1; prior claim >48h). Next: finish remaining #36385 deliverables (web-request column / chart / README scrub) or next v2 bench programs after #36462.

[PurHur]

note: #36385 deliverables largely landed (#36459/#36462). Pivoted this run to Part of #36403 (VM Concern extract / size-budget ratchet) — lane B.

[PurHur]

claim: implementer-worker-lane-b — starting this run (lane B tie-break #36385 % 3 == 1; lane overflow: #36143–#36148 / north-star children closed; #36387 has open PR #37237 claimed <48h)