Skip to content

Foundation: bench-gate catches AOT performance regressions (#36196) - #36300

Merged
PurHur merged 1 commit into
masterfrom
agent/issue-36196-bench-gate
Sep 2, 2026
Merged

Foundation: bench-gate catches AOT performance regressions (#36196)#36300
PurHur merged 1 commit into
masterfrom
agent/issue-36196-bench-gate

Conversation

@PurHur

@PurHur PurHur commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds script/bench-gate.sh + script/bench-gate.php — compares AOT/Zend wall-time ratios (best-of-3, same job) and LLVM IR line counts against benchmarks/BASELINE.json
  • Commits baseline with ratio targets from benchmarks/README.md (2026-07-27 table); IR counts from pinned php-compiler:22.04-dev
  • Wires as tier 3 in compiler-gate.yml for lib/JIT/**, lib/AOT/**, ext/standard/** path triggers

Gate behavior (intentionally red)

The gate fails on today's master for fibo(30) — proof it bites before the typed-int boxing fix (#36189 / #31980):

bench-gate: PERFORMANCE REGRESSION
  fibo(30).ratio: 0.3013 exceeds baseline 0.1687 by 78.6% (limit +30%)
  mandelbrot.ratio: 118.2140 exceeds baseline 1.1192 by 10462.0% (limit +30%)

mandelbrot also regressed (separate issue). Gate goes green after perf fixes + --update.

Test plan

  • ./script/phpunit.sh --filter BenchGateTest — 4/4 OK
  • make dev-verify-fast — OK (91s)
  • ./script/bench-gate.sh — exit 1, fibo(30) ratio regression reported (as designed)
  • make north-star5-verify-fast — not run (tooling-only change)

Closes #36196

Made with Cursor

Introduces script/bench-gate.sh comparing AOT/Zend wall-time ratios and LLVM
IR size against benchmarks/BASELINE.json. Gate is intentionally red on
fibo(30) until typed-int boxing is fixed; wired as compiler-gate tier 3.

Co-authored-by: Cursor <cursoragent@cursor.com>
@PurHur
PurHur merged commit dbe788a into master Sep 2, 2026
1 check failed
@PurHur
PurHur deleted the agent/issue-36196-bench-gate branch September 2, 2026 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant