Skip to content

AOT: skip the corpus warmup when the committed prelink cache is current (#24302) - #24351

Merged
PurHur merged 1 commit into
masterfrom
fix/skip-warmup-when-prelink-current
Jul 28, 2026
Merged

AOT: skip the corpus warmup when the committed prelink cache is current (#24302)#24351
PurHur merged 1 commit into
masterfrom
fix/skip-warmup-when-prelink-current

Conversation

@PurHur

@PurHur PurHur commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Fixes #24302 — compiling <?php echo "hi\n"; on a clean checkout took ~9 minutes.

first compile, clean checkout:  517 s  ->  5 s
second compile:                   5 s     (unchanged)

Root cause

warmForUserAotBuild() decided whether to emit the entire helper corpus based on one thing:

$marker = self::coreMarkerPath();     // build/helper-runtime-cache/<prefix><fingerprint>.ok
if (is_file($marker)) { return; }
...
$rc = self::runWarmupCommand($cmd);   // shells out to emit-helper-runtime-object.php

coreMarkerPath() lives under cacheDir() = build/helper-runtime-cache, and that directory is
gitignored (.gitignore:29, 0 files tracked). So a clean checkout can never have the marker, and
the first user AOT build re-emitted the corpus regardless of whether prelinked/helper-runtime/
was present and current
.

That is also why #24321 did not move the number: refreshing the committed cache fixes what the
consumption path can use, while the warmup decision never looked at it. Measured 533 s before,
517 s after — ~16 s of ~530.

Found by profiling, not by guessing

My first hypothesis (stale committed fingerprint) was measured and refuted. A SIGALRM sampler on
the real bin/compile.php then collected only 5 ticks in 441 s, top path:

HelperRuntimeCache::runWarmupCommand < warmForUserAotBuild < run < php_compiler_cli_dispatch

The parent is blocked on a subprocess, so parent-side sampling sees almost nothing — the near-empty
profile was itself the signal.

The change

Skip the warmup when the committed per-arch cache exists, carries a core_fingerprint equal to the
live one, and actually contains units. The consumption path already reads prelinkedUnitsDir()
alongside unitsDir(), so in that state there is nothing to emit.

committedCacheIsCurrent() is deliberately conservative — missing manifest, unreadable JSON,
mismatched fingerprint, or an empty units directory all fall through to the warmup. A stale or
partial committed cache is still emitted rather than silently producing a build with missing helpers.

Gate

No CI on lib/, and AOT is informational in compiler-gate.yml, so this is a named local gate.
Branch vs a clean master checkout, run sequentially, both with build/helper-runtime-cache
removed first so the measurement is cold:

branch master
cold aot-smoke 8/8 8/8
VM --repeat 2 110/110, exit 0 110/110, exit 0
AOT --repeat 3 16 failing 16 failing
regressions (branch-only): NONE
fixed (master-only)      : NONE

One note on method

My first attempt at that comparison read master's sweep while it was still running (73 of 110
cases) and showed four phantom regressions — j05, j06, k02, m02 — purely because those cases
had not been reached yet. The table above is from the completed run. Worth flagging because a
partially-written results file looks exactly like a real regression set.

What this does not do

It does not make the corpus emit itself faster, and it does not help when the committed cache is
genuinely stale — that path still pays the full ~9 minutes, correctly. Keeping
prelinked/helper-runtime/ current is what makes the fast path available, which is what #24321 and
its release-readiness gate are for.

…nt (#24302)

Compiling <?php echo "hi\n"; on a clean checkout took ~9 minutes. Second compile: 5 seconds.

warmForUserAotBuild() decided whether to emit the entire helper corpus based on one thing — whether
a marker file exists:

    $marker = self::coreMarkerPath();     // build/helper-runtime-cache/<prefix><fingerprint>.ok
    if (is_file($marker)) { return; }
    ...
    $rc = self::runWarmupCommand($cmd);   // shells out to emit-helper-runtime-object.php

coreMarkerPath() lives under cacheDir() = build/helper-runtime-cache, which is gitignored
(.gitignore:29, 0 files tracked). A clean checkout can therefore NEVER have the marker, so the first
user AOT build re-emitted the whole corpus regardless of whether prelinked/helper-runtime/ was
present and current.

That is also why #24321 (refreshing the committed cache) did not help: it fixes what the CONSUMPTION
path can use, while the warmup decision never looked at it. Measured before 533s, after 517s — ~16s
of ~530.

Found by profiling, not by guessing: a SIGALRM sampler on the real bin/compile.php collected only 5
ticks in 441s, top path HelperRuntimeCache::runWarmupCommand < warmForUserAotBuild. The parent is
blocked on a subprocess, which is why parent-side sampling sees nothing — the near-empty profile was
itself the signal.

Fix: skip the warmup when the committed per-arch cache exists, carries a core_fingerprint equal to
the live one, and actually contains units. The consumption path already reads prelinkedUnitsDir()
alongside unitsDir(), so in that state there is nothing to emit and warming is pure waste.

  first compile, clean checkout:  517s -> 5s

committedCacheIsCurrent() is deliberately conservative: missing manifest, unreadable JSON,
mismatched fingerprint or an empty units directory all fall through to the warmup, so a stale or
partial committed cache is still emitted rather than silently producing a build with missing
helpers.

Gate (no CI on lib/, and AOT is informational in compiler-gate.yml), branch vs a clean master
checkout, run sequentially, both with build/helper-runtime-cache removed first so the measurement is
COLD:

  cold aot-smoke : 8/8 both sides
  VM  --repeat 2 : 110/110 both sides, exit 0
  AOT --repeat 3 : master 16 failing, branch 16 failing
                   regressions (branch-only): NONE
                   fixed (master-only)      : NONE

Note on method: my first attempt at that comparison read master's sweep while it was still running
(73 of 110 cases) and showed four phantom regressions — j05, j06, k02, m02 — purely because those
cases had not been reached yet. The numbers above are from the completed run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@PurHur
PurHur merged commit dac8065 into master Jul 28, 2026
1 check passed
PurHur added a commit that referenced this pull request Jul 28, 2026
…nt (#24302) (#24351)

Compiling <?php echo "hi\n"; on a clean checkout took ~9 minutes. Second compile: 5 seconds.

warmForUserAotBuild() decided whether to emit the entire helper corpus based on one thing — whether
a marker file exists:

    $marker = self::coreMarkerPath();     // build/helper-runtime-cache/<prefix><fingerprint>.ok
    if (is_file($marker)) { return; }
    ...
    $rc = self::runWarmupCommand($cmd);   // shells out to emit-helper-runtime-object.php

coreMarkerPath() lives under cacheDir() = build/helper-runtime-cache, which is gitignored
(.gitignore:29, 0 files tracked). A clean checkout can therefore NEVER have the marker, so the first
user AOT build re-emitted the whole corpus regardless of whether prelinked/helper-runtime/ was
present and current.

That is also why #24321 (refreshing the committed cache) did not help: it fixes what the CONSUMPTION
path can use, while the warmup decision never looked at it. Measured before 533s, after 517s — ~16s
of ~530.

Found by profiling, not by guessing: a SIGALRM sampler on the real bin/compile.php collected only 5
ticks in 441s, top path HelperRuntimeCache::runWarmupCommand < warmForUserAotBuild. The parent is
blocked on a subprocess, which is why parent-side sampling sees nothing — the near-empty profile was
itself the signal.

Fix: skip the warmup when the committed per-arch cache exists, carries a core_fingerprint equal to
the live one, and actually contains units. The consumption path already reads prelinkedUnitsDir()
alongside unitsDir(), so in that state there is nothing to emit and warming is pure waste.

  first compile, clean checkout:  517s -> 5s

committedCacheIsCurrent() is deliberately conservative: missing manifest, unreadable JSON,
mismatched fingerprint or an empty units directory all fall through to the warmup, so a stale or
partial committed cache is still emitted rather than silently producing a build with missing
helpers.

Gate (no CI on lib/, and AOT is informational in compiler-gate.yml), branch vs a clean master
checkout, run sequentially, both with build/helper-runtime-cache removed first so the measurement is
COLD:

  cold aot-smoke : 8/8 both sides
  VM  --repeat 2 : 110/110 both sides, exit 0
  AOT --repeat 3 : master 16 failing, branch 16 failing
                   regressions (branch-only): NONE
                   fixed (master-only)      : NONE

Note on method: my first attempt at that comparison read master's sweep while it was still running
(73 of 110 cases) and showed four phantom regressions — j05, j06, k02, m02 — purely because those
cases had not been reached yet. The numbers above are from the completed run.

Co-authored-by: PurHur <tedyyyyy@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
PurHur added a commit that referenced this pull request Jul 28, 2026
…iterion (#24302) (#24361)

Until #24351 this took ~9 minutes; it is now ~5 seconds. Nothing was watching that number, and this
repo's history is mostly wins that decayed quietly, so this makes it a gate.

script/cold-build-check.sh points PHP_COMPILER_HELPER_RUNTIME_CACHE_DIR at a fresh temp directory,
so it measures the cold path a new user gets WITHOUT destroying the developer's real cache, and
bounds the run with a timeout so a regression surfaces as a fast failure rather than a nine-minute
wait. Budget COLD_BUILD_MAX_SECONDS, default 120s — a budget, not a benchmark: far enough above the
measured 5s to absorb a slow CI runner while still catching a return to ~500s.

Validated in both directions, which is the part that matters (AGENTS.md §4):

  master (fix present)   ok — 5s, exit 0
  a279902 (pre-fix)    FAIL (timeout) after 120s, exit 1

Wired into release-readiness next to the existing helper-runtime-prelink gate. They are NOT the same
check and both are needed:

  helper-runtime-prelink --strict   the committed cache is CURRENT   (precondition)
  cold-build-check                  a clean checkout is actually FAST (outcome)

At a279902 the committed cache was current and the fingerprints matched, so the prelink gate would
have passed while a clean-checkout build took ~9 minutes — the warmup never consulted the cache. An
outcome gate catches that; a precondition gate structurally cannot.

Also emits --json for machine consumption, matching how release-readiness reports gen0_provenance.

Co-authored-by: PurHur <tedyyyyy@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
PurHur added a commit that referenced this pull request Jul 28, 2026
* json_decode: native HT runtime materialization for assoc arrays (#24137)

Replace int-wire JsonDecodeJitHelper with tag-dispatched native __hashtable__*
building via phpc_native_ht_* (ParseStr #13827 shape), wired through a single
__compiler_json_decode bridge with ensureNativeHtInternalProxies before NestedJIT.

AOT green for runtime literal JSON strings; json_encode→decode roundtrip still
blocked (encode boxes __string__* in __value__* — separate issue path).

Co-authored-by: Cursor <cursoragent@cursor.com>

* json_decode: raise shrink line budget; add encode→decode AOT diag (#24137)

StringJsonDecode grew to 409 lines with native HT materialization; bump the
shrink ceiling. Add a small AOT diag repro for the NestedJIT heap-string
param gap that still blocks json_encode→json_decode.

Co-authored-by: Cursor <cursoragent@cursor.com>

* AOT json_decode: box assoc HT in __value__* (#24137)

Raw __hashtable__* cast to __value__* was misread as int/NULL after
NestedJIT native HT fill. Peer #24298 boxHashtableValue shape.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Foundation: refresh helper-runtime prelink (35 stale → 0) (#24336) (#24349)

DOM and introspection helper units drifted after #24321; release-readiness
--strict failed with 258 fresh / 35 stale. Re-emit via --prelink in the
pinned Docker env so the committed cache matches live fingerprints again.

Co-authored-by: PurHur <PurHur@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* DOM: getNamedItem matches Attr local name, not QName (#24332) (#24350)

php-src namednodemap.c uses xmlHasProp (local name) for attribute maps.
Wire JIT/AOT instance-method proxies for getNamedItem(NS) so :object
attributes receivers bind the same path.

Co-authored-by: PurHur <PurHur@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* AOT: skip the corpus warmup when the committed prelink cache is current (#24302) (#24351)

Compiling <?php echo "hi\n"; on a clean checkout took ~9 minutes. Second compile: 5 seconds.

warmForUserAotBuild() decided whether to emit the entire helper corpus based on one thing — whether
a marker file exists:

    $marker = self::coreMarkerPath();     // build/helper-runtime-cache/<prefix><fingerprint>.ok
    if (is_file($marker)) { return; }
    ...
    $rc = self::runWarmupCommand($cmd);   // shells out to emit-helper-runtime-object.php

coreMarkerPath() lives under cacheDir() = build/helper-runtime-cache, which is gitignored
(.gitignore:29, 0 files tracked). A clean checkout can therefore NEVER have the marker, so the first
user AOT build re-emitted the whole corpus regardless of whether prelinked/helper-runtime/ was
present and current.

That is also why #24321 (refreshing the committed cache) did not help: it fixes what the CONSUMPTION
path can use, while the warmup decision never looked at it. Measured before 533s, after 517s — ~16s
of ~530.

Found by profiling, not by guessing: a SIGALRM sampler on the real bin/compile.php collected only 5
ticks in 441s, top path HelperRuntimeCache::runWarmupCommand < warmForUserAotBuild. The parent is
blocked on a subprocess, which is why parent-side sampling sees nothing — the near-empty profile was
itself the signal.

Fix: skip the warmup when the committed per-arch cache exists, carries a core_fingerprint equal to
the live one, and actually contains units. The consumption path already reads prelinkedUnitsDir()
alongside unitsDir(), so in that state there is nothing to emit and warming is pure waste.

  first compile, clean checkout:  517s -> 5s

committedCacheIsCurrent() is deliberately conservative: missing manifest, unreadable JSON,
mismatched fingerprint or an empty units directory all fall through to the warmup, so a stale or
partial committed cache is still emitted rather than silently producing a build with missing
helpers.

Gate (no CI on lib/, and AOT is informational in compiler-gate.yml), branch vs a clean master
checkout, run sequentially, both with build/helper-runtime-cache removed first so the measurement is
COLD:

  cold aot-smoke : 8/8 both sides
  VM  --repeat 2 : 110/110 both sides, exit 0
  AOT --repeat 3 : master 16 failing, branch 16 failing
                   regressions (branch-only): NONE
                   fixed (master-only)      : NONE

Note on method: my first attempt at that comparison read master's sweep while it was still running
(73 of 110 cases) and showed four phantom regressions — j05, j06, k02, m02 — purely because those
cases had not been reached yet. The numbers above are from the completed run.

Co-authored-by: PurHur <tedyyyyy@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Stdlib: ftp_connect Reflection/named args use Zend hostname (#23644) (#24352)

InternalArgInfo still advertises host; override BuiltinParamNames so
Reflection and named hostname:/port:/timeout: match ext/ftp/ftp.stub.php.

Co-authored-by: PurHur <PurHur@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Language: UnhandledMatchError formats variable subjects (#24329) (#24353)

Snapshot the match subject into a fresh temp before the message helper so
ARG_SEND cannot bind the null match-result seed. Also refuse remapping
unhandled-match helpers / named CVs onto that seed.

Co-authored-by: PurHur <PurHur@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* Language: preserve bound closure calledClass across CFG edges (#24335) (#24354)

Closure::bind scope was lost on ?? / if / try branch frames, so private
static reads threw even when direct single-block access worked.

Co-authored-by: PurHur <PurHur@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

* AOT: persist disableRefcount typeinfo store (#24137)

compile{} expand of `$ref.typeinfo = …` discarded insertValue (#24226 sibling),
so disableRefcount was a no-op and NestedJIT heap-string delref still freed
payloads. Hand-store the cleared REFCOUNTED bit; pin json_decode bridge copies.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Stdlib: AOT json_decode assoc via decodeInto + LLVM tag peek (#24137)

NestedJIT cannot use VmJsonFormat or return HT pointers; bridge allocates
the HT and peeks the first JSON byte in LLVM (resultTag ARG_RECV emptied
string formals). Skip NestedJIT string ARG_RECV rebind that UAF'd payload
bytes. Parse into phpc_native_ht_* with ord-based ints and substr slices
for nested arrays.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: PurHur <PurHur@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: PurHur <tedyyyyy@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
PurHur added a commit that referenced this pull request Jul 29, 2026
The plan still reads as if nothing has landed. Adds a status table (P1.1 CI done, P1.3 suites
runnable, P1.4 mechanism-only, P2.5 four increments in, P3.9 blocked with root cause mapped) and the
three corrections measurement forced on it:

  - Phase 2.5 buys nothing for user binary size — unused extensions are already never linked
    (76-ext vs 6-ext hello world differs by 360 bytes)
  - the plan's '~20 min sharded' assumes parallel runners; one box means ~2h sequential
  - 'clone and compile hello world' was ~9 minutes, not seconds, until #24351

Also records the pattern behind nearly every defect found this week: a check that reported success
without executing. Six variants, three of them in code I wrote. None was a wrong answer; all were
absent answers wearing a clean result.

Docs only. NOT merging while the clean baseline run is in flight — merging requires a pull, which is
the hazard documented in #24537.

Co-authored-by: PurHur <tedyyyyy@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@PurHur
PurHur deleted the fix/skip-warmup-when-prelink-current branch July 29, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Release blocker: committed helper-runtime cache is stale, so a clean checkout cannot compile hello world in 2 minutes

2 participants