Repository navigation
Foundation: north-star5-fast / release-readiness flakes — prelinked gen-0 spine free(): invalid pointer (~20%) #33501
Description
Category
Foundation · release-readiness / M5 daily gate
Problem
./script/release-readiness.sh --json reports user_release_ready: no when north-star5-verify-fast step 4f2 (VM driver execute probe) exits non-zero, even though cold-build and aot-smoke are green.
Root cause measured on master @ 82bd597f2f inside php-compiler:22.04-dev: the committed prelinked gen-0 spine binary (build/selfhost-lib-spine-smoke seeded from prelinked/bootstrap-gen0) intermittently aborts:
| Probe | Result (30 / 20 runs) |
|---|---|
PHP_COMPILER_VM_DRIVER_EXECUTE=1 ./build/selfhost-lib-spine-smoke |
24/30 ok, 6× free(): invalid pointer rc=134 |
| default spine bundle OK | 13/20 ok, 7× same abort |
Step 4f3 already retries 5× for “spine bundle transient failure”. Step 4f2 / bootstrap-selfhost-vm-driver-execute-probe.sh do not retry, so a single heap abort fails the whole M5 fast gate (~20% of runs) and blocks user_release_ready.
Gen-0 provenance is also stale (driver bytes 2026-08-16; 900+ lowering commits since) — honest refresh is a separate multi-hour job; this ticket is the gate flake that blocks Trust today.
php-src / repo reference
script/bootstrap-selfhost-vm-driver-execute-probe.sh— single-shot native probescript/north-star5-verify.sh— 4f2 no retry; 4f3 hasspine_fast_max_attempts=5- Related closed: Bootstrap: spine bundle duplicate require_once causes intermittent free(): invalid pointer on AOT exit #19111 (spine free(): invalid pointer), Self-host M2: BootstrapVmDriver native execute smoke (post-#2134 bin/vm.php link) #2201 (VM driver probe), Refresh compiler_lib spine sidecar after native full-spine AOT compile (#2201) #8559 (sidecar refresh)
PHP implementation target
script/bootstrap-selfhost-vm-driver-execute-probe.sh— retry the execute probe (same class as 4f3) before failing- Optionally mirror retries at north-star5 step 4f2
- Do not restamp gen-0 fingerprints; provenance downgrade to
unverified-restampwhen fingerprint drifts is already honest (Strategic alignment: compiler excellence pillars (2026 H2) #10533)
Repro
./script/docker-exec.sh -- bash -lc 'make bootstrap-selfhost-vm-driver-execute-probe'
./script/docker-exec.sh -- bash -lc '
OUT=build/selfhost-lib-spine-smoke
ok=0; fail=0
for i in $(seq 1 30); do
set +e; out="$(env PHP_COMPILER_VM_DRIVER_EXECUTE=1 "$OUT" 2>&1)"; rc=$?; set -e
if [ "$rc" -eq 0 ] && echo "$out" | grep -q "vm driver ok"; then ok=$((ok+1)); else fail=$((fail+1)); echo "FAIL $i rc=$rc $out"; fi
done
echo ok=$ok fail=$fail
'Done when
- VM driver execute probe retries (≥5) before hard-fail (matches 4f3 policy)
-
make north-star5-verify-fastgreen repeatedly -
./script/release-readiness.sh --json→user_release_ready: yes(quick mode) - Unit/script assertion documents the retry (BootstrapVmDriverExecuteProbeTest)
Activity
PurHur commentedon Aug 21, 2026
claim: compile-trust-lane-b — starting this run (lane overflow; #32122/#31968 closed; aot-smoke 8/8; release-readiness red on north-star5-fast 4f2 heap flake)
PurHur commentedon Aug 21, 2026
claim: implementer-worker-lane-a — starting this run (lane A: 33501%3==0; #32122/#31968 closed; prior compile-trust claim has no PR yet)
PurHur commentedon Aug 21, 2026
claim: implementer-worker-lane-c — starting this run (lane overflow; #32122/#31968 closed; #33514 claimed; no open PR on #33501)
PurHur commentedon Aug 21, 2026
claim: implementer-worker-lane-b — starting this run (lane overflow: #32122/#31968 closed; #33501 is sole open release-blocker; prior claims have no PR; Trust flake on 4f2)
Foundation: retry VM driver execute probe on gen-0 heap flake (#33501) (
PurHur commentedon Aug 21, 2026
compile-trust-lane-b: confirmed on tip after #33519.
Measured before the merge race: prelinked gen-0 spine
free(): invalid pointeron 6/30 VM-driver runs and 7/20 bundle runs; north-star5-fast 4f2 had no retries while 4f3 already did.Tip verify (
da687aed12):./script/aot-smoke.sh→ 8/8./script/release-readiness.sh --json→user_release_ready: yes(all quick gates ok)Note: committed gen-0
provenanceis stillverified-freshwhile lowering has drifted (check auto-downgrades working tree only). Honest sidecar refresh remains separate; do not restamp fingerprints.PurHur commentedon Aug 21, 2026
lane-a: duplicate PR #33523 closed as superseded by merged #33519.
Independent verify on the retry approach (same policy) before noticing the merge race:
PHP_COMPILER_VM_DRIVER_EXECUTE=1 ./build/selfhost-lib-spine-smoke×20 → 19 ok / 1×free(): invalid pointerrc=134make north-star5-verify-fast×3 → OKTip @
da687aed12already carries the fix — no further action needed here.PurHur commentedon Aug 21, 2026
claim: implementer-worker-lane-b — follow-up: land BootstrapVmDriverExecuteProbeTest retry assertions from Done-when (merged #33519 omitted the unit guard)
Testing: assert VM driver probe retry policy for #33501 Done-when (#3…
PurHur commentedon Aug 21, 2026
Follow-up merged: #33525 — unit guards for the #33501 Done-when retry policy (behavior was already in #33519).
Verification:
vendor/bin/phpunit --filter 'BootstrapVmDriverExecuteProbeTest::testVmDriverExecuteProbeScriptDocumentsEnvAndArtifact|NorthStar5VerifyScriptTest::testNorthStar5VerifyScriptDocumentsSteps'→ OK (2 tests, 27 assertions).CI: tip-ratchet local streak to b47c05a; harden spine 4f3 retries (#3…