Self-host M4: gen-2 native driver segfaults compiling full spine (bootstrap-loop-gen2-recompile-spine exit 139)

[PurHur]

Milestone

M4 (bootstrap loop / gen-2→gen-3 spine) — parent #1492.

Problem

make bootstrap-loop-probe is blocked by a native segfault while the gen-2 AOT driver recompiles the full spine (test/selfhost/compiler_lib_spine_smoke/main.php). The crashing native binary produces no stdout/stderr; the wrapper only sees exit 139.

This is compiler/runtime correctness work on the self-host path: gen-2 must compile inventory-scale PHP without crashing.

Scope (start here)

• Repro entry: test/selfhost/compiler_lib_spine_smoke/main.php
• AOT emit/link pipeline: lib/AOT/* + runtime floor lib/AOT/runtime/*
• LLVM lowering: lib/JIT.php, lib/JIT/*
• Self-host AOT stubs (PHP_COMPILER_SELFHOST_AOT) that may be masking invalid IR/pointers on larger bundles

Repro (today on master)

Host preflight (do not run docker info inside the docker-exec container):

docker info >/dev/null
./script/docker-exec.sh -- bash -lc 'make bootstrap-loop-probe'

First red sub-step (direct repro):

docker info >/dev/null
./script/docker-exec.sh -- bash -lc 'bash -x ./script/bootstrap-loop-gen2-recompile-spine.sh'

Observed symptoms:

• make bootstrap-loop-probe:
  • bootstrap-loop-gen2-recompile-spine: gen-2 native spine compile failed (exit 139)
• bash -x shows the crashing command is:

/compiler/build/bin-compile-aot -o /compiler/build/bootstrap-loop-gen3-full-spine /compiler/test/selfhost/compiler_lib_spine_smoke/main.php

…and the captured output is empty (out="", code=139).

Done when

• ./script/docker-exec.sh -- bash -lc 'make bootstrap-loop-probe' exits 0 on clean master
• ./script/docker-exec.sh -- bash -lc './script/bootstrap-loop-gen2-recompile-spine.sh' exits 0 and prints an expected compile OK line
• Root cause summarized and linked to the fixing PR

Suggested labels

If desired for filtering: self-host, bootstrap, IMPORTANT

[PurHur]

Re-tested on current in Docker tar-fallback mode and could not reproduce the exit 139.

Repro commands:

• ./script/bootstrap-loop-gen2-recompile-spine.sh
  compile_smoke_m3_emit: compile OK -> /compiler/build/bootstrap-loop-gen3-full-spine
  bootstrap-loop-gen2-recompile-spine: OK gen-2=/compiler/build/bin-compile-aot gen-3=/compiler/build/bootstrap-loop-gen3-full-spine (726/726 spine, argv -o)
• ./script/bootstrap-loop-probe.sh
  === M4 bootstrap-loop probe (M4: Bootstrap loop — native binary rebuilds next compiler revision #1498) ===
  mode: full (M3 HelloWorld strict before gen-1; gen-1→gen-2 + gen-2→gen-3 spine + full-revision per Self-host M4: bootstrap-loop-gen1-link.sh defaults native gen-2 emit (no Makefile env) #2611/Self-host M5: bin/compile.php AOT sidecar segfault — missing .m3_bin_compile_aot_blob at emit-helper link #2697/Self-host M4: full revision rebuild — native argv bin/compile.php compiles compiler tree #2880)

Exit codes: 0=green gate | 1=hard failure | 2=LLVM skip or M4 gen-2/gen-3 blocked | 3=reserved

==> lint bootstrap_loop_smoke bundle entry
==> lint bootstrap_loop_smoke compile driver
==> M0 self-host native link (compiler_minimal bundle)
bootstrap-selfhost-link: OK /compiler/build/selfhost
==> M2 lib spine smoke (native link + run)
bootstrap-selfhost-lib-spine-smoke-link: OK /compiler/build/selfhost-lib-spine-smoke
==> M3 HelloWorld probe (Makefile parity — strict native emit, #2612)
bootstrap-selfhost-helloworld-probe: OK emit_path=native /compiler/build/selfhost-helloworld-emit -> /compiler/build/helloworld-aot

Prerequisites OK through M3 HelloWorld (strict; same env as Makefile bootstrap-selfhost-helloworld).

==> M4 gen-1 link + gen-2 compile attempt (script defaults native emit when LLVM present; #2611)
bootstrap-loop-gen1-link: OK emit_path=native gen-1=/compiler/build/bootstrap-loop-gen1 gen-2=/compiler/build/bootstrap-loop-gen2
bootstrap-loop-probe: gen-2 native emit OK (incremental M4 slice)

==> M4 exit status (M3 HelloWorld strict already verified above)
bootstrap-loop-probe: M4 gen-1→gen-2 native slice OK

==> M4 gen-2→gen-3 spine recompile (726/726, #2697)
bootstrap-loop-gen2-recompile-spine: OK gen-2=/compiler/build/bin-compile-aot gen-3=/compiler/build/bootstrap-loop-gen3-full-spine (726/726 spine, argv -o)
==> M4 full-revision argv probe (gen-2 bin/compile.php → gen-3, #2880/#2898)
bootstrap-selfhost-full-revision-probe: OK gen-2=/compiler/build/bin-compile-aot gen-3=/compiler/build/bootstrap-full-revision-gen3-compile emit_path=native (argv full revision #2880)
bootstrap-loop-probe: M4 full ladder OK — gen-1→gen-2 native + gen-2→gen-3 spine + full-revision (exit 0)

Both now complete successfully (gen-2 recompiles and the full M4 ladder exits 0).

Minor follow-up merged: #2942 makes print the spine ratio dynamically (now ) instead of a hardcoded , to keep the M4 log consistent with the M2 ratio SSOT.

[PurHur]

Re-tested on current master in Docker tar-fallback mode and could not reproduce the exit 139.

Repro commands:

• ./script/docker-exec.sh -- bash -lc 'make bootstrap-loop-gen2-recompile-spine'
• ./script/docker-exec.sh -- bash -lc 'make bootstrap-loop-probe'

Both now complete successfully (gen-2 build/bin-compile-aot recompiles compiler_lib_spine_smoke and the full M4 ladder exits 0).

Minor follow-up merged: #2942 makes bootstrap-loop-gen2-recompile-spine print the spine ratio dynamically (now 726/726) instead of a hardcoded 725/725, to keep the M4 log consistent with the M2 ratio SSOT.

[PurHur]

Closing as resolved: unable to reproduce on current master; bootstrap-loop-probe is green in Docker tar-fallback repro.

If this resurfaces, please paste the exact build/bin-compile-aot invocation and whether it was bind-mount or tar-fallback mode, plus any dmesg/core output.