CI: select the compliance suite; add a spine-chunk split-TU probe

[PurHur]

Two independent pieces, both about measuring things that were previously measured by hand and got the wrong answer.

1. compliance-nightly.yml takes a suite input (VMTest | JITTest)

The cron nightly still runs VMTest, the suite with a committed baseline. JITTest (~472 failures) has none, so --suite=JITTest --diff is expected to refuse until a collect run has been reviewed and landed by a human — that is the script doing its job, not a break.

schedule supplies no inputs, so every expression falls back to VMTest. The concurrency group is keyed on the suite too, so a JITTest collect cannot cancel a VMTest nightly on the same ref.

2. script/spine-chunk-probe.sh

Replaces the hand-built fixtures under build/micro/chunk/ used to test whether the spine can be split into separately-compiled TUs (RELEASE-PLAN Phase 3 item 9). This matters because the gen-1 bootstrap compiles all 6,519 spine files as one TU and has now failed four times purely on memory — OOM at 6, 24 and 48 GiB, and a fourth run pinned at a 12 GiB cap for 4h55m in direct reclaim (memory.events max=1426, oom_kill=0 — it survived by swapping, not by fitting). Capacity is not the constraint; unbounded growth in a single TU is.

The hand-built fixtures gave two wrong answers. The script encodes both.

They were not valid PHP programs. A bare require_once list over ext/<name>/ references classes declared elsewhere (ModuleAbstract, VmClassMethod, Func\Internal), so Zend fatals on all seven with rc=255. "The compiler failed to build this chunk" could not be distinguished from "this chunk is not a program" — which undercuts the premise of #24429's title. The generated fixture prepends the autoloader, and the Zend baseline is checked rather than assumed.

rc=0 plus a non-empty binary is not a pass. ext/ds compiles to an 18.5 MB binary that exits 0, while the compiler itself reports 61 method calls — including hexdec, substr, is_string, str_starts_with — lowered to a silent null because their class is not in the module (#579, #24227). Against the bare fixture that binary exits 0 where Zend exits 255. Silent wrong output is the characteristic failure of this project (AGENTS.md §3), so an unbound stub is a failure here, not a warning.

Measured at fd3b1fedc with PHP_COMPILER_SPINE_CHUNK=1 (#24496)

chunk	result
ds	compiles, 18.5 MB, 61 unbound stubs → HOLLOW
types	LLVMAbstract/Module.php exception, rc=255
msgpack	unimplemented TYPE_BITWISE binary op
sockets	Static call class must be a literal (lib/JIT.php:18097)
vm	same, at 202s
spl	SIGSEGV (rc=139) at 71s

So #24496 did what it claimed — the chunks get past the old Call to undefined method object::… abort — but no chunk is yet usable, and the blocker moved from "compile aborts" to "compiles, silently wrong", which is the more dangerous state.

One detail worth a follow-up: in the sockets/vm case Block::getOperand() returns null for both the class and name slots, so the class operand is absent rather than dynamic. The throw message describes a different condition than the one occurring.

Validation

Validated by making it fail: exit 1 on a real chunk failure, exit 1 on a bogus --ext, and the pass path exercised via ext/ds.

🤖 Generated with Claude Code

[PurHur]

Validated by running it, not by reading it

This PR has no checks — compiler-gate.yml triggers on lib/** and ext/**, and this touches neither. CLEAN here means "no checks configured" (CLAUDE.md §1), so naming the gate I actually ran:

Dispatched the changed workflow from this branch: run 30455848639 — mode=collect suite=JITTest.

• job names rendered as JITTest shard 0..23, not VMTest — the suite input reaches the matrix
• all 24 shards succeeded
• the collect step produced artifact JITTest-baseline-42361f4b… containing 489 failing names, header # shards: 24 failing: 489

So the suite plumbs through the matrix, the shard script, the artifact paths and the baseline collector. This matters because the last CI gate I added failed on arrival for want of exactly this check.

script/spine-chunk-probe.sh was validated by making it fail: exit 1 on a real chunk failure, exit 1 on a bogus --ext, and the pass path exercised via ext/ds (which is how the HOLLOW case was found in the first place).

Not landing a JITTest baseline from this run. 489 names from a single run is one measurement; the two-run protocol requires a second collect to quarantine what flips. Run 2 is dispatched. A single-run baseline was measured wrong in 9 places on VMTest (#24672).

[PurHur]

The executed-set change paid for itself immediately

Pushed two more commits and validated the whole thing end-to-end on real CI runs.

The gate had a hole, and the nightly walked into it

Nightly 30458065584 reported a FIXED list. Four of those cases do not execute at all at a634624db — verified by filtering for them directly on a dev box:

language/asymmetric_visibility_write
language/asymmetric_visibility_public_protected_set
language/static_asymmetric_visibility
language/match_typed_class_const

The prescribed response to a FIXED entry is "regenerate the baseline", which would have deleted them from tracking permanently. shard-compliance.sh now writes a .executed set per shard and the collector splits FIXED / DROPPED / NEWLY EXECUTED.

It degrades honestly: with no .executed files the heading becomes FIXED OR DROPPED — execution NOT verified and says not to regenerate from it. The first version of that code printed "executed, passing now" over a list whose execution had never been checked — caught by testing the degraded path rather than only the happy one.

First properly pinned pair, and the quarantine was wrong

Two complete 24-shard collects at the SAME commit a3f32662b on the same image — 30461221640 and 30461258793. Both ran to completion, neither cancelled the other, which is the concurrency fix working.

8,112 executed cases. The two runs disagreed about exactly one.

Verdict on all 26 quarantined names:

verdict	n	action
genuinely FLAKY	1	stdlib/hrtime_nanosecond_precision kept, issue filed (#24870)
stable FAIL	6	real failures the quarantine was HIDING — now baselined
stable pass	13	never flaky; removed
not executed	6	property_hook_* not yielded at this tree; removed

The six that were hidden include highlight_file_br, highlight_file_multiline_br and highlight_string_pre_wrapper — introduced by 0ae0f941c, which fixed four other highlight_* cases and broke these three. Quarantining them meant the gate could never report them again.

Baseline is now 428 names plus the first VMTest.executed at 8,112.

Gate I ran

This PR gets no CI checks (compiler-gate.yml triggers on lib/** and ext/**). Named explicitly:

• suite input: dispatched mode=collect suite=JITTest — jobs rendered as JITTest shard 0..23, 489 names collected
• JITTest twice at one SHA (42361f4be): 489 vs 489, zero flips
• VMTest twice at one SHA (a3f32662b): 428 vs 428 failing, 8,112 vs 8,112 executed, one flip
• baseline collected from run 2 diffs clean (exit 0) against run 1 — the run it was not built from
• compliance-baseline.sh three-way split validated on synthetic fixtures across all four paths
• spine-chunk-probe.sh validated by making it fail: exit 1 on a real failure, exit 1 on a bogus --ext