What
test/BaseTest.php parses --SKIPIF-- into $sections['SKIPIF'] like any other section, then never looks at it again.
- it is not in
UNSUPPORTED_SECTIONS, so it does not throw
grep -n "SKIPIF" test/BaseTest.php returns nothing — the key is never read
So every --SKIPIF-- block in the corpus is dead. It reads as authoritative gating and does nothing, which is this project's characteristic failure mode (AGENTS.md §4: a gate that reports without executing).
Measured at cc50fbb8f
|
|
cases carrying --SKIPIF-- |
671 |
| of those, executed |
455 |
| of those, failing in the committed baseline |
81 |
of those 81, whose SKIPIF evaluates to skip |
≥ 19 |
The 19 confirmed, with what their own guard says:
ext/odbc_prepare_fetch_roundtrip skip no odbc_prepare
ext/pgsql_basic skip no pg_connect (libpq FFI)
intl/intl_error_functions skip intl error functions not advertised
intl/msgfmt_create_unmatched_braces skip MessageFormatter withheld until extension_loaded('intl') (#19670)
language/lazy_ghost_basic skip ReflectionClass::newLazyGhost requires PHP 8.4+
language/lazy_ghost_zero_prop_isset skip ReflectionClass::newLazyGhost requires PHP 8.4+
language/lazy_object_ghost skip ReflectionClass::newLazyGhost requires PHP 8.4+
language/lazy_object_introspection skip lazy object introspection requires PHP 8.4+
language/lazy_object_proxy skip ReflectionClass::newLazyProxy requires PHP 8.4+
language/reflection_new_lazy_ghost skip ReflectionClass::newLazyGhost requires PHP 8.4+
stdlib/get_defined_functions_phantom_forward_profile skip requires PHP_COMPILER_PROFILE=8.4
stdlib/inotify_enum_operand skip inotify unavailable
stdlib/lazy_object_introspection_property_materialize skip ... requires PHP 8.4+
stdlib/mysqli_prepare_select skip no MYSQLI_TEST_DSN
stdlib/mysqli_real_connect skip no MYSQLI_TEST_DSN
stdlib/mysqli_refresh_connection_stats skip no MYSQLI_TEST_DSN
stdlib/mysqli_transaction_roundtrip skip no MYSQLI_TEST_DSN
stdlib/randomizer_getfloat_83 skip requires PHP_COMPILER_PROFILE >= 8.3
stdlib/socket_atmark skip socket_atmark
≥ because 59 of the 81 produced other output under my standalone evaluator (which runs the block from the case directory without the harness's env setup) and are inconclusive, not "would run". Only 3 were silent.
These are not compiler defects. no MYSQLI_TEST_DSN means the test needs a database nobody configured; requires PHP 8.4+ means the feature is not in the target profile. They are counted as failures because the harness ignores the case's own statement of when it is meaningful.
Why it matters beyond the count
VMTest::providePHPTests() has grown ~1,100 lines of hand-written str_contains($name, …) gating rules — the mechanism that does work. Cases that instead rely on --SKIPIF-- fall through it, and the two disagree:
language/interface_typed_const_inherit_ok guards on supportsInterfaceTypedConstants() (true at HEAD) while the feature it uses is gated by supportsTypedClassConstants() (false after 7236ecdb0). Both the compiler and host Zend 8.2.32 reject its body identically:
syntax error, unexpected identifier "X", expecting "="
So the compiler is correct and the case is mis-gated — yet it appears in the baseline as a failure, and appeared in the nightly as a regression.
This also bears on #24697 (intl/gmp release-scoping): part of the intl failure rate is cases whose own guard says the extension is unavailable.
Options
- Execute
--SKIPIF-- in BaseTest::parsePHPT/the runner, as php-src's run-tests.php does, and skip the case when it prints skip….
- If SKIPIF is deliberately unsupported, add it to
UNSUPPORTED_SECTIONS so the 671 files fail loudly instead of lying, and move their conditions into the provider.
Either is defensible. Silently ignoring it is not — the current state means 671 files document a contract the harness does not honour.
What
test/BaseTest.phpparses--SKIPIF--into$sections['SKIPIF']like any other section, then never looks at it again.UNSUPPORTED_SECTIONS, so it does not throwgrep -n "SKIPIF" test/BaseTest.phpreturns nothing — the key is never readSo every
--SKIPIF--block in the corpus is dead. It reads as authoritative gating and does nothing, which is this project's characteristic failure mode (AGENTS.md §4: a gate that reports without executing).Measured at
cc50fbb8f--SKIPIF--skipThe 19 confirmed, with what their own guard says:
≥because 59 of the 81 produced other output under my standalone evaluator (which runs the block from the case directory without the harness's env setup) and are inconclusive, not "would run". Only 3 were silent.These are not compiler defects.
no MYSQLI_TEST_DSNmeans the test needs a database nobody configured;requires PHP 8.4+means the feature is not in the target profile. They are counted as failures because the harness ignores the case's own statement of when it is meaningful.Why it matters beyond the count
VMTest::providePHPTests()has grown ~1,100 lines of hand-writtenstr_contains($name, …)gating rules — the mechanism that does work. Cases that instead rely on--SKIPIF--fall through it, and the two disagree:language/interface_typed_const_inherit_okguards onsupportsInterfaceTypedConstants()(true at HEAD) while the feature it uses is gated bysupportsTypedClassConstants()(false after7236ecdb0). Both the compiler and host Zend 8.2.32 reject its body identically:So the compiler is correct and the case is mis-gated — yet it appears in the baseline as a failure, and appeared in the nightly as a regression.
This also bears on #24697 (intl/gmp release-scoping): part of the intl failure rate is cases whose own guard says the extension is unavailable.
Options
--SKIPIF--inBaseTest::parsePHPT/the runner, as php-src'srun-tests.phpdoes, and skip the case when it printsskip….UNSUPPORTED_SECTIONSso the 671 files fail loudly instead of lying, and move their conditions into the provider.Either is defensible. Silently ignoring it is not — the current state means 671 files document a contract the harness does not honour.