Repository navigation
Release plan: correct the intl/gmp rates for the ignored SKIPIF guards - #24918
Merged
Merged
Conversation
The Phase 1 numbers count cases the corpus itself declares inapplicable. --SKIPIF-- was parsed into $sections['SKIPIF'] by test/BaseTest.php and then never read, and it was not in UNSUPPORTED_SECTIONS either, so it did not throw. Every skip guard in the corpus was dead (#24888, fixed by #24897). Measured at cc50fbb: 671 cases carry a guard, 455 execute, 81 are FAILING in the committed baseline, and at least 19 of those carry a guard that evaluates to "skip" — no MYSQLI_TEST_DSN, no pg_connect (libpq FFI), requires PHP 8.4+, inotify unavailable. For the scoping decision in #24697 specifically: of 56 failing intl cases, 46 carry an ignored guard, but only 2 are CONFIRMED skips. The other 44 were inconclusive under a standalone evaluator — recorded as inconclusive, not as "would run". So the honest intl number may fall by anywhere between 2 and 46, and the 88.9% rate cannot be relied on until a collect is taken with SKIPIF honoured. gmp is unaffected: 6 failing, none of them skip cases. Also generalises "the pattern worth carrying forward" with the three failures found today, two of them in this plan's own tooling: the dead SKIPIF, the quarantine that hid 6 real failures by comparing two collects 15 commits apart, and a baseline of failing names that could not distinguish "fixed" from "no longer run". The through-line is sharper than the original wording: it is not enough for a check to run — it must be impossible to satisfy by ABSENCE. A case vanishing, a guard never evaluated, a set never compared all produce the same clean output as success. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Phase 1 numbers count cases the corpus itself declares inapplicable.
--SKIPIF--was parsed into$sections['SKIPIF']bytest/BaseTest.phpand then never read — and it was not inUNSUPPORTED_SECTIONSeither, so it did not throw. Every skip guard in the corpus was dead (#24888, fixed by #24897).Measured at
cc50fbb8f--SKIPIF--skipno MYSQLI_TEST_DSN,no pg_connect (libpq FFI),requires PHP 8.4+,inotify unavailable— not compiler defects.Effect on the #24697 scoping decision
Of 56 failing
intlcases, 46 carry an ignored guard, but only 2 are confirmed skips. The other 44 were inconclusive under a standalone evaluator that runs the block outside the harness env — recorded as inconclusive, not as "would run".So the honest intl number may fall by anywhere between 2 and 46, and the 88.9 % rate cannot be relied on until a collect is taken with SKIPIF honoured. Scoping intl out, or committing to fix 56 cases, would both be priced off a contaminated figure.
gmpis unaffected: 6 failing, none of them skip cases.Also sharpens "the pattern worth carrying forward"
Three failures of the same shape landed today, two of them in this plan's own tooling:
--SKIPIF--— 671 cases stated when they were inapplicable and the harness discarded every statement, while ~1,100 lines of hand-writtenstr_contains($name, …)rules grew inVMTest.phpto do the same jobThe through-line is sharper than the original wording: it is not enough for a check to run. It must be impossible to satisfy by absence. A case vanishing, a guard never evaluated, a set never compared all produce the same clean output as success.
🤖 Generated with Claude Code