[iliaal]

Summary

Adds rtk phpt, a filter for php run-tests.php. It's the driver php-src and PHP extensions use for their .phpt test suite.

A full php-src run emits one status line per test plus a ========DIFF======== block per failure. On a 5322-test run that's about 1.3 MB of output, 99%+ of which is PASS chatter. After filtering:

Scenario	Raw	Filtered	Reduction
php-src full run (4738 PASS, 467 FAIL, 116 SKIP, 1 XFAIL)	1,301,105 B	6,279 B	99.5%
Single passing test	~200 B	2 lines	~85%
Small all-pass dir (11 tests)	~4 KB	2 lines	~97%

Filtered output keeps what the agent needs: aggregate counts, PHP version/SAPI/OS, and up to 20 failures with the first 6 diff lines each.

Example

phpt: 4738 passed, 467 failed, 116 skipped, 1 xfail  (5322 total, 391.4s)
PHP 8.4.20  SAPI cli  OS Linux

FAILURES (467):
  Zend/tests/__debugInfo_reference.phpt -- __debugInfo with reference return
    001- object(Test)#1 (1) {
    002-   ["x"]=>
    003-   int(1)
    004- }
    001+ Fatal error: __debuginfo() must return an array in /home/.../__debugInfo_reference.php on line 12

  Zend/tests/abstract_implicit.phpt -- Abstract methods not allowed in classes (GH-16067)
    001- Fatal error: Class NotAbstract declares abstract method bar() ... in %s on line %d
    001+ Fatal error: Class NotAbstract contains 1 abstract method ... on line 11

... +447 more failures

Usage

cd /path/to/php-src
rtk phpt                           # full suite
rtk phpt Zend/tests/                # directory
rtk phpt Zend/tests/67468.phpt     # single test
rtk phpt -j8 ext/standard/         # args pass through to run-tests.php

Implementation

src/cmds/php/phpt_cmd.rs is a new filter module modeled on python/pytest_cmd.rs: stateful line scan, regex anchors, inline tests.

Status tokens match via (?:^|\])(PASS|FAIL|SKIP|BORK|WARN|LEAK|XFAIL)\s+(.*?)\s*\[([^\[\]]+\.phpt)\]. The (?:^|\]) anchor is what prevents the trailing FAILED TEST SUMMARY block from being re-counted. Its lines are description [path] with no status tag.

ANSI is stripped via the existing utils::strip_ansi. Execution goes through runner::run_filtered with stdout_only().tee("phpt"), same as pytest. The command is added to the is_operational_command whitelist so it flows through the hook pipeline.

Tests

12 inline unit tests:

• All-pass run
• Single FAIL with DIFF block
• Diff truncation (more than 6 lines)
• Failure-list truncation (more than 20 failures)
• SKIP and XFAIL don't produce a FAILURES section
• BORK counts as a failure
• Empty run (no tests ran)
• FAILED TEST SUMMARY block doesn't re-count failures
• ANSI stripping
• Compression ratio on a synthesized 1000-test pass run (>99%)
• PHP_OS kernel info trimmed to just the OS family
• Summary-line counts override scanned counts when capture is truncated

Test plan

• cargo test -p rtk phpt passes 12/12
• cargo fmt --all -- --check clean
• cargo clippy -p rtk no new warnings in diff (pre-existing errors unchanged)
• Live rtk phpt Zend/tests/67468.phpt on real php-src, correct output
• Live rtk phpt Zend/tests/ArrayAccess/ (11 tests), correct output
• Captured 5322-test output run through filter: 99.5% reduction, 467 failures with diffs

[CLAassistant]

All committers have signed the CLA.

[pszymkowiak]

[w] wshm · Automated triage by AI

📊 Automated PR Analysis

✨ Type	feature
🟢 Risk	low

Summary

Adds a new rtk phpt filter command for PHP's run-tests.php test runner, reducing output by ~99% by keeping only aggregate counts, environment info, and bounded failure diffs. The implementation follows the existing pytest filter pattern with a stateful line scanner, regex-based parsing, and 12 inline unit tests.

Review Checklist

• Tests present
• Breaking change
• Docs updated

---

Analyzed automatically by wshm · This is an automated analysis, not a human review.

[KuSh]

Hi @iliaal, you'll have to rebase your work now that your other PR has been merged. Few nitpicks and suggestions

[iliaal]

Rebased onto develop — conflict cleared, PR is mergeable again. All six comments addressed in 6aea5c7; details inline. Thanks for the thorough pass.

[iliaal]

Heads-up on the red benchmark check: it's the flaky live-network endpoint, not this PR. The only failing item is rtk curl https://mockhttp.org/json (38 → 61, flagged negative). This branch doesn't touch the curl filter at all — the diff is phpt-only — so the filter binary is identical to develop's, where the same benchmark passed (curl json 38 → 38, 0 negative) on b52e02b. The difference is purely the live mockhttp.org response between the two runs (the same commit that switched benches to mockhttp.org notes httpbin was "struggling"). All correctness checks — test on macOS/Ubuntu/Windows, clippy, fmt, test presence, security scans — are green. A benchmark re-run should clear it.

[KuSh]

Heads-up on the red benchmark check: it's the flaky live-network endpoint, not this PR.

Yes I'll check the benchmark, I've fixed a reliability problem but introduced a flakiness :/ Sorry about that

[KuSh]

Hi @iliaal, sorry for the delay I was away for 2 weeks. I've reverted the insta part of your code as it was a mistake, our documentation was outdated and I blindly followed it, my bad, sorry.

Everything looks good to me, but in the meanwhile, never_worse and emit_guarded have been induced to avoid emitting more tokens than the raw command, see transparency. Could you rebase your code and take that into account?

Thanks

[iliaal]

Rebased onto develop (5d32d07), and thanks for reverting the insta commit yourself.

On the guard: phpt goes through runner::run_filtered, which already routes emission via print_with_hint → emit_guarded → guard::never_worse, so the never-worse property holds without a filter change. What the guard did surface is a bug. RunOptions::stdout_only() dropped php's stderr, and php writes its startup diagnostics there, so running rtk phpt from the wrong directory left the guard with an empty base and rtk printed nothing at all. Exit code 1, zero output.

Verifying that against real php-src turned up two more, both mine from the original commit:

Serial runs lost every per-test status. run-tests.php redraws its progress line, so without -j the status arrives after a carriage return: TEST 1/2 [bad.phpt]\rFAIL failing sample [bad.phpt]. The (?:^|\]) anchor never matched, so counts came only from the summary block and the FAILURES section printed "(per-test details unavailable)". My 5322-test validation ran with -j8, and workers suppress the progress line, which is why I missed it. The filter now splits on CR.

No diffs on multi-test runs. run-tests.php emits ========DIFF======== only with --show-diff or when exactly one test is selected, so the diff lines this PR advertises never appeared for a directory run. rtk now passes --show-diff unless you passed your own --show-* flag. That grows raw output, which flatters the savings ratio, so I'd rather flag it than bury it: if you want the filter to stay passive, I'll drop the injection.

Verified on php-src, serial run of ext/standard/tests/strings/:

phpt: 666 passed, 12 failed, 55 skipped  (733 total, 20.9s)
PHP 8.4.23  SAPI cli  OS Linux

FAILURES (12):
  ext/standard/tests/strings/chr_out_of_range.phpt -- chr() with out of range values
    001- Deprecated: chr(): Providing a value not in-between 0 and 255 ...

153,884 → 4,476 bytes, 97.1%. The wrong-directory case now prints Could not open input file: run-tests.php. Two new tests cover the CR shape and the passthrough.

fmt, clippy --all-targets and test --all are clean apart from hooks::rewrite_cmd::tests::unattestable_passthrough::{test_plain_command_still_rewrites,test_fd_dup_redirect_still_rewrites}, which fail identically on a clean develop worktree here.