Stdlib: strtok() — tokenize strings (ext/standard/string.c parity)

[PurHur]

Category

stdlib

Problem

strtok() is missing from ext/standard. Zend uses it for simple parsers, config scanners, and legacy code paths. Calling it today fails at runtime (function_exists('strtok') → false).

php-src reference

• php/php-src ext/standard/string.c — PHP_FUNCTION(strtok)
• php/php-src ext/standard/php_string.h — php_strtok_r state machine

Repro (failure today)

<?php
$s = "a,b,c";
echo strtok($s, ","); // a
echo strtok(",");     // b
echo strtok(",");     // c
echo strtok(",") === false ? "end" : "bad";

./script/docker-exec.sh -- bash -lc 'source script/php-env.sh && php bin/vm.php repro.php'

Zend PHP: abcend
This compiler: fatal / undefined function

Scope (this repo)

• ext/standard/strtok.php — VM implementation with per-request static strtok state (Zend semantics)
• ext/standard/Module.php — register builtin
• lib/JIT/Builtin/ — optional JIT lowering after VM green
• test/compliance/cases/stdlib/strtok.phpt

Done when

• Repro prints abcend on VM (and JIT/AOT if lowered)
• Two-argument form resets internal pointer; one-argument form continues same string
• ./script/ci-fast.sh --filter strtok green
• docs/capabilities.md row via php script/capability-matrix.php

Related

• Epic: Self-host critical path — compiler compiles itself (M3→M5) #1492 self-host — string helpers in lib/
• 🗺️ ROADMAP: Self-host + web-capable PHP subset (living) #78 ROADMAP Phase 4 stdlib

[PurHur]

claim: PHP Compiler STD Lib — starting this run