[inureyes]

Summary

The counting routes encoded the rendered template text, where each image, clip or audio part is a single placeholder; the soft-token expansion only happens on the worker, inside the per-family call that builds the media embeddings. A client budgeting context with input_tokens undercounted every media request by the size of its media.

• One shared counter (routes::prompt_inspection::count_prompt_tokens) now serves /chat/completions/input_tokens, /responses/input_tokens and /v1/messages/count_tokens (plus their unversioned aliases). The Anthropic handler renders and counts through the shared helpers instead of its own copy.
• Text-only prompts are still encoded on the request thread, unchanged.
• A prompt with resolved media goes to the worker as a count-only request (ServerGenerateOptions::count_prompt_only). The batch scheduler runs the generation path's own tokenization and media preparation, answers Done with the expanded length, and closes the sequence before any prefill, which releases the per-sequence state the binds attached. Media bytes are the ones the render fetched, and the preparation fills the vision feature cache a following generation reads. Workers that do not read the flag serve a one-token generation, which reports the same prompt_tokens.
• Two families keep media state outside the per-sequence bindings: Llama 3.2 Vision (model-wide cross-attention states, no prompt expansion) now answers a count without preparing media, and Phi4MM's address-keyed embedding registration is dropped when count embeddings are discarded.

Real checkpoint

mlxcel-server -m ~/models/mlx/gemma-3-4b-it-4bit --video-max-frames 4, MLXCEL_VIDEO_DIR_ALLOWLIST set to a directory with two 4 s ffmpeg lavfi clips (testsrc2, smptebars, 320x240), and a 224x224 testsrc PNG as a data URI:

Body	usage	input_tokens before	input_tokens after
chat, text	17	17	17
chat, image	275	16	275
chat, two clips (frame fallback)	2,124	52	2,124
responses, text	15	15	15
responses, image	275	16	275
anthropic, text	15	15	15
anthropic, image	275	16	275

Each "after" figure was identical on both paths of the route (/v1/... and the unversioned alias). "Before" is the #1748 binary.

Tests

• routes/prompt_count_route_tests.rs: with a fake worker that expands each media item into 1,000 tokens, input_tokens equals usage for image bodies on all three routes and for the two-clip video body (that one is #[ignore], needs ffmpeg, and was run here with --ignored); a text-only count stays local and never reaches the worker.
• batch/scheduler/prompt_count_tests.rs: a count-only request answers its length from admission, queues nothing, runs no forward, leaves no live sequence, and a following request still prefills.
• Revert checks: with the route counter forced back to the local encode, the three image tests and the video test fail; with the scheduler short-circuit removed, the scheduler test fails.
• Narrow CUDA runs (--profile test-fast --features cuda, --test-threads=1) of prompt_count, server::routes::prompt_inspection, server::routes::anthropic, server::batch::scheduler::request_memory, server::model_provider and phi4mm pass; clippy --lib --tests -D warnings, fmt and the contract checks are clean.

Not covered

• Streaming /v1/messages still fills message_start.usage.input_tokens from a local encode of the placeholder text, so for image bodies it disagrees with this count and with non-streaming usage. That predates this change.
• The counting routes do not apply the reasoning re-echo the generating routes use (fix: reasoning-model prompt cache misses when clients do not echo reasoning #2110), so with the echo store on, a body with content-only assistant history can still count below its usage. Also pre-existing, text bodies included.

Closes #1788