[inureyes]

Summary

Issue 1441 landed its route separation in #1467 and was deliberately left open, because closing it would have forced manifest entries into supported that were not. This PR closes the implementable half of what remained: response_fields, stream_options and stream_options.include_usage, the real prefill figures on timings_per_token frames, and the whole n_predict value domain including the n_predict: 0 prompt-only case, which is now verified against the pinned binary rather than assumed.

Issue 1441 stays open, and this body carries no closing keyword on purpose. POST /completion and POST /completions still carry divergences and therefore stay deferred under it; the validator requires a deferred entry's issue to be OPEN, so resolving it now would turn the llama-compat manifest job red for every open PR in the repository. What remains is listed at the bottom.

The seven entries and where they land

Entry	Before	After
field:response_fields	deferred, no divergence recorded, simply unimplemented	supported
field:stream_options	deferred, no divergence recorded	supported
field:stream_options.include_usage	deferred, no divergence recorded	supported
field:timings_per_token	deferred, zeroed prompt_n / prompt_ms per frame	supported
field:n_predict	deferred, n_predict: 0 unverified	supported
POST /completion	deferred, 3 divergences	deferred, 3 divergences (one reworded to name #1440, one added)
POST /completions	deferred, 3 divergences	deferred, 3 divergences (same)

Evidence: measured before and after

Every expectation is a capture of the pinned b10621 binary (build 10621, commit c1d0e7a00, Qwen3-0.6B-Q8_0.gguf) against mlxcel serving models/mlx/qwen3-0.6b. The schema descriptions are wrong about two of these fields, so the binary is the reference, not the source text.

response_fields

Before: silently ignored, all nine probes answered the full 16-key object. After: identical to b10621 on all nine.

Probe	b10621	mlxcel before	mlxcel after
["content","tokens_predicted"]	["content","tokens_predicted"]	16 keys	["content","tokens_predicted"]
["content","generation_settings/n_predict","timings/predicted_n"]	same three keys, slash kept in the key, generation_settings/n_predict = 8	16 keys	identical
["content","no_such_field"]	["content"]	16 keys	["content"]
["timings/predicted_n","content","index"]	that order	16 keys	that order
[""]	{}	16 keys	{}
["content","content"]	["content"]	16 keys	["content"]
"content" (a string)	200, 16 keys	200, 16 keys	200, 16 keys
["content/x","tokens/0"]	{}	16 keys	{}
streaming	per-token frames unprojected, final frame projected	not projected at all	identical to b10621

The slash form is the trap: the description says generation_settings/n_predict "moves n_predict to the root", and the binary instead emits a root key literally named generation_settings/n_predict. The implementation follows the binary.

stream_options and include_usage

Probe	b10621	mlxcel before	mlxcel after
{"include_usage": true} vs absent	byte-identical native body, so the field is inert	inert, but as an unknown key that was never parsed	inert, and declared and type-checked
"stream_options": "garbage"	200	200 (unknown key)	200
{"include_usage": "yes"}	400, Field 'include_usage': [json.exception.type_error.302] type must be boolean, but is string	200, value dropped	400, Field 'include_usage': type must be boolean, but is string

Inert on the native route is the correct behavior, not a shortcut: the native final frame already carries tokens_predicted, tokens_evaluated and the whole timings object whether or not the field is sent, which is why upstream's own /completion ignores it. mlxcel's OpenAI routes already honor the same block the way b10621's do.

timings_per_token

Streaming a fresh 22-token prompt with timings_per_token: true, per frame:

	b10621	mlxcel before	mlxcel after
tokens_evaluated	22 on every frame	0 until the final frame	22 on every frame
timings.cache_n	0	0	0
timings.prompt_n	22 on every frame	0 until the final frame	22 on every frame
timings.prompt_ms	176.368 on every frame	0.0 until the final frame	80.0 on every frame
timings.predicted_ms at frame 1	0.001	21.0 (it was counting prefill)	0.0

The mechanism is a one-shot GenerateEvent::Prefill snapshot emitted by SequenceInfo::mark_first_token when the scheduler stamps the first token, which reaches the streaming route before the first frame. The route could not previously learn the prefill figures until GenerateEvent::Done.

Two derivations inside timings were wrong against the binary and are corrected in the same change, so a client that graphs them will see different numbers:

• prompt_n is tokens_evaluated - cache_n, not tokens_evaluated. Measured on a repeated prompt upstream: tokens_evaluated: 8, cache_n: 7, prompt_n: 1.
• predicted_per_token_ms and predicted_per_second divide by predicted_n - 1, because predicted_ms starts at the first token. Measured: predicted_n: 4, predicted_ms: 15.132, predicted_per_token_ms: 5.044 (15.132 / 3) and predicted_per_second: 198.255 (1e3 / 15.132 * 3). The old test asserting ms / n was named timings_rates_are_derived_the_way_upstream_derives_them and was not.

n_predict

Probe	b10621	mlxcel before	mlxcel after
0, prompt "A fresh unique prompt zzz 12345 alpha beta gamma"	content: " delta", tokens_predicted: 1, tokens_evaluated: 15, stop_type: "limit"	identical, but unverified	identical, now verified
0, streaming	2 frames, final tokens_predicted: 1	2 frames	2 frames
-1 with stop: [" 9"]	tokens_predicted: 12, stop_type: "word", stopping_word: " 9"	HTTP 422, invalid value: integer -1, expected usize	tokens_predicted: 12, stop_type: "word", stopping_word: " 9"
-2	400, Field 'n_predict': Value must be between -1 <= value <= 2147483647, but got -2	HTTP 422, serde type error	400, same message text

The n_predict: 0 case is worth stating plainly, because the schema description says "no tokens will be generated but the prompt is evaluated into the cache" and the binary emits exactly one token. mlxcel emits the same one token on the same prompt, so the two agree with each other and both disagree with the description. -1 is upstream's "as many as the context allows" and now resolves through the existing effective-context clamp instead of being refused.

What changed

• src/server/model_provider.rs: PrefillStats and GenerateEvent::Prefill, a drain_generation_events_with_prefill drain, and generate_streaming_native_reserved for the native route. The new variant does not open the bounded decode window, so it cannot start the hang timer while the first token is still being sampled.
• src/server/batch/sequence.rs: SequenceInfo::mark_first_token, idempotent, replacing the two bare first_token_time = Some(...) assignments in scheduler.rs and speculative_burst.rs. prompt_ms uses created_at as its origin, the same origin take_generation_result hands to finish_with_cache, so the streamed figures and the final frame cannot disagree.
• src/server/types/native_completion.rs: select_response_fields, and NativeTimings::new now takes tokens_evaluated and derives prompt_n and the corrected rates.
• src/server/types/request.rs: n_predict becomes Option<i64> with resolve_n_predict, plus response_fields and stream_options held as raw values with response_field_paths and validate_stream_options, so upstream's tolerant-parse-then-reject behavior is reproducible rather than pre-empted by serde.
• src/server/routes/native_completion.rs: projection applied to the non-streaming body and the final streaming frame through one project_native_response, and the streaming frames read the prefill snapshot.
• src/server/llama_compat_tests.rs: the native-field claim gate gained a rule for entries with a non-null parent, which are modeled by a nested struct rather than a root key. It is documented as a deliberately weaker ratchet than the root-field rule, with the behavioral half asserted in native_route_tests.rs.
• compat/llama-server/b10621/routes.json and docs/llama-server-compat.md.

What moved to #1440, and why

POST /completion and POST /completions carried a divergence reading "id_slot is always -1: the continuous-batching scheduler exposes no stable per-request slot number (#1441)". Exposing a slot number is #1440's scope ("Expand /slots to expose compatible slot state"), and #1440 is already an owner of routes.json, so the line now reads "... so exposing one is #1440's scope, not this entry's (#1440)". Only that divergence line moved: the entries stay filed under #1441, because their other divergences are #1441's.

What remains on #1441, and why it stays open

POST /completion and POST /completions keep three divergences:

1. generation_settings echoes the settings mlxcel resolved rather than b10621's full task_params block (49 keys), so keys mlxcel has no analogue for are absent. Reporting an invented value for them would be worse than their absence.
2. tokens_cached counts the prompt tokens mlxcel's KV prefix cache supplied, while b10621 reports the slot's cache occupancy after the request, which is tokens_evaluated + tokens_predicted - 1. Measured on a repeated 51-token prompt answered with 4 tokens: b10621 tokens_cached: 54, mlxcel its own cache-supplied count. This divergence is newly recorded by this PR; it was not in the list before, and I found it while measuring the cache-hit path.
3. return_tokens, return_progress, verbose, n_indent, t_max_predict_ms and n_cmpl above 1 are refused with a diagnostic rather than honored. Each has its own not_applicable entry, and those do not block the issue.

Items 1 and 2 are the substantial remainder and are probably worth their own issue, since both are response-body fidelity work with no dependency on the field work here. I have not filed it; say the word and it can be filed with the measurements above.

Two smaller observations recorded here rather than in the manifest, because neither belongs to an entry this issue owns:

• mlxcel's error envelope reports "code": null where b10621 reports the numeric status ("code": 400). It is the shared ErrorResponse across every mlxcel route, not a property of these fields, so recording it on field:stream_options would misattribute it.
• On the cache-hit probe above, mlxcel's prompt prefix cache did not adopt a repeated 51-token prompt under its defaults (cache_n: 0) where b10621 reused 50 tokens. That is prompt-cache behavior, owned by the prompt-caching issues, not by these entries.

Test plan

• cargo test --profile test-fast --features metal,accelerate --lib -- server:: (2207 passed, 0 failed, 8 ignored)
• cargo test --profile test-fast --features metal,accelerate --lib -- server::batch:: (387 passed, 0 failed, 7 ignored)
• cargo test --profile test-fast --features metal,accelerate --test llama_compat_manifest (4 passed)
• cargo clippy --lib --bins --tests --features metal,accelerate --profile test-fast -- -D warnings
• cargo fmt --all -- --check
• python3 scripts/ci/check_llama_compat_manifest.py
• python3 scripts/ci/check_cross_repo_refs.py
• Side-by-side comparison of every behavior above against the pinned b10621 binary and models/mlx/qwen3-0.6b

Refs #1441, #1440