[inureyes]

Summary

gemma-4-12b-it (model_type: gemma4_unified) carries the <|channel> / <channel|> thinking markers, so on startup the server resolves a think-marker pair and defaults the enable_thinking chat-template kwarg to true, mirroring ml-explore/mlx-lm#1114. Per the maintainer decision this default stays as-is: the marker detection in src/tokenizer/mod.rs and set_default_enable_thinking in src/server/startup.rs are untouched. The real bug is that the non-streaming /v1/chat/completions response silently discarded the thinking scratchpad, so a verbose thinking model looked like it returned an empty or truncated answer at a normal max_tokens.

This surfaces the reasoning as reasoning_content on the non-streaming response, which closes the gap for every thinking family at once (qwen3 <think> and gemma4 <|channel>), and documents the thinking default plus a recommended higher max_tokens for gemma4_unified.

What changed

• src/server/types/response.rs: add an optional reasoning_content: Option<String> to the response ChatMessage (#[serde(skip_serializing_if = "Option::is_none")], so non-thinking responses keep their exact wire shape), matching the OpenAI/vLLM convention and the streaming delta.reasoning_content. Add a with_reasoning_content chained setter that mirrors with_cached_tokens.
• src/server/routes/chat.rs: add extract_reasoning_content, which replays the raw generation through the same StreamFilter the streaming path constructs, then wire it onto all three non-streaming return points (tool-calls, tools-enabled-no-calls, plain). The existing content computation (strip_unclosed_primed_thinking / clean_structural_tokens / tool-call parsing) is unchanged; reasoning is purely additive.
• docs/supported-models.md: document the gemma4_unified thinking default, the recommended max_tokens (at least 512), the reasoning_content surfacing on both streaming and non-streaming paths, and that per-request/CLI/env enable_thinking=false disables it.
• docs/block-diffusion.md: correct the stale claim that non-streaming responses strip reasoning_content.

StreamFilter reuse (streaming / non-streaming parity)

Rather than write a new marker parser, extract_reasoning_content builds the same StreamFilter the streaming SSE path builds (primed-open vs fresh, keyed on is_prompt_primed_open_thinking), feeds the whole generated string once, calls flush(), and accumulates the reasoning pieces. The filter state machine is deterministic over the concatenated input, so feeding the whole string yields the same reasoning/content split as feeding it token-by-token. A unit test (reasoning_split_identical_whole_vs_chunked) locks this equivalence, which is what guarantees streaming and non-streaming surface identical reasoning. Per-request chat_template_kwargs.enable_thinking, --chat-template-kwargs, and LLAMA_ARG_CHAT_TEMPLATE_KWARGS precedence is untouched (merge_server_and_request unchanged).

Real-model validation (gemma-4-12b-it-4bit, temperature 0)

Startup log confirms the intended default is preserved: Tokenizer recognizes a think marker pair; defaulting chat_template kwarg enable_thinking=true (upstream PR #1114) think_start=Some("<|channel>thought") think_end=Some("<channel|>").

Prompt "What is a hash table? Answer in one sentence." at max_tokens=400, default thinking:

finish_reason: stop
completion_tokens: 275
has reasoning_content key: True
reasoning_content len: 980
reasoning_content (head): "*   Topic: Hash table. ... *Draft 1:* A hash table is a data structure that maps keys to values by using a hash function ..."
content: "A hash table is a data structure that maps keys to values by using a hash function to compute an index into an array, allowing for efficient data storage and retrieval."

Same prompt with chat_template_kwargs={"enable_thinking": false} (per-request precedence preserved):

finish_reason: stop
completion_tokens: 33
has reasoning_content key: False
content: "A hash table is a data structure that maps keys to values by using a hash function to compute an index into an array, allowing for efficient data retrieval and storage."

This also answers the issue open question 3: the over-thinking is marker-delimited (<|channel>thought ... <channel|>), since the filter captured the full 980-char block into reasoning_content while content holds only the answer (an empty content plus finish_reason=length at a tiny budget was the tokens being stripped as a marked thinking block, not leaked unmarked text).

This fixes qwen3 too

qwen3-4b-4bit thinking request "What is 17 times 23? Answer in one sentence." at max_tokens=512, temperature 0:

finish_reason: stop
completion_tokens: 375
has reasoning_content key: True
reasoning_content len: 932
reasoning_content (head): " Okay, the user is asking for 17 multiplied by 23 ... 340 + 51 equals 391 ..."
content: "17 times 23 equals 391."

Test plan

• cargo check --lib --tests --features metal,accelerate
• cargo clippy --lib --tests --features metal,accelerate -- -D warnings
• cargo test -p mlxcel --lib --features metal,accelerate for the new tests: server::routes::chat::tests::extract_reasoning_* (5), server::routes::chat::tests::reasoning_split_identical_whole_vs_chunked, server::types::response::tests::reasoning_content_* (2). All 8 pass.
• cargo fmt --check
• Real-model smoke on gemma-4-12b-it-4bit and qwen3-4b-4bit (above)

Closes #352

[inureyes]

Follow-up commit 8402a75e1 addresses the HIGH finding from review: the non-streaming chat content leaked the reasoning text and a literal </think> for a primed-open <think> checkpoint that emits a close-only </think> (no opening tag in the output). The bare-leading-close pre-pass in strip_thinking handled only the Gemma 4 <channel|> primed case; it is now generalized to both (open, close) pairs, so chat non-streaming matches the split already done by /v1/responses, /v1/anthropic, and the chat streaming filter.

Added parser tests strip_thinking_primed_qwen_close_only and clean_structural_tokens_primed_qwen_close_only. Verified locally under the pinned toolchain (rust 1.93.1, matching CI): clippy -D warnings clean, 47 parser tests pass, the 5 extract_reasoning tests and the whole-vs-chunked parity test pass, fmt clean.