refactor: route mlxcel run and chat through the in-process server engine #2173

Part of #2166. Phase 5. Depends on #2172 (Phase 4b: engine step API, scheduler on it). It also needs the maintainer-confirmed default decision table from #2167 (ADR 0007). Phase 6 builds on this.

Problem / Background

mlxcel run and the chat REPL still use CxxGenerator (through MlxInferenceSession, src/lib/mlxcel-core/src/session.rs:47-60). They also keep their own copies of features the server already has:

• Reasoning filter: ReasoningFilter in src/reasoning_stream.rs:84, used by src/commands/chat.rs. The server uses StreamFilter in src/server/tool_calls/stream_filter.rs:460.
• VLM embedding dispatch: compute_vlm_embeddings in src/commands/generate_vlm.rs:496. The server uses prepare_request_vlm_embeddings in src/server/model_worker.rs:1533, which says it mirrors the CLI.
• DFlash: the offline driver serves only Laguna (src/commands/generate_dflash.rs:43-75, which rejects Qwen 3.5 with "runs through mlxcel-server"). The server uses DFlashTargetModel in src/server/batch/dflash_target.rs:184.

llama.cpp solved the same split by making llama-cli a client of llama-server. tools/cli/cli-server.h runs llama_server() in a thread, and tools/cli/cli-context.cpp posts chat requests to it (ggml-org/llama.cpp#17824, ggml-org/llama.cpp#24948). Now that #2172 puts the server on the batch-native engine, the CLI can do the same and get the same output.

Proposed Solution

mlxcel run (with and without -p) and the chat REPL drive generation through the server engine, in-process:

• Start the model worker with spawn_model_worker_with_batch_config (src/server/model_worker.rs:244) at max_batch_size=1.
• Build requests with the same code the /v1/chat/completions handler uses: chat-template rendering, then build_server_generate_options (src/server/request_options.rs:278).
• Consume GenerateEvent (src/server/model_provider.rs:180).

Rejected alternative: a loopback HTTP server on a random port, as llama-cli does. It adds a socket, SSE serialization and startup cost, and gives no extra code sharing beyond the request parse. A --server-base <url> HTTP client for a remote server is optional and may be a follow-up issue.

What comes from the server path:

• Chat templates and multi-turn history.
• Reasoning splitting: retire ReasoningFilter and src/reasoning_stream.rs in favor of StreamFilter.
• Tool calls and stop strings.
• VLM embedding preparation: retire compute_vlm_embeddings in favor of prepare_request_vlm_embeddings.
• MTP and DFlash through the server's speculative burst: DFlashTargetModel for DFlash, which also brings Qwen 3.5 DFlash to the CLI, and the MTP slice and burst (run_mtp_burst in src/server/batch/speculative_burst.rs, over the per-target adapters in src/models/gemma4_mtp_target.rs and src/models/qwen3_5_mtp_target.rs) for MTP. Since fix(gemma4): make MTP verify byte-identical to decode on CUDA #2185 the B=1 Gemma 4 MTP prefill splits a chat prompt at the same history boundary as classic serving (history_boundary_split, src/server/batch/scheduler/prompt_cache.rs:1212), and on CUDA the Gemma 4 12B and 31B pairs give greedy chat output byte-identical to classic decode.

Defaults follow the ADR 0007 decision table (repeat_last_n, DRY sequence breakers, loop detection, prefill chunk, and the samplers the CLI hard-codes off today at src/commands/generate.rs:1189-1260). Each CLI sampling flag maps one to one onto a server request option. A flag with no server equivalent is either added to the server request options or removed, with the change listed in the PR.

mlxcel generate (raw completion) stays on CxxGenerator until Phase 6. Update the module doc of src/commands/run.rs, which promises that run -p is byte-identical to generate because they share code. After this phase that promise holds through the equivalence invariant, not through shared code.

Scope

In scope: src/commands/run.rs, src/commands/chat.rs, retiring src/reasoning_stream.rs and the CLI VLM dispatch, CLI DFlash and MTP through the server path, the flag mapping, and docs.

Out of scope: mlxcel generate, bench_decode and the classic SpeculativeGenerator (Phase 6), and the remote --server-base client (optional follow-up).

Implementation Notes

• Reuse: the server's request parsing, template rendering, StreamFilter, tool-call parsers (src/server/tool_calls/) and VLM preparation. The CLI keeps only terminal I/O and REPL commands.
• Constraints: no HTTP listener is opened unless the user asks for one. CLI startup time must not regress noticeably, so measure it once at the end. Ctrl-C cancels the in-flight request through the worker's cancel path and does not kill the process.
• Edge cases: /clear resets the server-side conversation, multi-turn with images, a model whose template has no reasoning markers, and a non-batching family (supports_batching() == false), which the worker already serves sequentially.

Acceptance Criteria

• Under MLXCEL_SDPA_DETERMINISTIC=1, the Phase 0 harness shows mlxcel run -p and the server giving identical greedy output for the same request on Qwen3-1.7B and Llama-3.2-1B
• CLI single-stream decode stays within the Phase 0 threshold of the pre-epic CxxGenerator baseline, at short and long context
• Interactive chat works on a real checkpoint: multi-turn, /clear, Ctrl-C mid-answer, reasoning hidden or shown, and an image turn on a VLM. The REPL has no regenerate command today (src/commands/chat.rs:535-541). If one is added, it goes through the same path
• src/reasoning_stream.rs and compute_vlm_embeddings are removed. rg -n 'ReasoningFilter|compute_vlm_embeddings' src returns nothing
• run gains --draft-model and --draft-kind (it has neither today), mapped onto the server's existing options. mlxcel run -m <Qwen 3.5 target> --draft-model <DFlash drafter> -p ... works
• The token-exactness section of docs/CONTINUOUS_BATCHING.md (:65-79) and the CLI docs describe the shared engine
• run and chat actually execute through the server engine, with no leftover CxxGenerator call in src/commands/run.rs or src/commands/chat.rs

Verification

make verify                                   # macOS gate
make verify-fmt verify-test-cuda              # Linux/CUDA gate
MLXCEL_SDPA_DETERMINISTIC=1 make <Phase 0 parity target> MODEL=<Qwen3-1.7B-4bit path>
mlxcel run <Qwen3-1.7B-4bit path>             # manual: multi-turn, /clear, Ctrl-C mid-answer

References verified against main at fc21784 (2026-10-07).

Progress (PR #2224): every criterion except the single-stream decode measurement, which the epic measures at its end, is implemented and checked; the parity criterion is gated by engine-parity arms e:run vs f:server (same chat request).