perf: restore exaone-4.0 CLI decode lost by the generate-on-engine refactor (#2225) #2494

Problem

exaone-4.0-1.2b-4bit single-sequence CLI decode regressed from about 198 tok/s on v0.7.0 to about 158 tok/s (-21%) on GB10 (NVIDIA GB10, sm_121, CUDA) at main c25570f, measured with mlxcel generate -n 384 --temp 0 on a quiet box.

Bisect result

A bisect over (a6e0754, c25570f] using exaone decode tok/s as the metric (3 runs per commit, median verdict, 75 s cool-down after each build) points to 30ceea2 "refactor: run mlxcel generate on the engine and retire CxxGenerator (#2225)" as the first bad commit: median 163.28 tok/s against the parent's 193.97. The parent d379d2f still ran mlxcel generate through CxxGenerator, so the bisect compares the retired C++ loop against the new DirectEngine client directly.

Isolation

• MLXCEL_FORCE_SYNC=1 (the engine's synchronous path) is slower still: 136-138 tok/s, so the B=1 lookahead pipeline recovers part of the loss and the engine step itself is what got slower.
• The regression is not general to small models: llama-3.2-1b-instruct-4bit measures about 5% faster on main (256-258 vs 243-244 tok/s, same prompt and -n), so a uniform per-step host overhead does not explain it.
• The server's scheduler path is unaffected: refactor: route mlxcel run and chat through the in-process server engine #2224 moved run and chat onto the engine and the bisect measures it as good (193.97).
• refactor: run mlxcel generate on the engine and retire CxxGenerator #2225's acceptance item "bench_decode throughput against the Phase 0 threshold" was left unchecked (deferred to the epic orchestrator) and this regression is exactly that unverified threshold.

Acceptance criteria

• Profile DirectEngine's step against the retired CxxGenerator loop for exaone-4.0 (per-step host time and GPU gaps) and restore exaone CLI decode to within a few percent of v0.7.0's ~198 tok/s without regressing the other families.
• llama-3.2-1b keeps its current main speed (faster than v0.7.0); granite-4.0-h is tracked separately in perf: gate the granite-4.0-h fused single-token MoE routing per device (GB10 CUDA -25%) #2492.
• Verified with several interleaved runs per arm on a quiet box (no other GPU process or compiler running), with numbers re-measured on the box that ships the fix rather than copied from this issue.

Measurement setup

mlxcel generate -m exaone-4.0-1.2b-4bit -p <fixed prompt> -n 384 --temp 0 under gpu-lock, main c25570f release build vs the official v0.7.0 release binaries; the bisect verdict log is reproducible with the same step script.