Repository navigation
docs: record the unified engine final measurement on GB10 - #2226
Merged
Merged
Conversation
End-of-run measurement for epic #2166 against the pre-epic CxxGenerator build (4c44e31) and the final engine build (30ceea2), with the raw records and drivers. - mlxcel generate decodes within the 1.0 percent threshold in every cell (+0.19 to +0.82 percent). - The server engine at B=1 with dense storage, the path mlxcel run and chat take since #2173, is 1.1 to 2.1 percent below the pre-epic CLI, the same scheduler overhead Phase 0 measured, and misses the threshold in three of four cells. - Batched serving is unchanged on Qwen3 and 16 to 18 percent higher with Turbo4 at concurrency 4 and 8. - Prefill chunk 2048 vs 512 under live decode: 2048 halves the admitted request's TTFT and raises live streams' p95 ITL 2.3 times. - Gemma 3 paged vs dense under concurrency is unresolved.
inureyes
added
type:docs
7 tasks
This was referenced Oct 8, 2026
inureyes
added a commit
that referenced
this pull request
Oct 8, 2026
…nt (#2231) ## Summary One-line correction to `docs/benchmark_results/unified-engine-final-gb10-2026-10-08.md` (PR #2226). The server engine at B=1 with dense storage measured -1.44, -1.14, -1.24 and -2.05 percent against the pre-epic CLI. That misses the -1.0 percent threshold in all four cells, not in three as the summary line said. The table was already correct. The follow-up is #2227. ## Test plan - [x] Text-only change; the table values are unchanged Part of #2166
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
End-of-run measurement for epic #2166 (ADR 0007's regression threshold and the speed questions the phases left open), on GB10 against two release builds: the pre-epic
CxxGeneratorbuild (4c44e31, PR #2194) and the final engine build (30ceea2, PR #2225). Addsdocs/benchmark_results/unified-engine-final-gb10-2026-10-08.md, its raw records and drivers underdocs/benchmark_results/data/unified-engine-final-gb10-2026-10-08/, and an index entry indocs/benchmarks.md. No code changes.Results
mlxcel generate(the engine'sDirectEngineclient) decodes within the 1.0 percent threshold in every cell: +0.82 and +0.31 percent on Qwen3-1.7B (256 and 8192-token prompts), +0.51 and +0.19 percent on Llama-3.2-1B. TTFT at 256 tokens is about 1 ms higher (+4.4 percent unresolved on Qwen3, +9.0 percent on Llama).mlxcel runand chat take since refactor: route mlxcel run and chat through the in-process server engine #2173, is 1.14 to 2.05 percent below the pre-epic CLI and misses the threshold in three of four cells. Phase 0 measured the same scheduler overhead before the epic, butrunusedCxxGeneratorthen, sorunusers see a 1 to 2 percent decode regression.--parallel 4) decodes as before; its 8192-token TTFT dropped 19 percent (Qwen3) and 12 percent (Llama) with the 2048-token chunk.Test plan
scripts/engine_bench_rounds.py, 5 interleaved rounds with a null arm and the quiet-host gate, both builds throughpick.sh, under the hostgpu-lockwith no other GPU workscripts/bench_serving_concurrency.pyandscripts/bench_mixed_step_admission.py, 3 alternating rounds per armPart of #2166