[jvmenen]

Why

FR-Spec is meant to make the draft step cheaper. On packs whose draft step runs through the generic MTP route, such as the dense Qwen3.8-27B, MTPLX_FRSPEC_DRAFT=1 builds the smaller draft head and reports installed: True, but the draft step keeps using the original full head, so the switch has no effect. The only working alternative was the older legacy lane (MTPLX_FRSPEC_LEGACY=1), which in turn rules out two other draft speedups: the K20 prescatter and the sampled draft chain.

How it works, in plain terms

In speculative decoding a small draft head guesses the next tokens and the full model checks them. Scoring all of the roughly 248,000 tokens in the vocabulary is a large part of each draft step. FR-Spec gives the draft head a reduced output layer with only the 65,536 most frequent tokens. The full model still checks every guess against the whole vocabulary, so the output stays exactly the same; only how often a guess is accepted can change. MTPLX already built that reduced layer, but on this route it never reached the place the draft step reads from. This change puts it there (_mtplx_draft_lm_head). The tokens that were left out get a score so low that they are never picked, so everything after the draft step still sees ordinary token ids. Defaults do not change: outside Flash-Next, FR-Spec stays opt-in.

What changes

install_frspec_draft_head (mtplx/frspec_draft.py) now makes the full-vocabulary wrapper live on the configured draft-head route too. A new helper _bind_full_head keeps the Qwen4 native route as it was (bind hook, _mtplx_draft_lm_head untouched) and, when the model has no bind hook, puts the wrapper in _mtplx_draft_lm_head, which is the head the generic mtp_patch draft forward projects through on every step. The configured head is kept at _mtplx_frspec_saved_head, and the install report gains binding. Two small guards come with it: a rotated packed head (Bonsai's HadamardQuantizedLinear) is refused instead of pruned without its activation transform, and _draft_head_identity hashes the wrapper's pruned head instead of an empty attribute set.

Results

M5 Pro 64 GB, Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed, profile turbo, depth 3, capture_commit, draft head 4-bit g64, draft temperature 0.6, fans default, 2026-09-30. Arm A: FR-Spec off; arm B: MTPLX_FRSPEC_DRAFT=1 with the builtin qwen38-code-64k table, bound by this change. Fresh server per run, runs alternated.

By text type (10 prompts, 800 tokens each, pooled over both runs per arm):

Text	A tok/s	B tok/s	A acceptance	B acceptance
Code, reasoning on	29.2	31.7 (+8.6%)	0.82 / 0.64 / 0.48	0.82 / 0.63 / 0.47
English prose, reasoning on	26.5	28.5 (+7.5%)	0.76 / 0.53 / 0.36	0.78 / 0.54 / 0.38
Dutch summaries, reasoning on	27.0	26.1 (-3.5%)	0.81 / 0.58 / 0.39	0.70 / 0.46 / 0.31
Dutch summaries, thinking off	24.4	21.4 (-12.4%)	0.72 / 0.48 / 0.27	0.60 / 0.29 / 0.13

With the builtin table, code and English gain 7-9% while Dutch loses, because the table is ranked on code and too many likely Dutch tokens fall outside the kept 65,536 rows. That is why this PR keeps FR-Spec opt-in. The binding works with an operator table too (MTPLX_FRSPEC_VOCAB=<file.npy>): a 65,536-row table ranked on an operator's own, mostly Dutch, agent traffic, on the same model and settings (two boots, compared with three FR-Spec-off boots): code +14% (29.3 -> 33.5 tok/s), English prose +9% (27.1 -> 29.4), Dutch with reasoning +11% (27.6 -> 30.7), Dutch without thinking +10% (24.1 -> 26.4), with Dutch acceptance unchanged (D1 0.82 -> 0.81). So the binding itself carries the gain; which table fits depends on the workload.

An agent tool-calling task (13 requests, 18 tool calls) completed with tests green with FR-Spec on.

Tests

• tests/test_frspec_configured_route.py: the wrapper is live in _mtplx_draft_lm_head with a full-vocabulary output equal to the configured head on kept rows; the native route is unchanged; a rotated head is refused; the draft-head identity follows the pruned head. All fail on main.
• Existing FR-Spec, prescatter, env-flag, draft-head, family-settings and session-bank tests: all pass. Ruff: no new findings in the touched files.

[youssofal]

Not in 2.12.1, which was a stability release. I'll review this for the next release.