[Klaud Cold] minimaxm3 H200+H100 MTP: day-zero MiniMax-M3 EAGLE3 (MTP) recipes

[functionstackx]

Summary

Adds the EAGLE3 speculative-decoding (spec-decoding: mtp) siblings of the day-zero MiniMax-M3 MXFP8 H200/H100 recipes (PR #1731, still open against main): MiniMaxAI/MiniMax-M3-MXFP8 target paired with the Inferact/MiniMax-M3-EAGLE3 draft head. This PR targets main and includes only the H100-mtp and H200-mtp variants.

New benchmark scripts

• benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_h200_mtp.sh
• benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_h100_mtp.sh

Both are based on the non-MTP H200/H100 scripts from #1731 (mandatory --block-size 128 for MSA sparse/index cache, --language-model-only, HF_HUB_OFFLINE serve to dodge the shared-FS download-lock race, Hopper-compatible MXFP8 MoE backends). H100 keeps its DEP memory-headroom block (higher gmu + per-rank-bounded capture). Additions for MTP:

• --speculative-config '{"method": "eagle3", "model": "Inferact/MiniMax-M3-EAGLE3", "num_speculative_tokens": 3, "attention_backend": "FLASH_ATTN"}'
• Drafter pinned to FLASH_ATTN: the EAGLE3 head is MHA, and FlashInfer only supports page size 128 (mandatory here for MSA) through its GQA-only trtllm-gen kernel — engine init dies in FlashInferMetadataBuilder otherwise. This exact failure hit the B300 MTP canary (#1733); FLASH_ATTN takes any multiple-of-16 block size and handles MHA.
• Cudagraph capture scaled to CONC * (1 + spec tokens) (each running request contributes the extra draft tokens per decode step), capped at 2048. On H100 DEP the per-rank-bounded capture is likewise spec-token-scaled.
• --use-chat-template on the benchmark client so draft acceptance reflects real text rather than random tokens.

Config (nvidia-master.yaml)

• minimaxm3-fp8-h200-vllm-mtp: TP4 / TP8 (latency), TP4+EP4 / TP8+EP8 (TEP), TP8+EP8 dp-attn (DEP) across 1k1k and 8k1k — non-MTP search space trimmed at the extreme-concurrency end (dsv4 / minimaxm3 b300-mtp precedent).
• minimaxm3-fp8-h100-vllm-mtp: TP8-only (no room below TP8 at ~56 GB weights/GPU); DEP omitted (KV-cache init fails at high conc on the non-MTP entry, draft only tightens it).

Launcher fix (required plumbing)

The three H100 launchers (launch_h100-cw.sh, launch_h100-cr.sh, launch_h100-dgxc-slurm.sh) hardcoded _h100.sh and never gained the _mtp routing the H200 launchers have carried since #392. Without this, an mtp config on H100 silently runs the non-MTP script. Added SPEC_SUFFIX to all three (no-op for non-mtp). This also fixes the same latent bug for the existing qwen3.5-fp8-h100-sglang-mtp config.

perf-changelog

Combined entry for both new config keys.

Validation

• generate_sweep_configs.py test-config --config-keys minimaxm3-fp8-h200-vllm-mtp minimaxm3-fp8-h100-vllm-mtp generates 42 configs cleanly (scenario-trimmed max-model-len 2304 / 9472, all spec-decoding=mtp).
• bash -n passes on both scripts and all three modified launchers.
• Script-name resolution simulated: mtp → _mtp.sh, none → _h100.sh.

🤖 Generated with Claude Code

---

Note

Low Risk
Benchmark and CI launcher routing only; no changes to application runtime, auth, or data paths beyond new vLLM serve flags in test scripts.

Overview
Adds EAGLE3 speculative decoding (spec-decoding: mtp) benchmark coverage for MiniMax-M3 MXFP8 on H200 and H100, pairing MiniMaxAI/MiniMax-M3-MXFP8 with the Inferact/MiniMax-M3-EAGLE3 draft (3 tokens, drafter FLASH_ATTN).

New scripts minimaxm3_fp8_h200_mtp.sh and minimaxm3_fp8_h100_mtp.sh extend the non-MTP recipes with --speculative-config, spec-scaled CUDA graph capture, chat-templated serving benchmarks, and draft-model HF caching. H100 keeps extra DEP memory tuning (higher GMU, per-rank capture bounds).

nvidia-master.yaml gains minimaxm3-fp8-h200-vllm-mtp and minimaxm3-fp8-h100-vllm-mtp with concurrency search spaces trimmed vs non-MTP (H100 TP8-only, no DEP). perf-changelog.yaml documents both keys.

H100 launchers (launch_h100-cw.sh, -cr.sh, -dgxc-slurm.sh) now set SPEC_SUFFIX so mtp configs run *_h100_mtp.sh instead of silently using the non-MTP script (parity with H200 since #392).

^{Reviewed by Cursor Bugbot for commit 6f9ee6c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27453501412
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27453501412

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27453501412
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27453501412

[functionstackx]

/reuse-sweep-run

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27456399802
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=27456399802