[NVIDIA] feat: MiniMax M3 Day 0 MTP (EAGLE3) support B200#1736
Conversation
EAGLE3 speculative decoding for MiniMax-M3 MXFP8 on B200, per the recipe spec_decoding feature (Inferact/MiniMax-M3-EAGLE3 draft head, 3 spec tokens, FLASH_ATTN). New minimaxm3_fp8_b200_mtp.sh (--speculative-config + --use-chat-template), minimaxm3-fp8-b200-vllm-mtp config (latency-end layouts), and launch_b200-dgxc.sh draft-head bind-mount. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you
PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
1 similar comment
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you
PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # .github/configs/nvidia-master.yaml # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27454032223 |
|
Shop closed for tonight! Please rest, claude! |
|
do not reopen else I will find you! |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=27454032223 |
|
starting an non-/goal PR here to prevent runway AI agents #1741 |
MiniMax-M3 MXFP8 EAGLE3 speculative-decoding sweep on B200 — the recipe's
spec_decodingfeature (https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3).minimaxm3-fp8-b200-vllm-mtpandbenchmarks/single_node/fixed_seq_len/minimaxm3_fp8_b200_mtp.sh.{"method": "eagle3", "model": "Inferact/MiniMax-M3-EAGLE3", "num_speculative_tokens": 3, "attention_backend": "FLASH_ATTN"}.--use-chat-templateon the benchmark (mandatory for*_mtp.sh— EAGLE acceptance collapses on raw random prompts).launch_b200-dgxc.sh(DRAFT_MODEL_PATH+EXTRA_MOUNTS).Builds on the merged non-MTP B200 recipe (#1723). Sweep results + acceptance-rate/speedup vs STP to follow.
🤖 Generated with Claude Code
Note
Low Risk
Benchmark and runner wiring only; no changes to auth, production serving, or core inference libraries.
Overview
Adds MiniMax-M3 MXFP8 on B200 with EAGLE3 speculative decoding to the NVIDIA benchmark matrix: new config key
minimaxm3-fp8-b200-vllm-mtpand scriptminimaxm3_fp8_b200_mtp.sh, following the vLLM recipe’sspec_decodingfeature (eagle3,Inferact/MiniMax-M3-EAGLE3, 3 speculative tokens,FLASH_ATTN).The benchmark script starts vLLM with
--speculative-config, scales CUDA graph capture for spec-decoding token volume, extends engine ready timeout for large weights + draft head, and runs serving with--use-chat-templateso EAGLE acceptance matches chat-trained behavior. Sweeps focus on low/mid concurrency (TP8, TP4, TP8+EP8) for 1k/1k and 8k/1k fixed-seq scenarios.launch_b200-dgxc.shstages the EAGLE3 draft beside main MiniMax-M3 weights (DRAFT_MODEL_PATH), bind-mounts it viaEXTRA_MOUNTS, and appends those mounts to the Slurm container launch.perf-changelog.yamlrecords the new config for sweep triggers.Reviewed by Cursor Bugbot for commit 7b5bd34. Bugbot is set up for automated code reviews on this repo. Configure here.