Add MiniMax-M3 NVFP4 B300 single-node vLLM benchmark (EAGLE3 spec decode)

[Ankur-singh]

Adds the minimaxm3-fp4-b300-vllm-mtp config: MiniMax-M3 NVFP4 (nvidia/MiniMax-M3-NVFP4) single-node aggregated vLLM on B300 with EAGLE3 speculative decoding (spec-decoding: mtp, 3 draft tokens via Inferact/MiniMax-M3-EAGLE3).

• Config: nvidia-master.yaml entry (fp4 / vllm / runner b300), every search-space row spec-decoding: mtp; sweeps tp 4/8 with and without EP and dp-attn at 1k1k and 8k1k, conc 1-512.
• Recipe: benchmarks/single_node/fixed_seq_len/minimaxm3_fp4_b300_mtp.sh — overlays vllm-project/vllm PR #46380 (MiniMax-M3 modelopt NVFP4 support, commit 6c08558) before serve; --speculative-config EAGLE3; --block-size 128 (MSA), --language-model-only; prompts routed through the chat template.
• Weights: target pre-staged read-only at /scratch/models/MiniMax-M3-NVFP4 — added MiniMax-M3-NVFP4 to the STAGED_MODELS allow-list in launch_b300-nv.sh. The EAGLE3 draft (not staged) is downloaded to the writable /data/models rather than next to the read-only target.
• perf-changelog entry appended.

[github-actions]

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

---

感谢你的贡献！对于 vLLM 与 SGLang，请确保你的 recipe 与官方 vLLM recipes 和/或 SGLang cookbook 保持一致

• https://docs.sglang.io/cookbook/intro
• https://recipes.vllm.ai/

如果不一致，请先创建一个 PR，之后我们才能将你的单节点 PR 合并到 master 分支。让我们确保文档保持一流水准，使整个 ML 社区都能从你的辛勤工作中受益！谢谢

• https://github.com/vllm-project/recipes
• https://github.com/sgl-project/sglang/tree/main/docs_new

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。如果选择重新运行失败的任务，PR 作者有责任确保其最终通过。参见 GitHub 关于重新运行失败任务的文档：https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

一般而言，PR 作者应先向相应公司的 CODEOWNERS 请求审阅并获得 PR 批准，然后再请求核心维护者审阅。

如需更多帮助，PR 作者可通过 Slack 联系核心维护者。

[claude]

🟡 The 3-file vLLM patch overlay loop (lines 25-32) runs curl -fsSL ... -o without set -e and without || exit 1, so a transient 5xx / rate-limit / commit-reachability failure on file #2 (modelopt.py) or file #3 (flashinfer_utils.py) returns non-zero and the loop continues silently. The post-patch python3 -c only imports TrtLlmNvFp4ExpertsModular from file #1, so a partial patch is undetected and vllm serve boots on the comment's "unsupported path". Match the sister recipe minimaxm3_fp8_b300_mtp.sh (which wraps its patch step with || { echo ...; exit 1; }) — either add || exit 1 to the curl, or assert all three modules import in the verification.

Extended reasoning...

The defect. Lines 25-32 fetch three vLLM source files from a pinned vllm-project/vllm commit and overwrite them in the installed package:

for f in \
  model_executor/layers/fused_moe/experts/trtllm_nvfp4_moe.py \
  model_executor/layers/quantization/modelopt.py \
  model_executor/layers/quantization/utils/flashinfer_utils.py
do
  curl -fsSL "https://raw.githubusercontent.com/vllm-project/vllm/6c08558/vllm/${f}" -o "${VLLM_DIR}/${f}"
done
python3 -c "from vllm.model_executor.layers.fused_moe.experts.trtllm_nvfp4_moe import TrtLlmNvFp4ExpertsModular; print('[nvfp4-patch] OK')"

The script has no set -e (set -x later is just command tracing) and no || exit 1 inside the loop. With curl -f, an HTTP 4xx/5xx exits curl with status 22 and leaves the target file untouched — the original installed copy stays in place.

Why the validation doesn't catch it. The post-loop python3 -c imports only TrtLlmNvFp4ExpertsModular from trtllm_nvfp4_moe.py (file #1). If file #2 (modelopt.py) or file #3 (flashinfer_utils.py) fails to download, that import still succeeds, the [nvfp4-patch] OK line prints, and the script proceeds to vllm serve with two unpatched modules. Per the script's own preamble: "without it vLLM does not recognise the NVFP4 quant config and falls back to an unsupported path" — exactly the failure modeled by files #2/#3 being unpatched.

Step-by-step proof.

1. curl -fsSL for trtllm_nvfp4_moe.py succeeds → file [NVIDIA] Add TRT-LLM 70B FP8 via slurm #1 overwritten.
2. curl -fsSL for modelopt.py hits a transient 503 from raw.githubusercontent.com → curl exits 22, no write to ${VLLM_DIR}/.../modelopt.py, original vLLM copy retained.
3. Bash loop sees no set -e, no || exit 1 → continues to file [NVIDIA] update vllm b200 image. TODO: add logic for docker runner. #3.
4. curl -fsSL for flashinfer_utils.py succeeds → file [NVIDIA] update vllm b200 image. TODO: add logic for docker runner. #3 overwritten.
5. python3 -c imports from trtllm_nvfp4_moe (the file that WAS patched). Import succeeds, prints [nvfp4-patch] OK.
6. vllm serve boots with the new trtllm_nvfp4_moe.py calling into an unpatched modelopt.py → NVFP4 quant config not recognized, fallback path triggers, benchmark fails opaquely at serve/inference time instead of at the patch step.

Why this is real. Verified that the script contains no set -[eE] or set -o errexit; the only set -e in benchmark_lib.sh is scoped to run_agentic_replay_and_write_outputs and doesn't propagate to callers. Verified curl 8.x behavior: -f -o file against a 404/5xx exits non-zero without writing the file. Pinned commit hashes on a stable CDN make this uncommon, but not zero — raw.githubusercontent.com does have transient 5xx windows.

Repo convention. The sister script benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_b300_mtp.sh:33 already gates its patch step with python3 - <<PYEOF || { echo ...; exit 1; } — explicit fail-fast. This recipe should match.

Fix. Minimal:

curl -fsSL "https://raw.githubusercontent.com/vllm-project/vllm/6c08558/vllm/${f}" -o "${VLLM_DIR}/${f}" || exit 1

Or assert the other two modules in the verification:

python3 -c "from vllm.model_executor.layers.fused_moe.experts.trtllm_nvfp4_moe import TrtLlmNvFp4ExpertsModular; from vllm.model_executor.layers.quantization import modelopt; from vllm.model_executor.layers.quantization.utils import flashinfer_utils; print('[nvfp4-patch] OK')"

(The second form only catches import-breaking download failures; the first is the strictly safer fix and matches the fp8 twin.)

[claude]

🟡 The perf-changelog entry for minimaxm3-fp4-b300-vllm-mtp ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX — the literal XXX placeholder was never filled in. Every other recent entry links to a real PR number (e.g. /pull/1927 directly above); this should be /pull/1929 to match this PR.

Extended reasoning...

What the bug is. perf-changelog.yaml line 4194 was added as part of this PR with the value pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. The XXX is a literal placeholder string that the author clearly intended to replace with the actual PR number before pushing, but didn't. Every other recent entry in the file links to a real PR — the immediately preceding entry uses /pull/1927, and earlier ones use /pull/1762, /pull/1865, etc. This is a documentation/metadata oversight.\n\nHow it manifests. Anyone clicking the changelog link for the minimaxm3-fp4-b300-vllm-mtp entry after merge will hit GitHub's 404 page for /pull/XXX rather than landing on the PR that introduced the config. Tooling that walks the changelog looking up PRs (e.g. cross-referencing config additions back to their PR for release notes) will fail to resolve the placeholder.\n\nThe specific code path. In the diff applied to perf-changelog.yaml, the new YAML block ending at line 4194 contains:\n\nyaml\n- config-keys:\n - minimaxm3-fp4-b300-vllm-mtp\n description:\n - "Add MiniMax-M3 NVFP4 ..."\n ...\n pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX\n\n\nThe XXX is verbatim text, not a template variable that any downstream tool would substitute.\n\nWhy existing code doesn't prevent it. The changelog is hand-edited YAML; there is no schema validator or CI step that pattern-matches pr-link against /pull/\d+ to catch placeholders. The PR number isn't knowable at commit time (you only learn it after opening the PR), so it has to be filled in as an amendment — exactly the step that was skipped here.\n\nStep-by-step proof.\n1. Open perf-changelog.yaml after this PR merges.\n2. Scroll to the last entry — config minimaxm3-fp4-b300-vllm-mtp.\n3. Read line 4194: pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX.\n4. Click / curl the link → GitHub returns 404 (/pull/XXX is not a valid PR id).\n5. Compare with line 4185 directly above: pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1927 — resolves correctly.\n\nHow to fix. Replace XXX with 1929 on line 4194 so the entry reads pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1929, matching this PR's number.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=28145405377
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=28145405377