Skip to content

[AMD] perf: enable MiniMax M3 CUDA graphs on MI355X #1754

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cquil11 merged 3 commits into from
Jun 14, 2026
Merged

[AMD] perf: enable MiniMax M3 CUDA graphs on MI355X #1754

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9257ae1
perf: enable MI355X MiniMax M3 CUDA graphs
cquil11 Jun 14, 2026
9e2258b
fix: disable MI355X MiniMax M3 prefix caching
cquil11 Jun 14, 2026
26bbb94
Merge branch 'main' into codex/minimaxm3-mi355x-cudagraph
cquil11 Jun 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion benchmarks/single_node/fixed_seq_len/minimaxm3_fp8_mi355x.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ fi

SERVER_LOG=/workspace/server.log
export VLLM_ENGINE_READY_TIMEOUT_S=3600
export VLLM_USE_BREAKABLE_CUDAGRAPH=0

if [ "${EVAL_ONLY}" = "true" ]; then
setup_eval_context
Expand All @@ -52,11 +53,11 @@ set -x
vllm serve "$MODEL" --port "$PORT" \
"${PARALLEL_ARGS[@]}" \
--block-size 128 \
--no-enable-prefix-caching \
--language-model-only \
--max-model-len "$MAX_MODEL_LEN" \
--kv-cache-dtype fp8 \
--attention-backend TRITON_ATTN \
--enforce-eager \
--tool-call-parser minimax_m3 \
--reasoning-parser minimax_m3 \
--enable-auto-tool-choice > "$SERVER_LOG" 2>&1 &
Expand Down
6 changes: 6 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3768,3 +3768,9 @@
- "Run the MiniMax-M3 MXFP8 MI300X EAGLE3 MTP recipe with CUDA graphs instead of --enforce-eager"
- "Drop --enforce-eager and set VLLM_USE_BREAKABLE_CUDAGRAPH=0 (matching the non-MTP MI300X recipe, #1750), which avoids the M3-decode breakable-cudagraph path that previously forced eager execution"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1756

- config-keys:
- minimaxm3-fp8-mi355x-vllm
description:
- "Enable CUDA graphs for MiniMax-M3 MXFP8 on MI355X, set VLLM_USE_BREAKABLE_CUDAGRAPH=0, and disable prefix caching"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1754
Loading