Skip to content

MXFP8 on the fine-grained machinery (Mxfp8Config) - #49780

Draft
IlyasMoutawwakil wants to merge 15 commits into
fg-split/7-nvfp4from
fg-split/8-mxfp8
Draft

IlyasMoutawwakil wants to merge 15 commits into
fg-split/7-nvfp4from
fg-split/8-mxfp8

Conversation

@IlyasMoutawwakil

@IlyasMoutawwakil IlyasMoutawwakil commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

🤗 Transformers CI

What.

  • Mxfp8Config and Mxfp8HfQuantizer add FP8 weights with a UE8M0 scale per 32, as MiniMax-M3 ships them. They run
    W8A8 or weight-only. tiny-random-MiniMaxM3-MXFP8 is the first user.
  • quant_method="mxfp8" routes here instead of the deprecated FineGrainedFP8Config path, which stays frozen.
  • The FineGrainedFP8Config deprecation points at Mxfp8Config for a [1, 32] UE8M0 block.
  • read_mxfp8 reads MiniMax's ignored_layers skip-list as modules_to_not_convert, e.g.
    MiniMaxAI/MiniMax-M3-MXFP8. The deprecated config keeps its own alias.
  • MiniMax-M3-VL docs point at the public checkpoints. Its parallel fixture comes from its model tester and pins the
    routing, so a near-tie in the router does not flip under sharding.
  • The shared tests extend their format lists with MXFP8.
  • The mixed-plan grid (MIXED_PAIRS) adds MXFP8 pairs. One runs MXFP4 experts with MXFP8 activations (W4A8) beside
    weight-only MXFP8 dense layers. MXFP8 weights with MXFP4 activations (W8A4) are not supported.
  • The FINEGRAINED test spec runs the tiny-checkpoint and load-path tests on a mixed plan. Its model is DeepSeek-V4,
    with block-FP8 layers and MXFP4 experts, and its fixtures exist since 🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan #49778. It sits here with the other spec
    changes to the shared tester.

Depends on. #49779.

How tested. tests/quantization/mxfp8 and the shared fine-grained tests.

The comments and docstrings that say why each change exists are there for readability. Feel free to ask for any of them to go if it's self-evident.

🤖 Generated with Claude Code

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@IlyasMoutawwakil
IlyasMoutawwakil removed this pull request from stack #49781 October 20, 2026 13:41
@IlyasMoutawwakil
IlyasMoutawwakil force-pushed the fg-split/8-mxfp8 branch 2 times, most recently from b7efdec to b6d0e7e Compare October 20, 2026 15:30
@IlyasMoutawwakil
IlyasMoutawwakil removed this pull request from stack #49788 October 20, 2026 16:25
@IlyasMoutawwakil
IlyasMoutawwakil force-pushed the fg-split/8-mxfp8 branch 2 times, most recently from 3dbb2ee to c90b019 Compare October 20, 2026 22:57
IlyasMoutawwakil and others added 14 commits October 23, 2026 13:35
- Mxfp8Config / Mxfp8HfQuantizer: float8_e4m3fn weights with a UE8M0 scale
  per 32 (MiniMax-M3), W8A8, W8A4 or weight-only, in the plain, modelopt and
  compressed-tensors layouts; quant_method="mxfp8" routes here instead of the
  legacy FP8 path, and the readers learn MXFP8.
- The FineGrainedFP8Config deprecation points at Mxfp8Config for a [1, 32]
  block with UE8M0 scales.
- MiniMax-M3: its block_sparse_moe rename names the MoE block only, so a save
  does not reverse the dense and vision `.mlp.` too; its reverse-mapping test
  runs.
- Tests: the MXFP8 suite and the shared tests that mix all four formats.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…essed-tensors exports load as

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fastest configuration runs the bf16 weights, so it loads `MiniMaxAI/MiniMax-M3`.
The MXFP8 repo is named as the smaller option. The rows keep only the measured bf16 configurations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tester's config, widened so a 2-way split keeps whole swizzled tiles, replaces the hand-written copy.
The fixture keeps sparse layers only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rewrite the comments, docstrings and docs prose this PR adds in short, plain sentences.
Drop comments that restate the code and history or benchmark narration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… default

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ense layer

A near-tie in the router scores flipped under sharded rounding, so the test compared routing, not sharding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… readers' PR

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: finegrained, finegrained_fp8, mxfp8

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants