Repository navigation
MXFP8 on the fine-grained machinery (Mxfp8Config) - #49780
Draft
IlyasMoutawwakil wants to merge 15 commits into
Draft
IlyasMoutawwakil wants to merge 15 commits into
IlyasMoutawwakil wants to merge 15 commits into
Conversation
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 11:08
a809602 to
41a1838
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 11:20
41a1838 to
9f645cd
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 13:33
9f645cd to
5547097
Compare
This was referenced Oct 20, 2026
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
2 times, most recently
from
October 20, 2026 15:30
b7efdec to
b6d0e7e
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 15:47
b6d0e7e to
41a7c02
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 16:04
41a7c02 to
120166b
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 16:25
120166b to
ce74ca4
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 16:43
ce74ca4 to
0b31e44
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 20, 2026 17:34
0b31e44 to
51c59a6
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
2 times, most recently
from
October 20, 2026 22:57
3dbb2ee to
c90b019
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
2 times, most recently
from
October 21, 2026 13:52
101228d to
851bff8
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
2 times, most recently
from
October 22, 2026 07:00
9246848 to
d09dea6
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 22, 2026 07:00
9246848 to
d09dea6
Compare
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
2 times, most recently
from
October 23, 2026 13:03
83b7244 to
51effe9
Compare
- Mxfp8Config / Mxfp8HfQuantizer: float8_e4m3fn weights with a UE8M0 scale per 32 (MiniMax-M3), W8A8, W8A4 or weight-only, in the plain, modelopt and compressed-tensors layouts; quant_method="mxfp8" routes here instead of the legacy FP8 path, and the readers learn MXFP8. - The FineGrainedFP8Config deprecation points at Mxfp8Config for a [1, 32] block with UE8M0 scales. - MiniMax-M3: its block_sparse_moe rename names the MoE block only, so a save does not reverse the dense and vision `.mlp.` too; its reverse-mapping test runs. - Tests: the MXFP8 suite and the shared tests that mix all four formats. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…essed-tensors exports load as Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fastest configuration runs the bf16 weights, so it loads `MiniMaxAI/MiniMax-M3`. The MXFP8 repo is named as the smaller option. The rows keep only the measured bf16 configurations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tester's config, widened so a 2-way split keeps whole swizzled tiles, replaces the hand-written copy. The fixture keeps sparse layers only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rewrite the comments, docstrings and docs prose this PR adds in short, plain sentences. Drop comments that restate the code and history or benchmark narration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… default Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ense layer A near-tie in the router scores flipped under sharded rounding, so the test compared routing, not sharding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… readers' PR Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IlyasMoutawwakil
force-pushed
the
fg-split/8-mxfp8
branch
from
October 23, 2026 13:35
51effe9 to
42b862c
Compare
Contributor
|
[For maintainers] Suggested jobs to run (before merge) run-slow: finegrained, finegrained_fp8, mxfp8 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What.
Mxfp8ConfigandMxfp8HfQuantizeradd FP8 weights with a UE8M0 scale per 32, as MiniMax-M3 ships them. They runW8A8 or weight-only.
tiny-random-MiniMaxM3-MXFP8is the first user.quant_method="mxfp8"routes here instead of the deprecatedFineGrainedFP8Configpath, which stays frozen.FineGrainedFP8Configdeprecation points atMxfp8Configfor a[1, 32]UE8M0 block.read_mxfp8reads MiniMax'signored_layersskip-list asmodules_to_not_convert, e.g.MiniMaxAI/MiniMax-M3-MXFP8. The deprecated config keeps its own alias.routing, so a near-tie in the router does not flip under sharding.
MIXED_PAIRS) adds MXFP8 pairs. One runs MXFP4 experts with MXFP8 activations (W4A8) besideweight-only MXFP8 dense layers. MXFP8 weights with MXFP4 activations (W8A4) are not supported.
FINEGRAINEDtest spec runs the tiny-checkpoint and load-path tests on a mixed plan. Its model is DeepSeek-V4,with block-FP8 layers and MXFP4 experts, and its fixtures exist since 🚨 MXFP4 on the fine-grained machinery, and DeepSeek-V4's FP8 + MXFP4 plan #49778. It sits here with the other spec
changes to the shared tester.
Depends on. #49779.
How tested.
tests/quantization/mxfp8and the shared fine-grained tests.The comments and docstrings that say why each change exists are there for readability. Feel free to ask for any of them to go if it's self-evident.
🤖 Generated with Claude Code