NVFP4 MoE experts are dequantized without weight_global_scale, silently #48371

What happens

Loading a compressed-tensors MoE checkpoint whose routed experts are stored per expert dequantizes them without their per-tensor weight_global_scale when the scheme is NVFP4 (tensor_group), so the experts come back scaled by a wrong constant — no exception, and the fused parameter is filled so missing_keys stays empty.

huggingface/transformers@323f24345ed92f88e68a14aeef7db78ee2b52475 (5.16.0.dev0)
compressed-tensors==0.18.0, torch==2.13.0+cpu, python 3.11.15, Linux x86_64

Reproduction

Standalone — compressed_tensors + transformers only, no checkpoint download. It drives DecompressExperts.convert with exactly the tensors update_weight_conversions collects today, and with the ones the checkpoint actually stores.

"""Standalone repro: NVFP4 MoE experts are dequantized without weight_global_scale."""
import types
import torch
from compressed_tensors.compressors import BaseCompressor
from compressed_tensors.compressors.format import infer_module_format
from compressed_tensors.quantization import (
    QuantizationArgs, QuantizationScheme, QuantizationStrategy, QuantizationType)
from compressed_tensors.quantization.utils.helpers import calculate_qparams, generate_gparam
from transformers.integrations.compressed_tensors import DecompressExperts

GROUP = 16
args = QuantizationArgs(num_bits=4, type=QuantizationType.FLOAT, group_size=GROUP,
                        strategy=QuantizationStrategy.TENSOR_GROUP, symmetric=True)
scheme = QuantizationScheme(targets=["Linear"], weights=args)
compressor = BaseCompressor.get_value_from_registry(infer_module_format(torch.nn.Linear, scheme))

torch.manual_seed(0)
packed, scales, shapes, global_scales, reference = [], [], [], [], []
for expert in range(3):                      # differing magnitudes -> differing global scales
    w = (torch.randn(32, 64) * (expert + 1) * 3.0).to(torch.bfloat16)
    gs = generate_gparam(w.min(), w.max())
    grouped = w.reshape(w.shape[0], -1, GROUP)
    sc, _ = calculate_qparams(grouped.min(dim=-1).values, grouped.max(dim=-1).values,
                              args, global_scale=gs)
    c = compressor.compress({"weight": w, "weight_scale": sc, "weight_global_scale": gs}, scheme)
    packed.append(c["weight_packed"]); scales.append(c["weight_scale"])
    global_scales.append(c["weight_global_scale"])
    shapes.append(torch.tensor(list(w.shape))); reference.append(w.float())

quantizer = types.SimpleNamespace(compressor=types.SimpleNamespace(
    quantization_config=types.SimpleNamespace(config_groups={"group_0": scheme})))
op = DecompressExperts(quantizer, scheme=scheme)

# What `update_weight_conversions` collects today: packed / scale / shape. No global scale.
today = op.convert({"experts.gate_proj.weight_packed": packed,
                    "experts.gate_proj.weight_scale": scales,
                    "experts.gate_proj.weight_shape": shapes}, [], [])
# What the checkpoint actually stores.
faithful = op.convert({"experts.gate_proj.weight_packed": packed,
                       "experts.gate_proj.weight_scale": scales,
                       "experts.gate_proj.weight_shape": shapes,
                       "experts.gate_proj.weight_global_scale": global_scales}, [], [])

ref = torch.stack(reference)
t = today["experts.gate_proj.weight_packed"].float()
f = faithful["experts.gate_proj.weight_packed"].float()
rel = lambda a, b: (a - b).norm().item() / b.norm().item()

print("per-expert global scales :", [round(g.item(), 1) for g in global_scales])
print(f"relative L2 vs reference  : today={rel(t, ref):9.3f}   with the global scale={rel(f, ref):.3f}")
print(f"max |weight|              : today={t.abs().max():9.2f}   with the global scale={f.abs().max():.2f}")
print("no exception was raised, and the fused parameter is filled (missing_keys stays empty)")

Actual output:

per-expert global scales : [218.0, 109.5, 79.5]
relative L2 vs reference  : today=  103.484   with the global scale=0.099
max |weight|              : today=  2688.00   with the global scale=33.75
no exception was raised, and the fused parameter is filled (missing_keys stays empty)

Expected: torch.testing.assert_close(decompressed, faithful_per_expert_decompression) — i.e. relative L2 against the reference weights should be the NVFP4 quantization error (~0.1), not ~103.

Root cause

quantizer_compressed_tensors.py#L210-L212 — update_weight_conversions derives the expert conversion sources from the base .weight patterns as _packed$ / _scale$ / _shape$. NVFP4 stores a fourth per-module tensor, weight_global_scale, which is therefore never collected; and compressed_tensors.py#L108-L113 gives DummyModule no slot for one even if it had been.

Why it is silent rather than a crash. NVFP4Compressor.decompress reads global_scale = state_dict.get("weight_global_scale", None) and passes it to dequantize, which treats None as "no global scaling". This is the difference from #47430, whose missing zero_point raises Asymmetric quant requires zero-point values. The global scale is per tensor (scale_data.max * quant_data.max / max_abs), so each expert is off by a different factor and the relative magnitudes between experts are destroyed too.

Not a regression — it never worked. DummyModule has had the same three-tensor signature since DecompressExperts was introduced in #45630 (09835700, 2026-07-03); weight_global_scale was never carried.

Blast radius

Every scheme that reaches the changed line, executed against this checkout:

Candidate fixes

1. Gate on the strategy (narrow). Collect weight_global_scale$ only when weights.strategy == "tensor_group" — exactly when initialize_module_for_quantization registers the parameter — and thread an optional global_scale through DummyModule. Mirrors fix(compressed_tensors): collect zero_point for asymmetric MoE expert decompression #47430's shape. Leaves the general problem: the next companion tensor a scheme adds fails the same way.
2. Derive the whole set (generic). Enumerate the per-module tensors the scheme actually registers instead of hardcoding suffixes, which would subsume fix(compressed_tensors): collect zero_point for asymmetric MoE expert decompression #47430's zero_point fix in one place. Larger change, and it touches the FP8 branch's semantics.
3. Make it loud (different repo). Have NVFP4Compressor.decompress raise when a tensor_group scheme has no global scale, so the omission can never be silent again. Belongs in compressed-tensors, and is complementary rather than an alternative.

Recommendation: 1, because the blast-radius table shows every non-tensor_group row is byte-identical under it, so it carries no regression risk for existing checkpoints — with 3 as a follow-up upstream in compressed-tensors.

Question

Would you prefer the narrow strategy-gated fix (option 1, mirroring #47430), or a single generic derivation (option 2) that enumerates the scheme's registered per-module tensors and would subsume #47430's zero_point fix as well?

Prior work checked

gh pr list --repo huggingface/transformers --state open --search "compressed_tensors MoE experts"
gh pr list --repo huggingface/transformers --state open --search "global_scale"
gh pr list --repo huggingface/transformers --state open --search "nvfp4"
gh api "search/issues?q=repo:huggingface/transformers+weight_global_scale+in:title,body"
gh api "search/issues?q=repo:huggingface/transformers+DecompressExperts"
gh api "search/issues?q=repo:huggingface/transformers+update_weight_conversions"

→ No open issue or PR covers weight_global_scale (total_count: 0 for the weight_global_scale search). The closest is #47430 (open), which fixes the same two functions for the asymmetric zero_point — a different tensor, a different scheme, and it crashes rather than corrupting silently. Adjacent but distinct: #47956 (weight_shape sharded away under TP/EP), #47315 (non-32-divisible bit widths), #47407 (run_compressed=False random init, different code path).