You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Metal] NVFP4 tensor-scale (global_scale) not implemented — throws runtime error on Metal backend #3911
PR #3022 added per-tensor scale (global_scale) support for NVFP4 quantization on CUDA and CPU backends. However, the Metal backend explicitly rejects it:
if (mode_ == QuantizationMode::Nvfp4 &&
static_cast<int>(inputs.size()) > base_size) {
throwstd::runtime_error(
"[QQMatmul] Global scale (tensor-scale nvfp4) is not supported ""on the Metal backend.");
}
Impact
Without tensor-scale support, NVFP4 on Metal has ~137x less dynamic range than NVIDIA Blackwell (unsigned UE4M3 vs signed E4M3 scales), which was the original concern in #2962. The fix (PR #3022) landed for CUDA/CPU but Metal users still hit the error.
This blocks NVFP4 quantization for Apple Silicon users running MoE models (DeepSeek-V3/V4, GLM-5.1, etc.) where tensor-scale is critical for expert routing accuracy.
Request
Please add Metal kernel support for consuming the global_scale inputs in the qqmm / fp_quantized Metal kernel paths, matching the CUDA/CPU behavior from PR #3022.
#3757 has added some basic support for global scale in QQMatmul, but at the moment not all fast kernels get the support and we are still working on improving things.
Activity
zcbenz commentedon Aug 5, 2026
#3757 has added some basic support for global scale in QQMatmul, but at the moment not all fast kernels get the support and we are still working on improving things.