perf(moe): re-validate the CUDA fused-MoE Dff cap sweep on MLX 0.32.1 (GB10) #712

Problem / Background

• The backend-aware fused decode-MoE Dff cap (Metal 4096 / CUDA 8192, MLXCEL_FUSED_MOE_MAX_DFF override) shipped in PR perf(cuda/moe): backend-aware default for the fused decode-MoE Dff threshold #643 with a GB10 sweep measured on MLX pin e9463bb (perf(cuda/moe): backend-specific default for the fused decode-MoE Dff threshold #626). The pin has since advanced to 0.32.1 (chore(mlx): bump upstream MLX pin to 0.32.1 #703/chore: bump MLX pin to 0.32.1 (#703) #704). Issue perf(moe): backend-aware fused-MoE Dff cap (CUDA crossover) and dispatch heuristic #330 / PR perf(moe): record CUDA fused-MoE Dff cap provenance and decline test #711 recorded the provenance and the pending re-validation in docs/benchmark_results/fused-moe-decode-kernel-design.md (2026-07-09 addendum) because no CUDA/GB10 hardware was reachable in that session.
• The CUDA default (8192) is the conservative floor of the measured crossover (~8000 by ratio interpolation: phi-3.5-moe 6400 +5%, llama-4-scout 8192 break-even, mixtral 14336 -1.2%), so pin drift is unlikely to have flipped it, but the acceptance bar of perf(moe): backend-aware fused-MoE Dff cap (CUDA crossover) and dispatch heuristic #330 ("justified by current data") wants a sweep on current binaries.

Task

• On a GB10 (or equivalent CUDA) box with the current main (MLX 0.32.1), run the harness recorded in the 2026-07-09 addendum of docs/benchmark_results/fused-moe-decode-kernel-design.md: mlxcel-bench-decode, prompt "Hello, how are you today?", 100 decode tokens after a 20-token warmup, median of 3, forcing MLXCEL_FUSED_MOE_MAX_DFF=1 vs =20000, sweeping phi-3.5-moe (Dff 6400), llama-4-scout (8192), mixtral (14336), with lfm2-8b-a1b (1792) as control.
• If the 0.32.1 crossover lands away from ~8000, update FUSED_MOE_MAX_DFF_CUDA in src/models/switch_layers.rs and the design-doc section together.

Acceptance criteria

• Sweep numbers on MLX 0.32.1 recorded in the design doc (hardware, pin, per-model tok/s both arms).
• FUSED_MOE_MAX_DFF_CUDA confirmed or re-tuned from that data.
• The 2026-07-09 "pending re-validation" note replaced by the result.

References

• Issue perf(moe): backend-aware fused-MoE Dff cap (CUDA crossover) and dispatch heuristic #330, PR perf(moe): record CUDA fused-MoE Dff cap provenance and decline test #711, PR perf(cuda/moe): backend-aware default for the fused decode-MoE Dff threshold #643, docs/benchmark_results/fused-moe-decode-kernel-design.md.