Repository navigation
[CUDA] Qwen3.6-35B-A3B Throughput Optimization #28987
Description
This is used to track the progress of Qwen3.6-35B-A3B Throughput Optimization.
Related PRs:
For Qwen 3.6:
-
olive-recipes
Add Qwen3.6-35B-A3B MoE VLM recipe (CUDA + CPU) -
onnxruntime-genai
Fix CUDA QMoE INT4 export for Qwen3.5/3.6 MoE models
Qwen3.6 MTP onnxruntime-genai#2218 -
cuda op / kernels:
[CUDA] Optimize QMoE SoftmaxTopK router for small-batch decode #28980
[CUDA] Add decode-optimized LinearAttention (GatedDeltaNet) kernels #28985
[CUDA] Add decode (M=1) GEMV fast path to MatMul #28986 (Not needed if we have shared expert optimization like below)
[CUDA] QMoE support shared experts #29028
[CUDA] QMoE GEMV fast path for batch-1 decode #29038 (Need to extend to block quantization)
Use user-supplied external initializer in place when already on the planned device #29013
[CUDA] Add NVFP4 native FP4xFP4 prefill for QMoE on SM120 #29824
[CUDA] Add MatMulBlockQuantizedFp4Weight contrib operator #29818
Related issues:
AddExternalInitializers copies device (GPU) OrtValues per session instead of using them in place
Activity
ShirasawaSama commentedon Jun 28, 2026
Wow, I’ve also been working on optimizing the ORT deployment for the Qwen3 series models recently. Thanks for your PR. If you need help, I’d be happy to pitch in with some of the simpler tasks.
microsoft-github-policy-service commentedon Aug 22, 2026
Applying stale label due to no activity in 30 days. Please react to the issue by assigning it or labeling it as 'contributions welcome', 'documentation', 'feature request', 'regression', or 'no stale'.
microsoft-github-policy-service commentedon Sep 23, 2026
Applying stale label due to no activity in 30 days. Please react to the issue by assigning it or labeling it as 'contributions welcome', 'documentation', 'feature request', 'regression', or 'no stale'.
microsoft-github-policy-service commentedon Sep 23, 2026
Closing issue due to no activity in 30 days