Cherry-pick: Reduce Python and Nuget GPU package size (#26002) - #26087
Merged
Conversation
### Description The package size limit for PyPI and Nuget are: - python package size under 300MB - Nuget package size under 250MB To meet the size limit, this PR firstly removes some old GPU arch support in CMAKE_CUDA_ARCHITECTURE. Secondly, it removes the FPA_INTB_GEMM support in Linux Python wheel. #### Python wheel | OS | cmake_cuda_architecture | CUDA kernel removal |Package size | Under 300MB| |---------|--------------------------------------------------------|-|-------------|---| | Linux | 60-real;70-real;75-real;80-real;86-real;90a-real;90a-virtual | |341 MB |No (original)| | Linux | 70-real;75-real;80-real;86-real;90a-real;90a-virtual | | 329 MB |No| | Linux | 75-real;80-real;86-real;90a-real;90a-virtual | |319 MB |No| | Linux | 80-real;86-real;90a-real;90a-virtual | |304 MB |No| | Linux | 60-real;70-real;75-real;80-real;86-real;90a-real;90a-virtual. | FPA_INTB_GEMM|287 MB |Yes| | Windows | 52-real;61-real;75-real;86-real;89-real;90a-virtual | | 272 MB |Yes (original)| #### Nuget | OS | cmake_cuda_architecture | CUDA kernel removal |Package size |Under 250MB| |---------|--------------------------------------------------------|---|--------------|---| | Linux | 60-real;70-real;75-real;80-real;90a-real;90a-virtual | |276 MB |No (original)| | Linux | 75-real;80-real;90a-real;90a-virtual | |253 MB |No| | Linux | 60-real;70-real;75-real;80-real;90a-real;90a-virtual |FPA_INTB_GEMM| 230 MB |Yes| | Windows | 52-real;61-real;75-real;86-real;89-real;90a-virtual || 264 MB |No (original)| | Windows | 61-real;75-real;86-real;89-real;90a-virtual || 254 MB |No| | Windows | 75-real;86-real;89-real;90a-virtual || 242 MB |Yes| ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
Changming Sun (snnn)
changed the title
Changming Sun (snnn)
requested review from
Chi Lo (chilo-ms) and
Tianlei Wu (tianleiwu)
September 18, 2025 22:21
Contributor
|
We need to cherry-pick this as well in order to disable FPA_INTB_GEMM kernel. |
### Description Add a build flag to enable/disable mixed gemm cutlass kernel. To disable the kernel, you can append the following at the end of build command line: `--cmake_extra_defines onnxruntime_USE_FPA_INTB_GEMM=OFF` ### Motivation and Context FpA IntB Gemm need a lot of time to compile. With such option, developer can speed up the build especially on build machine with limited memory.
Chi Lo (chilo-ms)
approved these changes
Sep 19, 2025
Tianlei Wu (tianleiwu)
approved these changes
Sep 19, 2025
This was referenced Sep 19, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The package size limit for PyPI and Nuget are:
To meet the size limit,
this PR firstly removes some old GPU arch support in
CMAKE_CUDA_ARCHITECTURE.
Secondly, it removes the FPA_INTB_GEMM support in Linux Python wheel.
Python wheel
| OS | cmake_cuda_architecture | CUDA kernel removal |Package size |
Under 300MB|
|---------|--------------------------------------------------------|-|-------------|---|
| Linux | 60-real;70-real;75-real;80-real;86-real;90a-real;90a-virtual |
|341 MB |No (original)|
| Linux | 70-real;75-real;80-real;86-real;90a-real;90a-virtual | | 329
MB |No|
| Linux | 75-real;80-real;86-real;90a-real;90a-virtual | |319 MB |No|
| Linux | 80-real;86-real;90a-real;90a-virtual | |304 MB |No|
| Linux | 60-real;70-real;75-real;80-real;86-real;90a-real;90a-virtual.
| FPA_INTB_GEMM|287 MB |Yes|
| Windows | 52-real;61-real;75-real;86-real;89-real;90a-virtual | | 272
MB |Yes (original)|
Nuget
| OS | cmake_cuda_architecture | CUDA kernel removal |Package size
|Under 250MB|
|---------|--------------------------------------------------------|---|--------------|---|
| Linux | 60-real;70-real;75-real;80-real;90a-real;90a-virtual | |276 MB
|No (original)|
| Linux | 75-real;80-real;90a-real;90a-virtual | |253 MB |No|
| Linux | 60-real;70-real;75-real;80-real;90a-real;90a-virtual
|FPA_INTB_GEMM| 230 MB |Yes|
| Windows | 52-real;61-real;75-real;86-real;89-real;90a-virtual || 264
MB |No (original)|
| Windows | 61-real;75-real;86-real;89-real;90a-virtual || 254 MB |No|
| Windows | 75-real;86-real;89-real;90a-virtual || 242 MB |Yes|
Motivation and Context