[Build] update cuda 13 package: fatbin compress mode and cuda archs - #26516
Merged
Conversation
Tianlei Wu (tianleiwu)
marked this pull request as draft
November 6, 2025 22:40
Contributor
|
Yes please, nightly feeds are not available publicly anymore and nobody raised this issue and all cuda 13 builds hardcode cuda 12 cublas and other libraries |
Contributor
|
Also this pr is required to fix cuda 13 builds #26518 |
Tianlei Wu (tianleiwu)
marked this pull request as ready for review
November 7, 2025 20:13
Tianlei Wu (tianleiwu)
pushed a commit
that referenced
this pull request
Nov 9, 2025
## Description Fixes runtime library loading failures when building with CUDA 13 by replacing hardcoded CUDA 12 references with dynamic version detection. Related to #26516 which updates CUDA 13 build pipelines, but this PR fixes the Python runtime code that was still hardcoded to CUDA 12. ## Problem The build system correctly detects CUDA 13 via CMake, but the runtime Python code had CUDA 12 hardcoded in multiple locations, causing "CUDA 12 not found" errors on CUDA 13 systems. ## Solution Modified onnxruntime/__init__.py and setup.py to dynamically use the detected CUDA version instead of hardcoded "12" strings. ## Changes - Dynamic CUDA version extraction from build info - Library paths now use f-strings with cuda_major_version - Added CUDA 13 support to extras_require and dependency exclusions - Fixed TensorRT RTX package to use correct CUDA version - Updated version validation to accept CUDA 12+ - Fixed PyTorch compatibility checks to compare versions dynamically ## Impact - CUDA 13 builds now load correct libraries - Backward compatible with CUDA 12 - Forward compatible with future CUDA versions ## Testing Verified with CUDA 13.0 build that library paths resolve correctly and preload_dlls() loads CUDA 13 libraries without errors.
Tianlei Wu (tianleiwu)
enabled auto-merge (squash)
November 10, 2025 03:30
Changming Sun (snnn)
approved these changes
Nov 10, 2025
Rohanjames1997
pushed a commit
to Rohanjames1997/onnxruntime
that referenced
this pull request
Dec 4, 2025
…t#26518) ## Description Fixes runtime library loading failures when building with CUDA 13 by replacing hardcoded CUDA 12 references with dynamic version detection. Related to microsoft#26516 which updates CUDA 13 build pipelines, but this PR fixes the Python runtime code that was still hardcoded to CUDA 12. ## Problem The build system correctly detects CUDA 13 via CMake, but the runtime Python code had CUDA 12 hardcoded in multiple locations, causing "CUDA 12 not found" errors on CUDA 13 systems. ## Solution Modified onnxruntime/__init__.py and setup.py to dynamically use the detected CUDA version instead of hardcoded "12" strings. ## Changes - Dynamic CUDA version extraction from build info - Library paths now use f-strings with cuda_major_version - Added CUDA 13 support to extras_require and dependency exclusions - Fixed TensorRT RTX package to use correct CUDA version - Updated version validation to accept CUDA 12+ - Fixed PyTorch compatibility checks to compare versions dynamically ## Impact - CUDA 13 builds now load correct libraries - Backward compatible with CUDA 12 - Forward compatible with future CUDA versions ## Testing Verified with CUDA 13.0 build that library paths resolve correctly and preload_dlls() loads CUDA 13 libraries without errors.
Rohanjames1997
pushed a commit
to Rohanjames1997/onnxruntime
that referenced
this pull request
Dec 4, 2025
…icrosoft#26516) ### Changes Update cuda 13 python packaging pipeline: (1) Use fatbin compress mode = size to reduce package size. This could significantly reduce package size. (2) Update CMAKE_CUDA_ARCHITECTURES for cuda 13. Since we reduced package size, we are able to add more architectures. (3) Fix cuda 13 packaging pipeline: - use correct (cuda13 instead of cuda12) manylinux docker. The new linxu docker has cuda 13.0.2 and cuDNN 9.14. - pass cuda version properly to run build_linux_python_package.sh in docker. (CUDA_VERSION in docker was 12.8.1, and now we pass "12.8" from yml to be consistent). Note that the compress mode and cuda archs settings are not changed for CUDA 12.8, so cuda 12 wheel size is larger than cuda 13 wheel size. We can update them in a separated PR if needed. The nuget pipeline for cuda 13 need extra code change, and this PR only fixes python packaging pipeline. ### Python GPU Wheel Size (Cuda Architectures + PTX) CUDA | Windows | Linux ----|---|--- 12.8 | 221 MB (52;61;75;86;89+90) | 271 MB (60;70;75;80;86;90a+90) 13.0 | 186 MB (75;80;86;89;90a;100a;120a+120) | 191 MB (75;80;86;89;90a;100a;120a+120)
| cmake_build_type: ${{ parameters.cmake_build_type }} | ||
| cuda_version: '12.8' | ||
| cudnn_folder: '' | ||
| cmake_cuda_archs: '52-real;61-real;75-real;86-real;89-real;90-virtual' |
Contributor
There was a problem hiding this comment.
Why these CUDA architectures? Why 61-real and 75-real and not 70-real?
CUDA kernels are only forward compatible within minor versions. That means it does not run e.g. on a Tesla V100 with SM 7.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Update cuda 13 python packaging pipeline:
(1) Use fatbin compress mode = size to reduce package size. This could significantly reduce package size.
(2) Update CMAKE_CUDA_ARCHITECTURES for cuda 13. Since we reduced package size, we are able to add more architectures.
(3) Fix cuda 13 packaging pipeline:
Note that the compress mode and cuda archs settings are not changed for CUDA 12.8, so cuda 12 wheel size is larger than cuda 13 wheel size. We can update settings for cuda 12.8 in a separated PR if needed.
The nuget pipeline for cuda 13 need extra code change, and this PR only fixes python packaging pipeline.
Python GPU Wheel Size (Cuda Architectures + PTX)