fix: gate CUDA directory checks on GPU vendor to prevent false CUDA detection#8942
Merged
mudler merged 2 commits intomudler:masterfrom Mar 12, 2026
Merged
Conversation
✅ Deploy Preview for localai ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
sozercan
force-pushed
the
fix/gate-cuda-dir-on-gpu-vendor
branch
from
4c4ab1e to
75d9ce8
Compare
sozercan
force-pushed
the
fix/gate-cuda-dir-on-gpu-vendor
branch
from
75d9ce8 to
47fb0f5
Compare
mudler
reviewed
Mar 11, 2026
pkg/system/capabilities_test.go
Outdated
| "testing" | ||
| ) | ||
|
|
||
| func TestGetSystemCapabilities(t *testing.T) { |
Owner
There was a problem hiding this comment.
could you move these to use ginkgo to be consistent with all other tests?
Contributor
Author
There was a problem hiding this comment.
Done — converted to Ginkgo with DescribeTable/Entry and added a system_suite_test.go. Follows the same pattern as pkg/vram and pkg/functions/peg.
…etection Container images that install CUDA runtime libraries (e.g., cuda-cudart-12-5 via apt) create /usr/local/cuda-12 directories as a side effect. The previous code checked for these directories before checking whether a GPU was present, causing CPU-only hosts to select a CUDA backend that crashes because libcuda.so.1 is absent. Reorder checks so CUDA directory existence only refines the capability when an NVIDIA GPU is actually detected, consistent with the arm64 L4T code path. Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
sozercan
force-pushed
the
fix/gate-cuda-dir-on-gpu-vendor
branch
from
90d3b3e to
eea2fd7
Compare
localai-bot
pushed a commit
to localai-bot/LocalAI
that referenced
this pull request
Mar 18, 2026
…etection (mudler#8942) Container images that install CUDA runtime libraries (e.g., cuda-cudart-12-5 via apt) create /usr/local/cuda-12 directories as a side effect. The previous code checked for these directories before checking whether a GPU was present, causing CPU-only hosts to select a CUDA backend that crashes because libcuda.so.1 is absent. Reorder checks so CUDA directory existence only refines the capability when an NVIDIA GPU is actually detected, consistent with the arm64 L4T code path. Signed-off-by: Sertac Ozercan <sozercan@gmail.com> Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
localai-bot
pushed a commit
to localai-bot/LocalAI
that referenced
this pull request
Mar 25, 2026
…etection (mudler#8942) Container images that install CUDA runtime libraries (e.g., cuda-cudart-12-5 via apt) create /usr/local/cuda-12 directories as a side effect. The previous code checked for these directories before checking whether a GPU was present, causing CPU-only hosts to select a CUDA backend that crashes because libcuda.so.1 is absent. Reorder checks so CUDA directory existence only refines the capability when an NVIDIA GPU is actually detected, consistent with the arm64 L4T code path. Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
libraries installed (e.g.,
cuda-cudart-12-5via apt), which create/usr/local/cuda-12directories as a side effectgetSystemCapabilities()so CUDA directory existence onlyrefines the capability when an NVIDIA GPU is actually detected, consistent with
the arm64 L4T code path that already gates on
GPUVendor == NvidiaProblem
Container images that install CUDA runtime libraries create
/usr/local/cuda-12or
/usr/local/cuda-13directories. The previous code checked for thesedirectories before checking whether a GPU was present, causing CPU-only hosts
to select a CUDA backend. The CUDA backend then crashes because
libcuda.so.1isabsent.
Previous PR that fixed a similar issue: #6149
Changes
pkg/system/capabilities.go— Reordered the non-arm64 path ingetSystemCapabilities():"default"(early exit)"default"with warningGPUVendor == Nvidiapkg/system/capabilities_test.go— New file with table-driven tests:"""default""nvidia""nvidia-cuda-12""nvidia""nvidia-cuda-13""nvidia""nvidia-cuda-13""amd""amd""""default""nvidia""nvidia""nvidia""default"Test plan
go test ./pkg/system/...— new unit tests pass (skipped on darwin as expected)go vet ./pkg/system/...— clean"default"instead of"nvidia-cuda-12"