Repository navigation
[Feature] Metal implementation of linalg APIs #1392
Description
Describe the bug
The mx.linalg.svd crashes
To Reproduce
Include code snippet
import mlx.core as mx
a = mx.array([ [1., 2], [2, 3] ])
u, singular_values, v_transpose = mx.linalg.svd(a=a)
libc++abi: terminating due to uncaught exception of type std::runtime_error: [SVD::eval_gpu] Metal SVD NYI.
Expected behavior
import numpy as np
import mlx.core as mx
a = mx.array([ [1., 2], [2, 3] ])
u, singular_values, v = np.linalg.svd(a=a)singular_values
array([4.236068 , 0.23606798], dtype=float32)
Desktop (please complete the following information):
MLX 0.17.2
Additional context
Add any other context about the problem here.
Reacted by Vladimir Iglovikov
Activity
awni commentedon Sep 3, 2024
That is unfortunately expected behavior. Right now
svd(and several other linalg operations are only supported on the CPU back-end. You can fix that by passing in the CPU stream to the operation like so:awni commentedon Sep 3, 2024
I'm going to change this from a bug to a feature request and mark it as such. Note it's not a trivial op to implement on the GPU so it could take some time to get this feature into MLX.
[-][BUG] SVD crashes[/-][+][Feature] Metal implementation of SVD[/+]vgoklani commentedon Sep 3, 2024
Thanks!
ArkAung commentedon Oct 7, 2024
Is anyone working on this? If not, I would like to start working on this. Happy to collaborate with other contributors who would like to collab on the PR.
awni commentedon Oct 8, 2024
I don't know of anyone working on this. Happy to accept a contribution.
huangrzh commentedon Dec 4, 2024
Hi, how is it going with SVD?
hehua2008 commentedon Jan 11, 2025
Any progress? mx.cpu is too slow！😂
intproduct commentedon Nov 2, 2025
Hello, I'd like to ask if there has been any progress on the GPU SVD feature?
awni commentedon Nov 5, 2025
No progress :\
ArkAung commentedon Nov 15, 2025
I was working on it for a while and got side-tracked because of other responsibilities. If people want this, I will properly dedicate this and revive the draft PR (now stale and closed). There are some challenges with making it work for large matrices. I am willing to take co-contributors to get this across the line.
[-][Feature] Metal implementation of SVD[/-][+][Feature] Metal implementation of linalg APIs[/+]feat(plasticity): port FIRE+DASH+ReDo from nanochat, GPU-native
sashko-zakharchuk commentedon Aug 11, 2026
Unless someone's already on the CUDA side of this, I'll start with a Cholesky PR (#1026)
backed by cuSOLVER, with inverse as a follow-up. Following the #3825 discussion: a vendor
library call with cached handles like the cuDNN path in the CUDA backend, no new hand-written
kernels (I saw where #3851 went), and no shape-based CPU dispatch; small matrices can keep
using the CPU stream. I'd benchmark where cuSOLVER starts winning on sm_120. Leaving SVD out
of scope for now.
ternaus commentedon Aug 22, 2026
Generic Metal
linalg.invinvestigationI implemented a generic float32 Metal prototype for
mx.linalg.invin #4375. It usesMPSMatrixDecompositionLUfollowed byMPSMatrixSolveLUagainst identity. The PR remains a draft because the performance results require a backend-dispatch decision before this route is suitable for merging.On an M4 Max (16-core CPU), macOS 26.4.1, these are medians of 20 runs after 5 warm-ups. Equivalent inputs were materialized on CPU or Metal before timing; host-device copies are excluded. Each timed run includes
inv,eval, andsynchronize.Command-buffer profiling shows that this is not primarily MLX handoff overhead. For 1024x1024, MPS itself occupies 14.0–14.6 ms of a 15.7 ms end-to-end Metal operation, while CPU takes about 5.2 ms. For a 32x64x64 batch, the MPS work takes about 21 ms and per-matrix host encoding adds about 2.1 ms; CPU takes about 0.7 ms.
I also tested native MPS batching outside MLX. This is a likely Apple MPS framework bug, not a MLX bug: a standalone program constructs two matrices with
matrices=2, setsbatchSize=2on both LU and solve operations, and runs with Metal API Validation enabled. It produces the correct inverse for batch 0 and zeros for batch 1, without a validation error. MLX therefore keeps the correct per-matrix encoding loop; the native MPS batch route is not safe to use. The repro needs escalation to Apple MPS engineering.The current benchmark and reproduction command are in #4375. The remaining design question is a generic dispatch policy for small and medium matrices versus a new custom Metal backend.