Repository navigation
Add full array-api tests #3514
Description
From our discussion here #48 (comment) I agree with @mtsokol that the sensible thing to do would be not to one-off #3484 but instead to fix things wholesale by adding a full test suite from https://github.com/data-apis/array-api-tests
Activity
mtsokol commentedon May 11, 2026
I think we can mimic JAX and NumPy when it comes to array-api-tests suite setup: #3526
ev-br commentedon Aug 23, 2026
Now that #4334 has landed, the array-api test suite does not need patching to run.
Local testing showed that about 3/4 of the test suite passes, and the rest are mostly small-ish issues, plus a couple of larger items which probably are just fundamental differences between MLX and the Array API spec (and the test suite has a mechanism to mark them as such).
data-apis/array-api-compat#450 tracks an overall status, and data-apis/array-api-compat#450 (comment) has a specific suggestion.
Overall, compatibility has been improving recently, mostly thanks to tireless work of @aaishwarymishra and @prady0t and reviewers (thanks!).
Would the maintainers be open to reviewing a PR setting up the CI run similar to gh-3526, with xfails for the currently known failures? Or would y'all prefer that we track the status externally and keep sending individual PRs for small issues found in local testing?
Either way, I'm happy to help reviewing from the Array API perspective if that'd be helpful.
zcbenz commentedon Aug 29, 2026
I'm good running array-api tests in our CI, with the known failures masked out.
zcbenz commentedon Sep 30, 2026
Closed by #4433