Repository navigation
metal: add GATED_LINEAR_ATTN support - #21859
Ra5hidIslam wants to merge 1 commit into
Conversation
|
Hey @Ra5hidIslam, nice work on this! Just wanted to flag that I have an existing PR for the same op over at #21452, filed April 5. It covers head_size 64 and 128, includes test-backend-ops updates, and regenerates the docs. Might be worth consolidating so we're not duplicating effort. Happy to collaborate if you want to combine forces! |
Thanks for the heads up, didn't see #21452 when I started. Your PR is further along (both head sizes + tests + docs), so it makes sense to consolidate there. Happy to review it and contribute anything useful from my implementation. Closing this one. |
|
Closing this one, as there is a prior PR with #21452 which is doing the same. |
Overview
Added Gated Linear Attention Op for Metal
Additional information
Passed all the tests, no performance test was there so couldn't test the performance.
Related to #14909
Requirements