metal-flash-attention support

[czkoko]

Can this project help for you? https://github.com/philipturner/metal-flash-attention

So far, metal-flash-attention can indeed provide the fastest generation speed for stable diffusion on MacOS.

[leejet]

Thank you for the feedback. I'm currently focusing on making it run faster, and I'll make time to take a look at this project and see if I can offer any assistance.

[sroussey]

I came here to suggest the same thing.

[Green-Sky]

linking this here for reference ggml-org/ggml#293

[GaidamakUA]

What needs to be done to make this happen? I'm not very good with cpp, but I want to help.

[Green-Sky]

#386
I am in the dark on metal flash attention support, or metal support in general.
So would be nice if someone with the hardware could test the pr. :)