[zcbenz]

This PR implements a QMM kernel using Ampere pipelining, it is a lot faster than naive QMM but still has bad memory bandwidth.

DGX:

M	N	K	QMM (GiB/s)	CUBLAS (GiB/s)	QMM (TFlop/s)	CUBLAS (TFlop/s)	Speedup (x)
16	4096	4096	393.4	249.2	11.68	3.96	2.95
64	4096	4096	314.3	235.1	35.76	14.59	2.45
16	16384	16384	190.1	231.5	5.70	3.70	1.54
64	16384	16384	180.2	195.1	21.39	12.39	1.73

A100:

M	N	K	QMM (GiB/s)	CUBLAS (GiB/s)	QMM (TF/s)	CUBLAS (TF/s)	Speedup (x)
16	4096	4096	333.3	1400.6	9.89	22.24	0.44
64	4096	4096	245.7	1282.7	27.95	79.61	0.35
16	16384	16384	606.7	1660.8	18.21	26.52	0.69
64	16384	16384	419.6	1577.0	49.81	100.14	0.50

I'm afraid with the original MLX quantization format it is impossible to achieve very high memory bandwidth, but I think we should make everything work first before looking into alternatives.

This kernel is only implemented for FP16 x INT8, and I'll add support for INT4/FP4/FP8 quants in later PRs. I'll also add a fallback naive QMM kernel that supports all settings.

[angeloskath]

Looks great!