quantized model ? Llama cpp?

[thiswillbeyourgithub]

Hi,

Reading your articles made me really curious about trying that but I was wondering of it was possible to use HuggingFace's quantized models or even llamacpp or if that required deep changes.

Thanks!

[vgel]

Working on a llama.cpp implementation!

[vgel]

There's now a PR live on the llama.cpp repo: ggml-org/llama.cpp#5970

[vgel]

That PR is merged, so closing this issue.