Feature Request: 'hy_v3'

[kalle07]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

0.00.148.466 E srv load_model: failed to load model, 'F:\LLM\models\repo\Hy-MT2-30B-A3B-Q6_K.gguf': error loading model: unknown model architecture: 'hy_v3'

not implemented in the base ...

https://huggingface.co/tencent/Hy-MT2-30B-A3B

Motivation

123

Possible Implementation

No response

[CISC]

Just as it says on the model card, pity they forgot to make a PR, or even a link to a fork...

Hy-MT2-30B-A3B uses the hy_v3 architecture. It requires a llama.cpp build that supports this architecture. If your llama.cpp build does not support it, loading the GGUF may fail with:

unknown model architecture: 'hy_v3'

Use a compatible llama.cpp build/branch for Hy-V3/Hy-MT2 models.

Either way this GGUF is useless, you can delete it.

[foldl]

FYI, here is my implementation of hy_v3:

https://github.com/foldl/chatllm.cpp/blob/14bd99441418f325bbbcec2febae1eecc126ed79/models/hunyuan.cpp#L2106

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.