Skip to content

Update convert_hf_to_gguf: support Qwen3.5 embedding models - #27920

Merged
ggerganov merged 3 commits into
ggml-org:masterfrom
SamMalayek:convert/qwen35-textmodel-embedding
Oct 8, 2026
Merged

ggerganov merged 3 commits into
ggml-org:masterfrom
SamMalayek:convert/qwen35-textmodel-embedding

Conversation

@SamMalayek

@SamMalayek SamMalayek commented Aug 29, 2026 •

Copy link
Copy Markdown
Contributor

Overview

Adds convert_hf_to_gguf.py support for Qwen3.5 embedding models using Hugging Face Qwen3_5TextModel, such as Rebine/Qwen3.5-Embedding-0.8B.

Additional Information

Manual testing:

  • Real HF -> GGUF conversion
  • Metadata: 24 blocks, LAST pooling, no MTP
  • llama-embedding runtime load
  • 1024-d embedding output
  • Unsupported pooling fails closed

Did not add new test file to /tests (agents guidelines)

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES - AI was used to assist with investigation, implementation, and testing. I manually reviewed the changes and validated them.

@github-actions github-actions Bot added testing Everything test related conversion labels Aug 29, 2026
@SamMalayek
SamMalayek force-pushed the convert/qwen35-textmodel-embedding branch from 7b99e0a to df61700 Compare August 29, 2026 01:36
@SamMalayek
SamMalayek force-pushed the convert/qwen35-textmodel-embedding branch from df61700 to 6588857 Compare September 14, 2026 16:38
@SamMalayek

SamMalayek commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor Author

No changes made in recent force-push. Rebase only (also tested locally again):

https://github.com/ggml-org/llama.cpp/blob/master/CONTRIBUTING.md#after-submitting-your-pr

If your PR becomes stale, rebase it on top of latest master to get maintainers attention

@SamMalayek
SamMalayek force-pushed the convert/qwen35-textmodel-embedding branch from 6588857 to efc4a35 Compare October 2, 2026 05:19
@SamMalayek

SamMalayek commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor Author

@CISC I rebased and the diff shows the patch is unchanged. I ran all the tests as well. Could you review this when you have time, or suggest another reviewer? Thanks.

Comment thread conversion/qwen.py Outdated
Comment thread conversion/qwen.py Outdated
Comment thread conversion/qwen.py Outdated
Comment thread conversion/qwen.py Outdated
@CISC

CISC commented Oct 2, 2026

Copy link
Copy Markdown
Member

You want to try that again?

@SamMalayek

SamMalayek commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor Author

You want to try that again?

Mostly moved some code into a couple functions. No change in behavior.

Thanks again for the review.

@SamMalayek

Copy link
Copy Markdown
Contributor Author

@CISC I’ve addressed the suggestions in the latest update. Ready for another review.

Comment thread conversion/base.py Outdated
@SamMalayek
SamMalayek force-pushed the convert/qwen35-textmodel-embedding branch from bdc954c to c3a5c57 Compare October 7, 2026 07:49
@CISC CISC added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Oct 7, 2026
@ggerganov
ggerganov merged commit 75118a3 into ggml-org:master Oct 8, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conversion merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants