Skip to content

Allow using Qwen3-VL as flux2 klein text encoder (again) - #14526

Merged
comfyanonymous merged 11 commits into
Comfy-Org:masterfrom
kijai:qwen3vl
Jun 17, 2026
Merged

comfyanonymous merged 11 commits into
Comfy-Org:masterfrom
kijai:qwen3vl

Conversation

@kijai

@kijai kijai commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Apparently even if not intended (as far as I know) it used to be possible to use the VL variant as the text encoder, functionality which broke in the Qwen3-VL text generation PR. This should restore that so users can generate text and flux2 prompt embeds with same load.

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3d01420c-5a51-4979-abf8-8f128280ede5

📥 Commits

Reviewing files that changed from the base of the PR and between a590d60 and fc1ca3c.

📒 Files selected for processing (1)
  • comfy/sd.py

📝 Walkthrough

Walkthrough

In comfy/sd.py, the load_text_encoder_state_dicts function gains a new conditional branch inside the single-TE-model path for Qwen3-VL-detected text encoders. When clip_type is CLIPType.FLUX or CLIPType.FLUX2, the function now loads the Klein text-encoder variant (klein_te), selecting QWEN3VL_8B or QWEN3VL_4B as the model type and pairing it with KleinTokenizer8B or KleinTokenizer based on the detected model size.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically references restoring Qwen3-VL functionality as a flux2 klein text encoder, which directly aligns with the changeset's purpose of re-enabling this capability.
Description check ✅ Passed The description explains the restoration of previously broken Qwen3-VL text encoder functionality for flux2, directly relating to the code changes that add conditional logic for CLIPType.FLUX and CLIPType.FLUX2.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@comfyanonymous
comfyanonymous merged commit ca3dbe2 into Comfy-Org:master Jun 17, 2026
14 checks passed
@LukeG89

LukeG89 commented Jun 17, 2026

Copy link
Copy Markdown

@kijai Something isn't working quite right with qwen3-vl-8b as TE for Klein. I wanted to try both bf16 and fp8_scaled, but the image tends to degrade during generation—sometimes, the more tokens you add to the prompt, the more it degrades. I had mixed results: in some cases, the degradation was barely visible, but other times the image went completely broken, especially in editing mode. With qwen3_8b, there are no issues at all; both generation and editing work perfectly.

Just reporting this to let you know (I will continue using the regular model as usual).

zhangp365 pushed a commit to zhangp365/ComfyUI that referenced this pull request Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants