Repository navigation
Qwen3.5-35B crashes with transformers chat #45362
Description
System Info
Using "transformers chat" with Qwen3.5-35B the moment a prompt is sent the server errors with an AttributeError
[snip]
File "/mnt/venvs/rocm6.4/lib64/python3.12/site-packages/transformers/cli/serving/chat_completion.py", line 174, in _streaming
queue, streamer = gen_manager.generate_streaming(model, processor, inputs, gen_config, request_id=request_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/venvs/rocm6.4/lib64/python3.12/site-packages/transformers/cli/serving/utils.py", line 565, in generate_streaming
streamer = DirectStreamer(processor._tokenizer, loop, queue, skip_special_tokens=True)
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Qwen3VLProcessor' object has no attribute '_tokenizer'. Did you mean: 'tokenizer'?
I'm running transformers 5.5.0 from pip along with the rocm6.4 version of pytorch (2.8.0+rocm6.4.4.gitc1404424). If you want a full pip version list let me know.
I've tried a couple of other random text models with transformers chat without issue, just Qwen so far that appears to be an issue.
Qwen's readme mentions transformers chat as a possible way to use it, so I'm assuming this used to work? Although they were using 4.57 dev version at the time the readme was written.
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
transformers serve
transformers chat Qwen/Qwen3.5-35B-A3B
Once loaded, type any text at the prompt, press enter and server will error.
Expected behavior
Model response streamed back.
Activity
sharziki commentedon Apr 11, 2026
Hi! I can take this. The root cause is that
generate_streaming()andgenerate_cb_streaming()inserving/utils.pyaccessprocessor._tokenizerto get the Rust tokenizer backend, but this only works forPreTrainedTokenizerFast—ProcessorMixinsubclasses likeQwen3VLProcessorexpose the tokenizer at.tokenizer(public attribute) instead.The fix is to resolve the fast tokenizer from the processor first, then access
._tokenizer:This handles both code paths (ProcessorMixin and PreTrainedTokenizerFast). Two locations to update: lines 565 and 664.
Happy to submit a PR. AI assistance (Claude Code) will be used — I'll review and validate every change.
fix(serving): resolve rust tokenizer from ProcessorMixin in streaming…
fix(serving): resolve rust tokenizer from ProcessorMixin in streaming…
fix(serving): resolve rust tokenizer from ProcessorMixin in streaming…