[inureyes]

Summary

Root cause of #1618 is not in the window path. The Youtu-VL chat template renders one <|image_pad|> per image, and the checkpoint's processor (YoutuVLProcessor.__call__) replaces it with one token per merged feature. insert_youtu_vl_image_tokens saw an image token and treated the prompt as already expanded, so the prompt carried 1 image token for 121 or 196 features and the masked scatter placed only the first (top-left) feature. A solid-color image reads correctly from any single token, which is why the 224 control passed and the "single window" pattern looked like a window bug.

• Placeholder expansion now mirrors the processor: one placeholder per image is expanded in place inside the template's framing; an already expanded prompt is a no-op; no placeholder keeps the splice-after-BOS fallback; any other count is a request error.
• YoutuVLModel::get_input_embeddings refuses a placeholder/feature count mismatch (upstream raises) instead of truncating silently.
• The two #[ignore]d content tests in tests/youtu_vl_parity.rs keep only the checkpoint requirement; stale comments attributing the 224 pass to the identity window permutation are corrected.

How the alternatives were ruled out

Stage-by-stage diff against transformers 4.56.0 (checkpoint's own remote code, f32, CPU):

fixture	grid	processor pixels vs HF	tower+merger vs HF, HF pixels (cos min / mean)
224	14x14	max abs 1.5e-8	0.99991 / 0.99991
336	22x22	lanczos vs bilinear resize, mean abs 0.003	0.914 / 0.998
448	28x28	max abs 3.0e-8	0.903 / 0.997

The mlxcel run is bf16, so per-token cosine near 1 with a few low tokens is precision, not a permutation. get_window_index, cu_window_seqlens (including partial windows and dedup), and the full-attention boundaries agree with the reference; a new unit test pins get_window_index to reference values on 11x11, 14x14, a batched pair, and a non-square grid. The decisive evidence was the prompt dump: mlxcel sent 128262, 128264, 128263 where the reference sends 128262, 128264 x 196, 128263.

Test plan

Before and after on CUDA (GB10), mlxcel generate ... -n 48 -t 0, "What shapes and colors are in this image? Answer briefly.":

fixture	before	after (CLI and mlxcel-server)	transformers greedy
224 orange	"The image is a solid, uniform orange color with no shapes or other colors present."	same	same
336 shapes	"The image contains a single white circle on a black background."	"three shapes: red square (top left), blue circle (top right), green triangle (bottom center) ..."	identical text
448 shapes	"The image contains a single, solid black circle on a plain white background."	"- Red square - Blue circle - Green triangle"	identical text

• youtu_vl_parity --ignored on the real checkpoint: 3 passed (both previously known-failing tests now pass)
• mlxcel-server /v1/chat/completions on all three fixtures: same text as the CLI, prompt_tokens 82 / 154 / 229, equal to the reference processor's token counts
• youtu_vl unit tests (37), including new prompt-expansion tests on a 28x28 multi-window grid and the mismatch error
• clippy --lib --tests --features cuda -D warnings, cargo fmt --check

Not verified on this host

The issue's evidence was on M1 Ultra (Metal). The failure reproduced word for word on CUDA and the fix is backend-independent (token ids only), but the Metal run itself was not repeated here.

Closes #1618