[rapsealk]

Summary

reverse_window_indices in the Youtu-VL vision encoder returned window_index itself instead of its inverse, so the merger output was un-reordered by applying the window permutation a second time. That is only correct when the permutation is an involution. The helper now builds argsort(window_index) directly, the same construction #1601 landed for Qwen2.5-VL, with checkpoint-free tests that pin the affected grids.

Related issues

Closes #1600. Same defect as #1596 / #1601 (Qwen2.5-VL), whose corrected helper this copies.

Type of change

• feat — new user-visible feature
• fix — bug fix
• perf — performance improvement (include before/after numbers in the PR body)
• refactor — internal restructuring without behavior change
• chore — build, CI, dependencies, release infrastructure
• docs — documentation only
• test — tests only

Which grids are affected

A Youtu-VL window is 8x8 merged tokens (window_size 256 over patch_size 16 times spatial_merge_size 2), and build_processor caps the processor at vision_config.num_patches (4096), so the patch grid tops out at 64x64 and the merged grid at 32x32. Enumerating every merged grid the loader can actually produce, the permutation is an involution only when the whole image fits in one window, that is when it resizes to 256x256 or smaller:

image	resized	patch grid	merged grid	windows	merged tokens misplaced by the old un-reorder
224x224	224x224	14x14	7x7	1x1	0 of 49 (the case that hid the defect)
256x256	256x256	16x16	8x8	1x1	0 of 64
336x336	352x352	22x22	11x11	2x2 padded	99 of 121
384x384	384x384	24x24	12x12	2x2 padded	120 of 144
448x448	448x448	28x28	14x14	2x2 padded	160 of 196
512x512	512x512	32x32	16x16	2x2	192 of 256
768x768	768x768	48x48	24x24	3x3	528 of 576
1024x1024 and larger	1024x1024	64x64	32x32	4x4	960 of 1024

Every merged grid from 9x9 through 32x32 is a non-involution; there is no second safe size above one window. A grid whose window count per edge equals the window width would be one, but that is a 64x64 merged grid, which needs 16384 patches and is past the 4096 cap, so it is unreachable.

For Qwen2.5-VL the 4-wide window makes the common 16x16 merged grid (448x448) an involution, which is why #1596 was invisible at the default fixture size. Youtu-VL is exposed at every size above a single window instead.

What changed

• src/vision/encoders/youtu_vl_window.rs: reverse_window_indices builds inverse[window_index[rank]] = rank directly, bounds-checked so a malformed grid cannot panic. The rustdoc records why the old construction degenerated. Name and signature unchanged, so the single call site is untouched.
• src/vision/encoders/youtu_vl.rs: the call-site comment no longer claims the old code was equivalent to argsort.
• src/vision/encoders/youtu_vl_tests.rs: four checkpoint-free tests, all failing on the pre-fix code: the minimal non-involution [2, 0, 1] (plus an involution), the 512x512 grid with its misplaced count pinned, the single-window identity case, and a two-image batch with padding.
• CHANGELOG.md: entry under Unreleased / Fixed, scoped to the token order at that stage and pointing at fix(youtu_vl): vision processor emits patches in the wrong order #1610 for the reason the family still misdescribes images.

Real-checkpoint validation, and why it does not show a difference

Acceptance criterion 3 asked for a Youtu-VL run at a non-involution size before and after. It was run on models/mlx/youtu-vl-4b-instruct (tencent/Youtu-VL-4B-Instruct) on an M1 Ultra, greedy (-t 0 -n 48), release build with metal,accelerate, at 448x448 and at 336x336, both non-involution per the table. The output is byte-identical before and after this change, and wrong in both.

fixture	before	after
test_image_shapes.png (448, 160 of 196 misplaced)	"The image contains a single, solid black circle on a white background."	identical
test_image_shapes_336.png (352, 99 of 121 misplaced)	"The image contains a single, solid black circle on a white background."	identical
test_image.png (224, one window, 0 misplaced)	"The image is completely black and contains no visible content, objects, text, or details."	identical

The third row is the diagnostic one. 224x224 is a single window, so its permutation is the identity and this change provably cannot alter it, yet the description of a solid orange square is already wrong. So a second defect upstream of the window reordering dominates the vision path at every size, and no window-inverse fix can show a content difference until it is fixed. That defect is filed separately as #1610 and is out of scope here per this issue's own Scope section.

This PR is therefore justified on the arithmetic, which is checked directly by the new unit tests rather than inferred from generation quality: the pre-fix construction is reproduced in the tests with its exact misplaced counts, and the new inverse round-trips to the identity on every grid tried.

Test plan

• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets --features metal,accelerate -- -D warnings
• cargo test --workspace --profile test-fast --features metal,accelerate
• cargo deny check (PR CI)
• Real checkpoint run at two non-involution sizes, recorded above

Notes for reviewers

The math is not in doubt: the tests reproduce the old construction's exact misplaced counts and prove the new inverse round-trips to the identity on every grid tried. What the real-checkpoint run adds is the knowledge that this family has a second, larger defect, which is why the changelog entry claims a corrected token order at that stage rather than a corrected image description.

Checklist

• PR title uses a Conventional Commits prefix (feat:, fix:, etc.)
• One logical change per PR (split unrelated changes)
• Updated docs/ if user-facing behavior or supported models changed (none needed)
• Updated // Used by: ... comments on any shared function I modified (unchanged: the module header already carries it)
• No secrets, credentials, or .env files committed