[ykhrustalev]

Overview

The transformers processor for LFM2 resizes images only with torchvision, and the d1-omni vision.py calls torchvision too. Since transformers 5 most other processors do the same, the Qwen-VL one too. For uint8 images on the CPU, torchvision uses the same algorithm as the Pillow port in mtmd-image.cpp, with one difference: it rounds the filter weights to int16 instead of 22 bits. #27594 removed the older non-Pillow resizes; the two new values here reuse the Pillow code and change only the weight rounding.

• adds RESIZE_ALGO_BILINEAR_TORCH and RESIZE_ALGO_BICUBIC_TORCH
• the LFM2 and d1-omni loader maps bilinear and bicubic to them, so existing GGUFs work without reconversion
• other models keep the Pillow algos

Additional information

Measured on an Apple M3 Ultra (Metal) and on a Xeon 8480+ with an H100 (CUDA), against PyTorch fp32 with torch 2.14.1, torchvision 0.29.1 and transformers 5.19.0.

Precise

Resize alone: 0 of 5000 random resizes differ from torchvision on arm64, and 0 of 3000 on x86 (AVX512 and AVX2). The current Pillow port differs on 80 to 99% of them, by 1 or 2.

Crops given to the vision encoder, out of 94 images, how many are identical to the PyTorch processors (the Mac and x86 give the same bytes):

model	master	this PR
d1-omni	8	94
LFM2.5-VL-450M	8	94
LFM2-VL-450M (mmproj with the bicubic key)	0	94
d1-3B, images up to 1 MP (88)	0	88

Answers on Metal, largest and mean difference in probability against PyTorch (d1: answer options, LFM2-VL: next token):

set	master	this PR
d1-omni, 150 POPE questions	0.0183 / 0.00048	0.0044 / 0.00018
d1-omni, 96 case suite	0.0098 / 0.0013	0.0079 / 0.00094
d1-3B, 450 POPE questions	0.0087 / 0.00028	0.0076 / 0.00021
LFM2.5-VL-450M, 52 images	0.0137 / 0.0011	0.0031 / 0.0004
LFM2-VL-450M (bicubic key), 52 images	0.0034 / 0.00085	0.0024 / 0.00058

These were measured with #30261 applied on 3bb63cb; today's master gives the same crops. No answer changes and no prompt length changes in any set. Per question, this PR is closer on 287 of 450 d1-3B and 93 of 150 d1-omni POPE questions. On CUDA the backend's own F16 differences are bigger than this, so the CUDA run shows no clear change either way.

Not slower

	M3 Ultra	Xeon 8480+ / H100
one resize	at most 0.5% slower	at most 0.2% slower
image preprocessing, this PR / master	0.99 to 1.02	1.000 to 1.003
whole request, LFM2.5-VL-450M	+0.31%	-0.45%
whole request, d1-omni	+0.37%	-0.25%
whole request, d1-3B	-0.12%	-0.56%
whole request, two servers of the same build	+0.22% to +0.39%	+0.04% to +0.71%

400 (Metal) and 600 (CUDA) paired requests per model, 5 image sizes up to 12 MP, builds interleaved. Two default builds of the same Pillow code already differ by up to 2% because of code placement; the x86 resize and preprocessing numbers are with -falign-functions=64 -falign-loops=64 -falign-jumps=64, which removes that.

Fits llama.cpp

• adds 34 lines and removes 10 in libmtmd; no new dependency, no GGUF, converter or API change
• CPU code that runs before any backend, so every backend gets the same crops; arm64 and x86 give the same bytes
• it applies to more models: the transformers processors of all 36 supported checkpoints I could load resize uint8 with torchvision this way (3 of them with lanczos, which would need a third value). For Qwen-VL, after mtmd : match the transformers Qwen-VL resize #30736, one more line makes the crops identical on all 94 images for Qwen2-VL, Qwen2.5-VL and Qwen3-VL (today 0, 0 and 6, with the same pixel limits on both sides), and the Qwen2-VL-2B mean difference in next token probability goes from 0.0097 to 0.0037. This can follow in its own PR.

Not fixed here:

• torchvision itself uses a float path for both modes on a GPU device, and for bilinear on x86 without AVX2. There 18 to 20% of the values differ from its uint8 path. The processors run on the CPU unless a device is passed.
• The published LFM2-VL (450M, 1.6B, 3B) and LFM2.5-VL-3B mmprojs, including ggml-org/LFM2-VL-450M-GGUF, have no resize key, so they use bilinear while their processors use bicubic. Reconverting writes the key (since model : add LiquidAI/d1-3B decision model #30110). For these files this PR is slightly worse: LFM2-VL-450M mean from 0.0284 to 0.0288, no answer changes.
• The d1-3B reference first shrinks images over 1 MP with PIL; llama.cpp does not.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. I used Claude (Anthropic, through Claude Code) to find where the LFM2 crops differ from the transformers processors, to write the two resize values and the test, and to run the precision and speed measurements on the Mac and the H100. I reviewed the code and can explain every line.