[waleedabujaish]

Overview

Fixes #25817.

For llava-projector models, the "skip the last ViT layer" step (LLaVA's
vision_feature_layer = -2) happens twice. The legacy converter already
writes block_count = num_hidden_layers - 1 and drops the last layer's
tensors, and the graph then subtracts again (il_last = hparams.n_layer - 1
in models/llava.cpp). Net effect: features are computed one layer short of
what the projector was trained on, and the last stored layer (v.blk.22 for
CLIP-L/14-336) is loaded but never executed. Static check: the mmproj metadata
says clip.vision.block_count = 23 (tensors v.blk.0..22) while the graph
builds 22 layers.

This PR sets il_last = hparams.n_layer and removes the MiniCPM-V/GLM-Edge
il_last += 1 compensation, which existed only to undo the subtraction.
Per-family arithmetic:

family	stored layers	built before	built after
LLaVA/BakLLaVA/ShareGPT4V/Yi-VL/MobileVLM (legacy converter, block_count = n-1)	23	22 (bug)	23 (fixed)
MiniCPM-V, GLM-Edge (full count, had += 1)	N	(N-1)+1 = N	N - unchanged
Granite Vision (explicit feature_layers)	max(fl)	override	override - unchanged

Verification (LLaVA-1.5-7B F16 mmproj, fixed input, fp32 reference validated
against HF get_image_features(); CLS ordering held at the graph's current
behavior - see companion #25814; method and raw data in the repo linked
below):

build	vs 22-layer (defect) simulation	vs 23-layer (intended) simulation
master	0.998 mean per-token cos	0.949
this PR	0.949	0.9986

With the companion #25814 fix also applied, output matches the HF reference at
0.9965 (the F16/flash-attn floor), and on LLaVA-1.6 (same block_count = 23
metadata) all 5 tiles match the correct computation at 0.99997.

Non-regression, proven bit-exact: GLM-Edge-V-2B (THUDM/glm-edge-v-2b-gguf,
SigLIP encoder, full block count - exercises the deleted += 1 path) produces
bit-identical encoder output on master and on this PR (np.array_equal over
the full 578x2048 embedding).

The tools/mtmd/tests.sh llava vision test invocations
(second-state/Llava-v1.5-7B-GGUF:Q2_K, cjpais/llava-1.6-mistral-7b-gguf:Q3_K_M)
pass with this fix alone and with both fixes (replicated against the fixed
build - the script rebuilds build/ in place). Vision encode gets slower for
the affected families (measured +5.9% against a same-session control;
theoretical +1/22 ~ +4.5%) - that's the previously-dead layer now executing,
i.e. the correct computation, not a regression. Prefill and decode are
unchanged.

Additional information

• Companion issue/fix: Eval bug: mtmd llava graph feeds projected [CLS] to the LLM and shifts patch position embeddings #25814 ([CLS] ordering) - independent fix on the same
  graph; happy to combine into one PR if preferred.
• Present since the original LLaVA support (Implement multimodal models (LLaVA) #3436): both subtractions shipped
  together, and every refactor since preserved them.
• A converter-side fix was considered and rejected: it would require
  regenerating every published mmproj file, while this graph-side fix
  ("stored block_count = layers to run when no explicit feature_layers")
  repairs all files in circulation. No current converter can produce a
  full-count GGUF for these projector types, so the reinterpretation is safe.
• Generated outputs will change for the affected families - expected, since
  features now come from the layer the projector was trained on.
• Full verification harness, raw comparison data, and commit-pinned
  references: https://github.com/waleedabujaish/llama.cpp-visual-token-pruning

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - AI-assisted analysis and verification tooling
  (embedding-comparison harness in the linked repo); the fix itself and all
  verification runs were executed and reviewed by me.

[ggml-gh-bot]

Hi @waleedabujaish, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 2 open PRs.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[waleedabujaish]

Closing to respect the single-open-PR rule for new contributors. The fix stays
on my mtmd-fix-llava-layer-count branch and I will reopen this after #25844
is resolved - or fold it into #25844 if maintainers prefer.