Eval bug: GLM-5.3-Flash decode stalls on Metal because fused Lightning Indexer falls back to CPU #29867

Name and Version

llama-cli --version
version: 0.5.0-dev (build 11349, commit fb4b273)
built with AppleClang 21.0.0.21000334 for Darwin arm64

Operating systems

Mac

GGML backends

Metal

Hardware

Apple M5

Models

GLM-5.3-Flash at UD-IQ3_XXS

Problem description & steps to reproduce

I've observed the following issue. Unfortunately, I don't fully understand the root cause; all I can tell after some trial and error is that disabling the LID results in a massive speedup. Would it be possible to expose the env var again (or could this point to a deeper underlying issue)? Currently, I have to patch the source code manually to make it usable under the conditions described below.

Problem: On Metal, GLM-5.3-Flash (glm5-next) decoding intermittently stalls for 1–9 s per token after a few dozen tokens. Without the stalls, it decodes at ~31 t/s; with them, the average drops to 0.5–10 t/s.

Context: Build b11347 (5fc4f3c), Apple M5, unsloth GLM-5.3-Flash-UD-IQ3_XXS (shard rewrite applied), -ngl 999 -fa on -c 65536. The stalls disappear when the fused lightning indexer is disabled.

Likely cause (suggested by AI):

• fused_lid is hard-enabled (llama-context.cpp#L237-L238). Because auto_flid is false, the existing Lightning Indexer probe in resolve_fused_ops() (llama-context.cpp#L569-L573) never runs, although it would detect exactly this CPU fallback and disable the fused op. model: add GLM-5-Next (GLM-5.3-Flash) #27754 exposed an env var (LLAMA_FUSED_LID_DISABLE) to disable the indexer on demand.
• The Metal lightning indexer kernel only supports exactly 64 indexer heads (ggml-metal-impl.h#L133); supports_op returns false otherwise (ggml-metal-device.m#L1772-L1776). GLM-5.3-Flash has 32 indexer heads, so the fused op falls back to the CPU, which presumably causes the stalls.

Suggested fix (suggested by AI, confirmed working locally): Set cparams.auto_flid = true so the probe runs, and/or re-expose LLAMA_FUSED_LID_DISABLE as a manual escape hatch.

Workaround: Build from source with the probe enabled in src/llama-context.cpp:

Line 238
Old: cparams.auto_flid = false;
New: cparams.auto_flid = true;

Build flags: -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON -DGGML_METAL_EMBED_LIBRARY=ON -DGGML_NATIVE=ON -DBUILD_SHARED_LIBS=OFF

Related: #27773 (GLM-5.3-Flash support)

First Bad Commit

No response

Relevant log output

see above