hrx: Q2_K weights on HRX0 (shared dequantizer, K-quant decode kernels)

[bong-water-water-bong]

Q2_K had no matmul on HRX0. llama.cpp's load-time buffer check (a 512-token MUL_MAT per weight) failed, so every Q2_K tensor landed in the CPU_REPACK buffer and each use split the graph. On Qwen3-4B Q2_K that was 709 MiB on the CPU and 217 splits. Every Unsloth UD GGUF carries some Q2_K.

Changes

• Shared dequantizer (motifs/dequant.loom): ggml_q2k_f32_vector4 / _f16_vector4 follow dequantize_row_q2_K, with the same group/packet addressing as Q3_K. Format 12 is in the tile/row byte tables and both decode dispatchers, so the prefill WMMA kernels, generic decode and GET_ROWS read Q2_K.
• Format enum: CommonMulMatWeightFormat::Q2K (ggml Q2_K ↔ config 12).
• K-quant decode kernels (ops/kquant_decode_f32.loom): Q2_K lane functions for the 1-token and 2-8-token kernels. One lane is one 16-value sub-block with one scale and one min; block size 84, run shape 16. dispatch-kquant-decode.cpp accepts Q2K.

Checked on strixhalo (gfx1151, HRX0), with Qwen3-4B requantized Q8_0 → Q2_K:

check	before	after
weights	709 MiB CPU_REPACK, 217 graph splits	1586 MiB HRX0, 73 splits
test-backend-ops -b HRX0		929/929
MUL_MAT q2_K	9 run, 2 not supported	11/11
pp512 (5 reps)	686	1287–1325
tg128 (5 reps)	47.0 ± 0.4	74.4–74.8 ± 14

The tg128 spread is most likely the first repetition paying for new Loom kernel compiles.

KLD against the Q8_0 model (wikitext-2, 6 × 512):

• prefill path: 1.1045;
• decode path (-b 1): 1.1031;
• CPU: 1.1140;
• previous HRX with Q2_K on the CPU: 1.1224.

GET_ROWS q2_K: 1 shape runs and passes; batched shapes are still declined. MUL_MAT_ID (MoE experts) doesn't take Q2_K yet. IQ2_XS and IQ2_XXS follow the same way in a separate PR.

🤖 Generated with Claude Code

[bong-water-water-bong]

Review (standing in for PR-Agent while the local model is paused)

Checked the Q2_K lane functions against dequantize_row_q2_K.

Correctness: all of these match.

• Block layout: 84 bytes = scales[16], qs[64], d, dmin. d is i16 word 40 and dmin is word 41; qs starts at word 8.
• Lane mapping: lane l = 8n + 2j + h owns values 128n + 32j + 16h + 0..15, with scale byte scales[l] and qs bytes 32n + 16h .., i.e. words 8 + 16n + 8h. This matches is++ in the reference loop.
• 2-bit extraction: the 32-bit shift by 2j followed by the 0x03030303 mask gives the right bits in every byte; bits shifted across byte boundaries are masked out.
• Dot product: d·sc·Σqx − dmin·m·Σx is the reference formula.
• Multi-token path: lane_weights gives q·dl − ml. Run shape 2 (16 contiguous) is right for one sub-block per lane.
• Other tables: ggml_kquant_block_bytes gives 84 for format 12. The dispatcher's kquant_format, the enum, the ggml type map and the config value 12 are consistent.

Nit: the header comment says format 12 is one "which only dispatch-kquant-decode.cpp uses". Format 12 is now also in the dequant motif's tile and row tables (prefill WMMA, generic decode, GET_ROWS), so that clause could go.

CI:

• The Windows x64-vulkan failure is test-thread-safety crashing with 0xc0000374 (heap corruption). That build does not compile ggml-hrx, and the same job passed on hrx: kquant matchers skip q8-only inputs; softplus kernel #48. Re-running it.
• build-cmake-pkg never starts on this fork (same as hrx: kquant matchers skip q8-only inputs; softplus kernel #48).

Good to merge once the re-run is green.