Repository navigation

AVX2: Speed up large batch size prompt processing of IQ models - #27402

Merged

bartowski1182 merged 13 commits into

ggml-org:masterfrom

bartowski1182:quant-run-speed

Aug 31, 2026

Commits on Aug 30, 2026

Batched gemm for grid IQ quants
bartowski1182
committed
1bc75a5
Add myself as iqp.* codeownder
bartowski1182
committed
b289413
Remove ggml_cpu_iqp_scratch_offset and ggml_cpu_iqp_src1_conv_size
bartowski1182
committed
62d8e25
Renaming and moving
bartowski1182
committed
a2222df
The other half of renaming and moving
bartowski1182
committed
1de3e6f
Move macros and ggml_cpu_iqp_mul_mat_id_min_batch definition
bartowski1182
committed
5ee21d9
Update ggml/src/ggml-cpu/iqp.h

bartowski1182
and
ggerganov
committed
22dd050
Add iqp_rows work buffer
bartowski1182
committed
e42b2d7
Revert "Add iqp_rows work buffer"
bartowski1182
committed
6a966e8
Add NUMA fallback
bartowski1182
committed
2a80ac4
Add 10 row batch tests for IQP coverage on all grid IQ types
bartowski1182
committed
a0877e8
Swap assert for return false in support check
bartowski1182
committed
f591762
Move IQP mul_mat_id test
bartowski1182
committed
1ed17b4