Repository navigation
Pin llama.cpp bd5b297: HRX remainder prefill, 32K+ prompts, mixed-format SwiGLU, Hadamard Q4_0 - #278
Conversation
…mat SwiGLU, Hadamard Q4_0 llama.cpp fork commits since 6e42b51 (figures from each PR, Qwen3.8-27B on HRX0, balanced mode): - #61: q8_1 x4 prefill takes the 256-aligned head of a Q5_K/IQ4_XS remainder chunk. UD-Q4_K_XL pp400 67.4 -> 103.4, pp1862 172.1 -> 234.6 tok/s; KLD vs 6e42b51 0.00215, same top token 97.19%; MUL_MAT HRX0 287/287. - #59: V stays row-major past copy_transpose_f16's 32768-row range. pp512 @ d32768 failed before, 227.4 tok/s now; a 34,816-token wikitext chunk gives PPL 6.2150. - #60: K-quant SwiGLU decode stages separate gate/up codebooks. UD-IQ2_S tg128 3.12 -> 8.32; UD-Q4_K_XL unchanged; KLD 0.00013; test-backend-ops HRX0 996/996. - #58: llama loads onebit.hadamard_q4_0 files through llama-hadamard (needed by #277). Q4_0-H32 vs BF16: PPL 6.021, KLD 0.0292, same top token 92.5%. Docs: docs/hrx.md #55 section (remainders now covered for Q5_K/IQ4_XS) and "Our patches". Registry regenerated (tools/registry_build.py; no mapping changes). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Docs7 for 1bit-monster/engine
Commit |
|
Review (PR-Agent duty): looks right. The submodule and registry/architectures.json both point at bd5b297; the registry diff is the one pin line. The #58, #59, #60 and #61 entries match the fork PRs and the figures their authors measured. One labelling nit in the new #55/#61 table in docs/hrx.md: the right column says Fine to merge after the smoke test and |
… (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Smoke test of the combined pin on strixhalo (fork bd5b297, HRX0, balanced mode, box shared with other jobs):
The pp512 and tg64 spreads come from thermal-run pauses and other GPU jobs on the box, not from the pin. The numbers in the docs come from each fork PR's own measurements. The #61 table column is relabelled to "6e42b51 + #61 (a36d547)" (review). |
llama.cpp fork commits since 6e42b51 (figures from each PR, Qwen3.8-27B on HRX0, balanced mode):
UD-Q4_K_XL pp400 67.4 -> 103.4, pp1862 172.1 -> 234.6 tok/s; KLD vs 6e42b51 0.00215,
same top token 97.19%; MUL_MAT HRX0 287/287.
before, 227.4 tok/s now; a 34,816-token wikitext chunk gives PPL 6.2150.
UD-Q4_K_XL unchanged; KLD 0.00013; test-backend-ops HRX0 996/996.
Q4_0-H32 vs BF16: PPL 6.021, KLD 0.0292, same top token 92.5%.
Docs: docs/hrx.md #55 section (remainders now covered for Q5_K/IQ4_XS) and "Our patches".
Registry regenerated (tools/registry_build.py; no mapping changes).
A smoke test of the combined pin (build, test-backend-ops MUL_MAT x2, llama-bench pp400/pp512/tg64 on Qwen3.8-27B UD-Q4_K_XL) is running on strixhalo; I'll post the results here.
Unblocks #276 (needs #59) and #277 (needs #58).
🤖 Generated with Claude Code