Skip to content

Pin llama.cpp bd5b297: HRX remainder prefill, 32K+ prompts, mixed-format SwiGLU, Hadamard Q4_0 - #278

Merged
bong-water-water-bong merged 2 commits into
mainfrom
pin-llama-bd5b297
Oct 2, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
pin-llama-bd5b297

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

llama.cpp fork commits since 6e42b51 (figures from each PR, Qwen3.8-27B on HRX0, balanced mode):

Docs: docs/hrx.md #55 section (remainders now covered for Q5_K/IQ4_XS) and "Our patches".
Registry regenerated (tools/registry_build.py; no mapping changes).

A smoke test of the combined pin (build, test-backend-ops MUL_MAT x2, llama-bench pp400/pp512/tg64 on Qwen3.8-27B UD-Q4_K_XL) is running on strixhalo; I'll post the results here.

Unblocks #276 (needs #59) and #277 (needs #58).

🤖 Generated with Claude Code

…mat SwiGLU, Hadamard Q4_0

llama.cpp fork commits since 6e42b51 (figures from each PR, Qwen3.8-27B on HRX0, balanced mode):
- #61: q8_1 x4 prefill takes the 256-aligned head of a Q5_K/IQ4_XS remainder chunk.
  UD-Q4_K_XL pp400 67.4 -> 103.4, pp1862 172.1 -> 234.6 tok/s; KLD vs 6e42b51 0.00215,
  same top token 97.19%; MUL_MAT HRX0 287/287.
- #59: V stays row-major past copy_transpose_f16's 32768-row range. pp512 @ d32768 failed
  before, 227.4 tok/s now; a 34,816-token wikitext chunk gives PPL 6.2150.
- #60: K-quant SwiGLU decode stages separate gate/up codebooks. UD-IQ2_S tg128 3.12 -> 8.32;
  UD-Q4_K_XL unchanged; KLD 0.00013; test-backend-ops HRX0 996/996.
- #58: llama loads onebit.hadamard_q4_0 files through llama-hadamard (needed by #277).
  Q4_0-H32 vs BF16: PPL 6.021, KLD 0.0292, same top token 92.5%.

Docs: docs/hrx.md #55 section (remainders now covered for Q5_K/IQ4_XS) and "Our patches".
Registry regenerated (tools/registry_build.py; no mapping changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit 446bff3

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Review (PR-Agent duty): looks right. The submodule and registry/architectures.json both point at bd5b297; the registry diff is the one pin line. The #58, #59, #60 and #61 entries match the fork PRs and the figures their authors measured.

One labelling nit in the new #55/#61 table in docs/hrx.md: the right column says bd5b297, but those numbers (pp400 103.4, pp1862 234.6, KLD 0.00215) were measured on 6e42b51 + #61, which is a36d547 without #58-#60. #58-#60 don't touch the Q5_K/IQ4_XS prefill path, so I expect the same numbers on bd5b297. Either relabel the column 6e42b51 + #61, or keep bd5b297 once your combined-build smoke test (pp400) confirms it.

Fine to merge after the smoke test and build pass.

… (review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Smoke test of the combined pin on strixhalo (fork bd5b297, HRX0, balanced mode, box shared with other jobs):

The pp512 and tg64 spreads come from thermal-run pauses and other GPU jobs on the box, not from the pin. The numbers in the docs come from each fork PR's own measurements.

The #61 table column is relabelled to "6e42b51 + #61 (a36d547)" (review).

@bong-water-water-bong
bong-water-water-bong merged commit 4d88a6a into main Oct 2, 2026
10 of 11 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pin-llama-bd5b297 branch October 2, 2026 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant