Skip to content

third_party: bump llama.cpp pin for the HRX decode-split multipass fix (engine#115) - #151

Merged
bong-water-water-bong merged 6 commits into
mainfrom
fix/hrx-decode-split-multipass-pin
Sep 26, 2026
Merged

bong-water-water-bong merged 6 commits into
mainfrom
fix/hrx-decode-split-multipass-pin

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Supersedes #121, which conflicted with main after #149 (ZAYA1-VL) advanced the same submodule pointer past where #121's target commit (f59f6d37a, the pre-merge tip of llama.cpp#13) was cut. This points at the current 1bit/hrx-vulkan-patched tip (8dd75eb), which is a strict superset of both #149's pin and llama.cpp#13's multipass fix - verified with git merge-base --is-ancestor for both.

Closes the remaining half of #115, same as #121 intended: onebit builds, and the exact >2048-token needle-retrieval repro is correct and deterministic (ran 4 times across two depths - 2124 and 3599 tokens - identical 4-8-1-5-9-2 each time, cache_prompt:false).

Re: the known residual fault (#123). llama.cpp#13 reported 0/5 GPU faults at depth 2100 in its own testing. Re-running the same llama-bench -p 0 -n 8 -d 2100 here faulted 4 times out of 4 attempts (HSA_STATUS_ERROR_MEMORY_FAULT, failed to decode generation batch, res = -3) - both as part of a combined -d 2100,3000 run and in three isolated repeats at 2100 alone. Confirmed this isn't a broader GPU/driver problem: Vulkan0 on the same box ran cleanly immediately before and after, and HRX0 at a short depth (500, below the multipass threshold) also ran clean. So the fault is real, reproduces consistently in this environment specifically at depths just above 2048, and is worse here than llama.cpp#13's own measurement - not the rare edge case the 1/5-at-d3000 number suggested. Posting this to #123 as well so the follow-up fix has accurate reproduction data.

Merging per explicit direction: the multi-pass fix's benefit (removing the 32% decode cliff and un-capping context past 2048) is worth taking now with the fault open, tracked as a known issue to fix next.

🤖 Generated with Claude Code

…x (engine#115)

Was 1e775cd, needed to move to include both the ZAYA1-VL work already on
main (f30cc43, via #149) and the multi-pass decode-split reduction from
llama.cpp#13 that #121 was trying to add - #121 conflicted because its
target commit predated #149's pin advance. This points at the current
1bit/hrx-vulkan-patched tip (8dd75eb), a strict superset of both.

Verified: onebit builds; the exact >2048-token needle-retrieval repro from
engine#115 is correct and deterministic (4 back-to-back runs at 2124 and
3599 tokens, cache_prompt:false). The known residual fault (engine#123) is
real and reproduces more readily here than llama.cpp#13's own report
(0/5 there; 4/4 here across two separate runs, isolated to depths >2048,
Vulkan0 and short-context HRX0 unaffected) - noted on #123 for whoever
picks up the follow-up fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit b71b296

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit b71b296)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

121 - Partially compliant

Compliant requirements:

Non-compliant requirements:

Requires further human verification:

  • Verification of the exact >2048-token needle-retrieval repro
  • Confirmation that the 32% decode cliff is gone
  • Testing of the GPU fault issue at depth 2100

149 - Partially compliant

Compliant requirements:

  • Pin third_party/llama.cpp to f30cc43
  • Pin third_party/llama.cpp-vulkan to 4a2c0656
  • Add support for ZAYA1-VL and Zamba2-VL
  • Fix Zamba2-VL chat-template
  • Update documentation
  • Regenerate registry/architectures.json

Non-compliant requirements:

  • No specific requirements not met in the PR code diff

Requires further human verification:

  • Verification of ZAYA1-VL and Zamba2-VL functionality on hardware
  • Confirmation of teacher-forced accuracy on image questions
  • Testing of F16 decode performance on Vulkan0

13 - Partially compliant

Compliant requirements:

  • Move third_party/llama.cpp to f1a0aca141de
  • Move third_party/hrx-system to 51b1739ae5fd

Non-compliant requirements:

  • CI build verification not shown in diff
  • Strix Halo verification not shown in diff

Requires further human verification:

  • CI build verification
  • Strix Halo verification with cmake and tests
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Submodule commit mismatch

The submodule commit for third_party/llama.cpp has been updated to 8dd75eb48f282198abdb0bd748aa0e4594745f1d, which is a superset of both the ZAYA1-VL pin and the multipass fix commit. While this is intended to be a strict superset, it's important to ensure that all changes from both commits are correctly integrated and that no regressions are introduced.

Subproject commit 8dd75eb48f282198abdb0bd748aa0e4594745f1d
Registry source commit update

The registry/architectures.json file has been updated to reflect the new commits for llama.cpp (vulkan) and llama.cpp (hrx). It's important to verify that these commits are correctly pinned and that the registry accurately reflects the current state of the submodules.

"llama.cpp (vulkan)": "a29be7f40c8b7825d9c009580119a0317b910726",
"llama.cpp (hrx)": "8dd75eb48f282198abdb0bd748aa0e4594745f1d",

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 5f7b59d

@bong-water-water-bong
bong-water-water-bong enabled auto-merge (squash) September 26, 2026 17:18
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 323e031

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Checked on strixhalo with this PR's build (llama.cpp 8dd75eb): Qwen3-Coder-30B-A3B Q4_K_M, llama-bench -p 0 -n 8 -d 2100 -dev HRX0, each run capped with MemoryMax.

decode-split Runs Result
off (GGML_HRX_DISABLE_DISPATCH=decode_split, what 1bit serve --device hrx sets since #148) 3 3 clean: 42.6, 43.4, 43.2 tok/s
on 2 1 clean at 49.5 tok/s; 1 HSA_STATUS_ERROR_MEMORY_FAULT

The multi-pass kernel is named flash_attention_decode_split_*, so #148's default turns it off along with the rest of decode-split. 1bit serve users don't hit #123's fault. They also don't get the multi-pass speedup, which is ~14% at this depth, until #123 and #140 are fixed and the default flips back. ONEBIT_HRX_DECODE_SPLIT=1 turns it on for testing. The pin is the tip of 1bit/hrx-vulkan-patched. Merging.

bong-water-water-bong and others added 2 commits September 26, 2026 14:53
…59f6d37a by accident)

# Conflicts:
#	third_party/llama.cpp
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 47b03de

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Merged main into this branch. #154 had moved third_party/llama.cpp back to f59f6d37a, a stale checkout that lacks ZAYA1-VL (f30cc43) and 23 other commits. I resolved it to this PR's 8dd75eb, which contains both. I also regenerated the registry for 8dd75eb / a29be7f: the counts are unchanged and only the recorded pins moved. Both pins are the tips of their branches.

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit b71b296

@bong-water-water-bong
bong-water-water-bong merged commit 038e4f3 into main Sep 26, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the fix/hrx-decode-split-multipass-pin branch September 26, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant