Skip to content

Pin llama.cpp 4380daf: ZAYA1-74B (sliding-window layers) on Vulkan - #127

Merged
bong-water-water-bong merged 2 commits into
mainfrom
pin-zaya-74b
Sep 26, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
pin-zaya-74b

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Moves third_party/llama.cpp from 1e775cd (#119) to 4380daf, the merge of 1bit-MONSTER/llama.cpp#11. That adds ZAYA1-74B-preview's sliding-window layers (4,097-token window, rope theta 1e4) and its second rope base (1e7). The pin is the head of 1bit/hrx-vulkan-patched.

Checked on Strix Halo:

  • ZAYA1-8B unchanged. Q4_K_M perplexity on Vulkan is 21.5731 before and after, and test-llama-archs -a zaya passes on Vulkan and CPU.
  • ZAYA1-74B-preview. Q4_K_M (45.7 GB) passes tests/serve_e2e.sh through 1bit serve --device vulkan --ctx-size 8192, run under mem-guard.sh. There's no reference check: a 74B FP32 transformers run doesn't fit on this machine, and docs/vulkan.md says so.

registry/architectures.json is regenerated. It had not been updated when #119 moved the pin, and still named 5556bf2; it now records 4380daf.

🤖 Generated with Claude Code

…bit-MONSTER/llama.cpp #11)

The pin adds sliding-window attention and a second rope base for ZAYA1-74B-preview on top of
1e775cd (#119). ZAYA1-8B is unchanged: Q4_K_M perplexity 21.5731 on Vulkan before and after,
and test-llama-archs -a zaya passes. ZAYA1-74B-preview Q4_K_M passes serve_e2e through
1bit serve --device vulkan.

registry/architectures.json, regenerated: it records the new pin (it still named 5556bf2
after #119 moved the pin to 1e775cd without regenerating it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit bed4e9f

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit bed4e9f)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

119 - Partially compliant

Compliant requirements:

Non-compliant requirements:

  • None

Requires further human verification:

  • None

11 - Partially compliant

Compliant requirements:

  • Move engine's llama.cpp to AMD's maintained HRX line
  • Pin to AMD's tested pair and keep current with daily workflow
  • Update submodules and build system
  • Update device name from HRX20 to HRX0
  • Update documentation and tests
  • Verify on Strix Halo with speed improvements

Non-compliant requirements:

  • None

Requires further human verification:

  • None
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Documentation Clarity

The documentation mentions ZAYA1-74B-preview's sliding-window layers and rope theta values, but does not specify the exact impact of these changes on performance or memory usage beyond the context of the vulkan device. Clarification on how these parameters affect inference behavior would be beneficial for users.

ZAYA1-74B-preview (1bit-MONSTER/llama.cpp #11) alternates sliding-window layers (a 4,097-token
window, rope theta 1e4) with full-attention layers (theta 1e7); the converter writes the window,
the per-layer pattern and the second rope base, and ZAYA1-8B, which has no sliding layers, is
unchanged (same perplexity to four decimals). Its Q4_K_M (45.7 GB) passes `tests/serve_e2e.sh`
on `--device vulkan` with `--ctx-size 8192`; there is no reference check for it here, since a
74B FP32 transformers run does not fit on Strix Halo.

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit bed4e9f

@bong-water-water-bong
bong-water-water-bong merged commit fa3e425 into main Sep 26, 2026
5 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pin-zaya-74b branch September 26, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant