Repository navigation
Pin llama.cpp c9283cf: HRX ZAYA decode 67 -> ~90 tok/s (fork #36-#40) - #232
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Docs7 for 1bit-monster/engine
Commit |
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
Moves
third_party/llama.cpp45c1770 -> c9283cf (fork #36-#40). Regenerates the registry, and adds adocs/hrx.mdentry.ZAYA1-8B HRX0 decode 67 -> ~90 tok/s. Each step was measured in interleaved A/B runs on strixhalo:
zaya.cpp. Bitwise the same output, +2.4%.Each new
.loomfile carries a reference kernel andcheck.casedifferentials that pass underiree-test-loom --sanitizer=access.RFC #213 decode gate (tg128, HRX = this pin, Vulkan = weekly build, interleaved):
The 27B is the open gap. HRX also draws 150 W and peaks at 91 °C, against Vulkan's 91 W and 59 °C. This UD file is mostly Q5_K and IQ4_XS, which miss HRX's Q4_K/Q6_K token-1 fast paths. Routing the generic WMMA matmuls to the decode GEMV only reaches 7.55 tok/s, so it needs its own profile (it's a hybrid with gated delta-net layers). Separately, the HRX IQ4 decode fix (fork branch
1bit/hrx-iq4-fix) isn't merged yet.🤖 Generated with Claude Code