You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
HRX: wrong answers for every prompt longer than one ubatch since fork #82 (graph inputs written back to host) #319
Found 2026-10-05 while root-causing #313 (strixhalo, performance mode). On the llama.cpp fork tip (e57beb9 = the pin merged in #311), Qwen3-0.6B Q4_K_M on HRX0 answers a 4,798-token prompt by repeating the question — 28/28 runs, same hash (8d294327), with and without the private add-on. Short prompts (one ubatch, ≤512 tokens) are unaffected, which is why the e2e and "Paris." gates passed.
Cause.qwen_attention_metadata binds the positions, the K/V cache indices and the mask as read_write, so they are queued as deferred host writebacks. Since fork #82 (e0f637b) every upload first publishes pending writebacks — which overwrites the next ubatch's fresh inputs with the previous ubatch's values. Every multi-ubatch Qwen prompt is affected.
Fix. Fork commit "ggml-hrx: never write a device-rewritten graph input back to host memory": a graph_input flag on such bindings — uploaded, never downloaded. Same PR carries "wait for a non-replay command program that reads registered host memory" (graph replay off answered "123" instead of 7341: the non-replay path returned before its own device-side input copy ran, and ggml overwrote the host buffer).
The NaN in #313 itself is a separate, third bug (Loom compiler GFX11 wave64 SGPR lane-mask hazard; see #313).
After the fork PR merges the engine pin moves; until then --device hrx with prompts over 512 tokens is wrong on main.
Fixed: graph inputs are never written back to host memory (fork #90, b4e7515), in the engine since llama.cpp pin 522dab47 (#320, main 11d6e2f). Verified on strixhalo at that pin: Qwen3-0.6B, 4,798-token needle prompt after a short request, 20/20 correct (hash 5fb9f8b5, names 7341), graph replay on and off identical.
Activity
bong-water-water-bong commentedon Oct 5, 2026
Fixed: graph inputs are never written back to host memory (fork #90, b4e7515), in the engine since llama.cpp pin 522dab47 (#320, main 11d6e2f). Verified on strixhalo at that pin: Qwen3-0.6B, 4,798-token needle prompt after a short request, 20/20 correct (hash 5fb9f8b5, names 7341), graph replay on and off identical.
HRX hardware correctness gate on Strix Halo (hw-gate) (#403)