Repository navigation
Pin llama.cpp cde002d: fix NaN after MTP rollback on Qwen3.5/3.8 (llama.cpp #52) - #257
Conversation
…the MTP-rollback NaN (llama.cpp #52) Qwen3.8-27B with --mtp on HRX0 aborted with NaN logits on some prompts (Q8_0, UD-Q5_K_XL, UD-Q6_K files). The same bug caused an AMDGPU memory fault on a repeated request without MTP. Both are fixed by the snapshot publish path forming c_t/c_s as exp(log c_t - log c_s). Registry regenerated (pin line); docs/hrx.md notes the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
left a comment
There was a problem hiding this comment.
Checked:
- Submodule:
764a256..cde002dcontains only llama.cpp #52 and its merge: one file,gated_delta_net_f32_wmma.loom, +28 / -12.cde002dis the current tip of1bit/hrx-vulkan-patched(checked withls-remote). - Registry: the
llama.cpp (hrx)line matches the new pin, and nothing else changed. - docs/hrx.md: the description matches what was measured, which is the 0/0 decay ratio in snapshot publishing, the prompt dependence, the affected files, and the memory fault on a repeated request without MTP.
One small suggestion for the note: the speeds (15.7 / 20.2 tok/s, and the UD-Q4_K_XL comparison) were measured in balanced power mode (85 W), so it's worth saying so next to the numbers, since the other figures in this file are from performance mode. Not blocking.
Fine to merge once checks pass.
🤖 Generated with Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Preparing review... |
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
Moves the llama.cpp pin from 764a256 to cde002d, the merge of llama.cpp #52.
The gated delta-net snapshot path now forms its decay ratios in log space. Previously,
exp(G_t) / exp(G_s)became 0/0 = NaN once a strongly forgetting head underflowed. This fixes two failures on HRX:--mtpaborting with "NaN logits at vocab index 0" on some prompts (Q8_0, UD-Q5_K_XL, UD-Q6_K files);Results, from #52:
--mtp: 0 NaN on prose and code (15.7 / 20.2 tok/s).--mtp: speed, draft acceptance and output text unchanged.Other changes in this PR:
--check-pinspasses.🤖 Generated with Claude Code