Skip to content

Pin llama.cpp cde002d: fix NaN after MTP rollback on Qwen3.5/3.8 (llama.cpp #52) - #257

Merged
bong-water-water-bong merged 2 commits into
mainfrom
pin/llama-cde002d
Oct 1, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
pin/llama-cde002d

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Moves the llama.cpp pin from 764a256 to cde002d, the merge of llama.cpp #52.

The gated delta-net snapshot path now forms its decay ratios in log space. Previously, exp(G_t) / exp(G_s) became 0/0 = NaN once a strongly forgetting head underflowed. This fixes two failures on HRX:

  • Qwen3.8-27B with --mtp aborting with "NaN logits at vocab index 0" on some prompts (Q8_0, UD-Q5_K_XL, UD-Q6_K files);
  • an AMDGPU memory fault on a repeated request without MTP.

Results, from #52:

  • Q8_0 with --mtp: 0 NaN on prose and code (15.7 / 20.2 tok/s).
  • Repeated request without MTP: clean.
  • UD-Q4_K_XL with --mtp: speed, draft acceptance and output text unchanged.

Other changes in this PR:

  • The registry is regenerated; only the pin line changes, and --check-pins passes.
  • docs/hrx.md gets a note on the fix.

🤖 Generated with Claude Code

…the MTP-rollback NaN (llama.cpp #52)

Qwen3.8-27B with --mtp on HRX0 aborted with NaN logits on some prompts
(Q8_0, UD-Q5_K_XL, UD-Q6_K files). The same bug caused an AMDGPU memory fault
on a repeated request without MTP. Both are fixed by the snapshot publish path
forming c_t/c_s as exp(log c_t - log c_s). Registry regenerated (pin line);
docs/hrx.md notes the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked:

  • Submodule: 764a256..cde002d contains only llama.cpp #52 and its merge: one file, gated_delta_net_f32_wmma.loom, +28 / -12. cde002d is the current tip of 1bit/hrx-vulkan-patched (checked with ls-remote).
  • Registry: the llama.cpp (hrx) line matches the new pin, and nothing else changed.
  • docs/hrx.md: the description matches what was measured, which is the 0/0 decay ratio in snapshot publishing, the prompt dependence, the affected files, and the memory fault on a repeated request without MTP.

One small suggestion for the note: the speeds (15.7 / 20.2 tok/s, and the UD-Q4_K_XL comparison) were measured in balanced power mode (85 W), so it's worth saying so next to the numbers, since the other figures in this file are from performance mode. Not blocking.

Fine to merge once checks pass.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

Preparing review...

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅

52 - Fully compliant

Compliant requirements:

  • Pin llama.cpp to cde002d commit
  • Fix NaN after MTP rollback on Qwen3.5/3.8
  • Update registry with new llama.cpp commit
  • Document the fix in docs/hrx.md
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Documentation Clarity

The documentation mentions that the fix resolves NaN logits at vocab index 0 and an AMDGPU memory fault, but does not specify the exact conditions under which these issues occurred or how to reproduce them. This lack of detail may make it difficult for others to understand the scope of the problem or verify the fix.

- **NaN after an MTP rollback on Qwen3.5/3.8
  ([llama.cpp #52](https://github.com/1bit-MONSTER/llama.cpp/pull/52)).** With `--mtp`, the gated
  delta-net kernel publishes one recurrent-state snapshot per verify token so a rejected draft can
  roll back. It formed each snapshot's decay ratio c_t/c_s by dividing two exponentials. When a
  strongly forgetting head drove both to 0, that gave 0/0 = NaN, and the next verify resuming from
  that snapshot returned NaN logits ("NaN logits at vocab index 0"). It depends on the prompt, and
  showed up on Qwen3.8-27B Q8_0, UD-Q5_K_XL and UD-Q6_K files. The same bug caused an AMDGPU memory
  fault on a repeated request without MTP. The ratios are now formed in log space, as the main
  recurrence already did. Q8_0 `--mtp` now runs clean on prose and code (15.7 / 20.2 tok/s, balanced power
  mode, 85 W); UD-Q4_K_XL `--mtp` speed and output are unchanged (also measured in balanced mode).

@context7

context7 Bot commented Oct 1, 2026

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ⚠️ Did not finish. View run

Commit 4a1c82b

@bong-water-water-bong
bong-water-water-bong merged commit 7abedc2 into main Oct 1, 2026
11 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pin/llama-cde002d branch October 1, 2026 10:39
bong-water-water-bong added a commit that referenced this pull request Oct 1, 2026
…255, #257) (#266)

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant