Eval bug: forcing full prompt re-processing in Qwen3-Coder-Next

Name and Version

version: 7900 (2634ed2)

Operating systems

Linux

GGML backends

CUDA

Hardware

3x3090

Models

Qwen3-Coder-Next

Problem description & steps to reproduce

(I couldn’t find an existing issue about this exact problem, please correct me if am wrong)

Is there any known workaround for:

forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory ...)

when using Qwen3-Coder-Next? it makes OpenCode practically unusable with this model, so I assume I am doing something incorrectly.

First Bad Commit

No response

Relevant log output

Logs

prompt eval time =   22316.21 ms / 16550 tokens (    1.35 ms per token,   741.61 tokens per second)
       eval time =     323.96 ms /    23 tokens (   14.09 ms per token,    71.00 tokens per second)
      total time =   22640.17 ms / 16573 tokens
slot      release: id  0 | task 0 | stop processing: n_tokens = 16572, truncated = 0
srv  update_slots: all slots are idle
srv  log_server_r: done request: POST /v1/chat/completions 192.168.0.196 200
srv  params_from_: Chat format: Qwen3 Coder
slot get_availabl: id  0 | task -1 | selected slot by LCP similarity, sim_best = 0.979 (> 0.100 thold), f_keep = 0.980
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 33 | processing task, is_child = 0
slot update_slots: id  0 | task 33 | new prompt, n_ctx_slot = 262144, n_keep = 0, task.n_tokens = 16589
slot update_slots: id  0 | task 33 | n_past = 16248, slot.prompt.tokens.size() = 16572, seq_id = 0, pos_min = 16571, n_swa = 1
slot update_slots: id  0 | task 33 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
slot update_slots: id  0 | task 33 | erased invalidated context checkpoint (pos_min = 16485, pos_max = 16485, n_swa = 1, size = 75.376 MiB)

Getting the same issue in Windows as well. (with both Vulkan and CUDA 13 builds)

CUDA 13 -> llama-b7950-bin-win-cuda-13.1-x64
Vulkan -> llama-b7956-bin-win-vulkan-x64, haven't tested in earlier version.

Are you using cache prompt? It should fix the problem.

I think OpenCode injects some remainders in the context that have the side effect of trashing the cache of recurrent models. Should be able to fix this by checkpointing a bit earlier:

diff --git a/tools/server/server-context.cpp b/tools/server/server-context.cpp
index b71d496ee..1de38e950 100644
--- a/tools/server/server-context.cpp
+++ b/tools/server/server-context.cpp
@@ -2507,7 +2507,7 @@ private:
                         slot.n_prompt_tokens_processed++;
 
                         // process the last few tokens of the prompt separately in order to allow for a checkpoint to be created.
-                        if (do_checkpoint && slot.task->n_tokens() - slot.prompt.n_tokens() == 64) {
+                        if (do_checkpoint && slot.task->n_tokens() - slot.prompt.n_tokens() == 1024) {
                             break;
                         }
                     }

Edit: nvm, we actually need to create checkpoints during the generation phase. Something similar to the proposal in #19298, but not during prompt processing, rather during generation.

I think OpenCode injects some remainders in the context that have the side effect of trashing the cache of recurrent models. Should be able to fix this by checkpointing a bit earlier:

Wow, I can’t reproduce this anymore after rebuilding current master with your fix.

Are you using cache prompt? It should fix the problem.

yes, 60000

Nice. I've seen that the reminders are being injected, but haven't looked to understand the exact logic of OpenCode to inject them, so atm not sure what is the best solution. It's possible that this patch is enough.

Taking a note of this for #19298 , I guess this should be configurable :)

I'm running into the same thing with v7955 when playing around with Coder Next in Opencode. Prompt caching works between tool calls within the session but then full re-processing is forced when the model is done and I write another message in the same session. Can also confirm that the patch fixes it for me on v7955!

are there any side effects?

It can skip creating checkpoints if -batch is less than 1024 (the default is 2048). More robust solution is to make this respect the batch size, such as min(n_batch, 1024).

@jacekpoplawski

@jacekpoplawski @konekohana Could you provide feedback if #19408 works OK?

This did not fix it for me. What information would I need to provide to help. Fresh build just now at 5pm est 2/8

This did not fix it for me. What information would I need to provide to help. Fresh build just now at 5pm est 2/8

llama-server complete logs

I think this happens less often after setting the cache size to a fixed value instead of using -1.

Maybe -1 doesn't reserve ram for the cache, so it is used for other purposes and not available when required.

I think this happens less often after setting the cache size to a fixed value instead of using -1.

Maybe -1 doesn't reserve ram for the cache, so it is used for other purposes and not available when required.

What cache value are you using now? I'm running into the same problems.

What cache value are you using now? I'm running into the same problems.

The 16 GB I had left, before RAM swapping and OOM killer started to kick in.

[39591] slot update_slots: id 0 | task 75 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/1lama.cpp/pull/13194#issuecomment-2868343055)
[39591] slot update_slots: id 0 | task 75 | erased invalidated context checkpoint (pos_min = 6892, pos_max = 6892, n
_tokens = 6893, n_swa = 1, size = 75.376 MiB)

#13194

CONFIGURATION:

LLAMA-CLI --VERSION: 8211 (a0ed91a)
model = Qwen3-Coder-Next-UD-Q6_K_XL-00001-of-00003.gguf
n-gpu-layers = 999
ctx-size = 128000
temp = 1.0
top-p = 0.95
top-k = 40
min-p = 0.01
threads = 12
flash-attn = on
mlock = off
mmap = off
fit = off
warmup = off
batch-size = 4096
ubatch-size = 512
cache-type-k = q8_0
cache-type-v = q8_0
jinja = true
direct-io = on
cache-prompt = true
cache-reuse = 256
cache-ram = 5000

Still getting the same issue with non -1 fixed ram cache size in params. Would appreciate direction here.

Still getting the same issue with non -1 fixed ram cache size in params. Would appreciate direction here.

Does it occur less often if you increase cache-ram ?

Does it occur under certain conditions, like when the agent is condensing the prompt?

This happens with both Qwen3-Coder-Next and Qwen3.5-122B-A10B, likely all similar models.

In my use case there is a very long system prompt (~22k tokens) and cannot afford more than 1 slot due to RAM constraint. Without the prompt cache working these (otherwise perfect) models are not usable in llama.cpp for such use cases.

The same problem persists, not sure why they keep closing it.

I reproduced this with Qwen3.5-122B-A10B Q4KL, talking through OpenCode in the same context window, and suddenly:

forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)

Both Qwen 3.5 9B and also Gemma 4 26B did it to me today. I'm using the TurboQuant branch from TheTom's repo, which is pretty up to date. Using Claude Code and pointing at llama-server running locally.

slot update_slots: id  0 | task 1008 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
slot update_slots: id  0 | task 1008 | erased invalidated context checkpoint (pos_min = 8191, pos_max = 8191, n_tokens = 8192, n_swa = 0, pos_next = 0, size = 50.251 MiB)

I have plans to debug exactly what OpenCode is doing and why llama.cpp is getting confused (that's my number one issue with llama.cpp - happens with Gemma 4 31B all the time)

The first problem with opencode is model-independent, for some reason opencode is moving system-reminder from place to place.

To reproduce:

start with plan mode, say something
switch to build model, say something
try to use tools
again say something even simple

notice unnecessary prompt processing - system-reminder is moving from the middle to the last part of prompt

server: fix context checkpoint restore for hybrid/recurrent models

Same problem with Qwen3.6 27B:

slot launch_slot_: id  0 | task 2505 | processing task, is_child = 0
slot update_slots: id  0 | task 2505 | new prompt, n_ctx_slot = 140032, n_keep = 0, task.n_tokens = 88675
slot update_slots: id  0 | task 2505 | n_past = 85490, slot.prompt.tokens.size() = 89584, seq_id = 0, pos_min = 89583, n_swa = 0
slot update_slots: id  0 | task 2505 | Checking checkpoint with [89323, 89323] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [89098, 89098] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88974, 88974] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88879, 88879] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88789, 88789] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88622, 88622] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88535, 88535] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88439, 88439] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88257, 88257] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88138, 88138] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [88000, 88000] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [87877, 87877] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [87749, 87749] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [87633, 87633] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [87512, 87512] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [87392, 87392] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [87268, 87268] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [87152, 87152] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86967, 86967] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86844, 86844] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86768, 86768] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86685, 86685] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86609, 86609] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86526, 86526] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86450, 86450] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86367, 86367] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86291, 86291] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86211, 86211] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86135, 86135] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [86055, 86055] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [85979, 85979] against 85490...
slot update_slots: id  0 | task 2505 | Checking checkpoint with [85899, 85899] against 85490...
slot update_slots: id  0 | task 2505 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 85899, pos_max = 85899, n_tokens = 85900, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 85979, pos_max = 85979, n_tokens = 85980, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86055, pos_max = 86055, n_tokens = 86056, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86135, pos_max = 86135, n_tokens = 86136, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86211, pos_max = 86211, n_tokens = 86212, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86291, pos_max = 86291, n_tokens = 86292, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86367, pos_max = 86367, n_tokens = 86368, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86450, pos_max = 86450, n_tokens = 86451, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86526, pos_max = 86526, n_tokens = 86527, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86609, pos_max = 86609, n_tokens = 86610, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86685, pos_max = 86685, n_tokens = 86686, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86768, pos_max = 86768, n_tokens = 86769, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86844, pos_max = 86844, n_tokens = 86845, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 86967, pos_max = 86967, n_tokens = 86968, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 87152, pos_max = 87152, n_tokens = 87153, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 87268, pos_max = 87268, n_tokens = 87269, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 87392, pos_max = 87392, n_tokens = 87393, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 87512, pos_max = 87512, n_tokens = 87513, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 87633, pos_max = 87633, n_tokens = 87634, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 87749, pos_max = 87749, n_tokens = 87750, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 87877, pos_max = 87877, n_tokens = 87878, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88000, pos_max = 88000, n_tokens = 88001, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88138, pos_max = 88138, n_tokens = 88139, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88257, pos_max = 88257, n_tokens = 88258, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88439, pos_max = 88439, n_tokens = 88440, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88535, pos_max = 88535, n_tokens = 88536, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88622, pos_max = 88622, n_tokens = 88623, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88789, pos_max = 88789, n_tokens = 88790, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88879, pos_max = 88879, n_tokens = 88880, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 88974, pos_max = 88974, n_tokens = 88975, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 89098, pos_max = 89098, n_tokens = 89099, n_swa = 0, pos_next = 0, size = 149.626 MiB)
slot update_slots: id  0 | task 2505 | erased invalidated context checkpoint (pos_min = 89323, pos_max = 89323, n_tokens = 89324, n_swa = 0, pos_next = 0, size = 149.626 MiB)

Please test this solution #22929

Still happening on

samuelcardillo/Qwen3-Coder-Next-Opus-4.6-Reasoning-Distilled-GGUF

Navigation Menu

Repository navigation

Eval bug: forcing full prompt re-processing in Qwen3-Coder-Next #19394

Name and Version

Operating systems

GGML backends

Hardware

Models

Problem description & steps to reproduce

First Bad Commit

Relevant log output

3 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Eval bug: forcing full prompt re-processing in Qwen3-Coder-Next #19394

Description

Name and Version

Operating systems

GGML backends

Hardware

Models

Problem description & steps to reproduce

First Bad Commit

Relevant log output

Activity

anubhavgupta commented on Feb 6, 2026

pwilkin commented on Feb 6, 2026

ggerganov commented on Feb 6, 2026

jacekpoplawski commented on Feb 6, 2026

jacekpoplawski commented on Feb 6, 2026

ggerganov commented on Feb 6, 2026

pwilkin commented on Feb 6, 2026

konekohana commented on Feb 6, 2026

jacekpoplawski commented on Feb 6, 2026

ggerganov commented on Feb 6, 2026

ggerganov commented on Feb 7, 2026

kj-c0d3s commented on Feb 8, 2026

kripper commented on Feb 16, 2026

3 remaining items

kripper commented on Mar 2, 2026

ShibeTemple commented on Mar 4, 2026

kripper commented on Mar 5, 2026

ShibeTemple commented on Mar 5, 2026

kripper commented on Mar 6, 2026

viktor-ferenczi commented on Mar 8, 2026

asf0 commented on Mar 24, 2026

Ezzz-dev commented on Mar 31, 2026

jhughes2112 commented on Apr 15, 2026

jacekpoplawski commented on Apr 15, 2026

jacekpoplawski commented on Apr 17, 2026

corrm commented on May 13, 2026

jacekpoplawski commented on May 13, 2026

de-flandres commented on May 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions