Misc. bug: PR19849 causes inconsistent sequence positions

Name and Version

$ llama-server --version
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce GTX 1080 Ti, compute capability 6.1, VMM: yes
version: 8151 (c747294)
built with GNU 13.3.0 for Linux x86_64

EDIT: Oops. I didn't know I'd need the "--version" output as part of a bug report, so I'd already rebuilt my binaries to the latest master without the changes that cause the issue. #19849 (commit d7d826b) and forward show the problem, the version shown above (from my cli output) does NOT show the problem.

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

llama-server \
        --port 8080 --jinja \
        --parallel 1 \
        -hf unsloth/Qwen3-Coder-Next-GGUF:UD-Q4_K_XL \
        --ctx-size 200000 \
        --batch-size 256 \
        --ubatch-size 128 \
        --temp 1.0 \
        --min-p 0.01 \
        --top-p 0.95 \
        --top-k 40 \
        -fitt 1

Problem description & steps to reproduce

Happens regularly for me by connecting claude code to the server and merely prompting it more than one or two times.

First Bad Commit

d7d826b

Relevant log output

Logs

[...]
init: the tokens of sequence 0 in the input batch have inconsistent sequence positions:
 - the last position stored in the memory module of the context (i.e. the KV cache) for sequence 0 is X = 43686
 - the tokens for sequence 0 in the input batch have a starting position of Y = 43688
 it is required that the sequence positions remain consecutive: Y = X + 1
decode: failed to initialize batch
llama_decode: failed to decode, ret = -1
[...]

Can confirm

LLAMA_SET_ROWS=1 /usr/local/bin/llama-server --repeat-penalty 1.10
--no-mmap -fa on -b 2048 -ub 2048 -cram 32768
--seed 3407 --temp 1.0 --top-p 0.95 --min-p 0.01 --top-k 40
-hf unsloth/Qwen3-Coder-Next-GGUF:UD-IQ3_XXS --ctx-checkpoints 256

slot update_slots: id 3 | task 1283 | prompt processing progress, n_tokens = 107099, batch.n_tokens = 647, progress = 0.995242
init: the tokens of sequence 3 in the input batch have inconsistent sequence positions:

the last position stored in the memory module of the context (i.e. the KV cache) for sequence 3 is X = 106450
the tokens for sequence 3 in the input batch have a starting position of Y = 106452
it is required that the sequence positions remain consecutive: Y = X + 1
decode: failed to initialize batch
llama_decode: failed to decode, ret = -1
srv update_slots: Invalid input batch. i = 0, n_batch = 2048, ret = -1
srv send_error: task id = 1283, error: Invalid input batch.
slot release: id 3 | task 1283 | stop processing: n_tokens = 107099, truncated = 0
srv stop: cancel task, id_task = 1283
slot prompt_clear: id 3 | task -1 | clearing prompt with 107099 tokens

Changing

return std::min((size_t)(max_pos + 1), tokens.size());

to

return std::min((size_t)(max_pos), tokens.size());

fixes.

Changing

return std::min((size_t)(max_pos + 1), tokens.size());

to

return std::min((size_t)(max_pos), tokens.size());

fixes.

can confirm. and ty. I was going crazy. this fix made unsloth/Qwen3.5-122B-A10B-GGUF:MXFP4_MOE stop invalidating every requests cache over 16k context.

Do you have repro steps that I can use? Preferably with the llama-server WebUI?

I did this
compile llama.cpp then run it
unsloth/Qwen3.5-122B-A10B-GGUF:MXFP4_MOE started failing consistently around 16k tokens of context.
unsloth/Qwen3-Next-80B-A3B-Instruct-UD-Q4_K_XL.gguf after an update of the model, started failing consistently around 100k tokens of context.

failed to truncate tokens with position >= [NUM] - clearing the memory
followed by clearing the slots..... I don't have the exact error anymore. but it was obvious and reproducible for me.

patch

--- a/tools/server/server-common.cpp
+++ b/tools/server/server-common.cpp
@@ -276,7 +276,7 @@ llama_pos server_tokens::pos_next(int64_t n_tokens) const {
 
 size_t server_tokens::size_up_to_pos(llama_pos max_pos) const {
     if (!has_mtmd) {
-        return std::min((size_t)(max_pos + 1), tokens.size());
+        return std::min((size_t)(max_pos ), tokens.size());
     }
 
     size_t idx = 0;

this made things much much more stable and I'm easily 125k tokens without issue now on unsloth/Qwen3.5-122B-A10B-GGUF:MXFP4_MOE

To add to the confusion, I first noticed something was up here, after an accidental update of the model. llama.cpp hadn't changed and was probably two weeks old.
https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF/discussions/17#699d8b9126f7521eb0c21b59

Then I grabbed Qwen3.5-122B-A10B-GGUF:MXFP4_MOE and had the same experience but at 16k onward. using a fresh build. reproed other model. and tested both after the patch.

apologies if this isn't a good repro, but it's what I noticed. happy to help if I can provide something more useful.

my current command after butchering it a bunch of different ways. The patch is what helped me though on both models.
./build/bin/llama-server -hf unsloth/Qwen3.5-122B-A10B-GGUF:MXFP4_MOE --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.00 --host 0.0.0.0 --port 39281 --threads 16 --chat-template-kwargs {"enable_thinking": false} -np 1

Could you confirm that #19924 resolves the issues?

[100%] Linking CXX executable ../../bin/llama-fit-params
[100%] Built target llama-fit-params
wer@blender:~/src/llama.cpp$ git status
HEAD detached at 6e0c296
nothing to commit, working tree clean

I tested the two models I mentioned, each for a few exchanges starting around 120k tokens and had no weirdness.

#19924 Seems to resolved my cache issue issue.

#19924 resolves for me as well.

Navigation Menu

Repository navigation

Misc. bug: PR19849 causes inconsistent sequence positions #19901

Name and Version

Operating systems

Which llama.cpp modules do you know to be affected?

Command line

Problem description & steps to reproduce

First Bad Commit

Relevant log output

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Misc. bug: PR19849 causes inconsistent sequence positions #19901

Description

Name and Version

Operating systems

Which llama.cpp modules do you know to be affected?

Command line

Problem description & steps to reproduce

First Bad Commit

Relevant log output

Activity

gundemirbas commented on Feb 25, 2026

tiau commented on Feb 25, 2026

billtown commented on Feb 26, 2026

ggerganov commented on Feb 26, 2026

billtown commented on Feb 26, 2026

ggerganov commented on Feb 26, 2026

billtown commented on Feb 26, 2026

tiau commented on Feb 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions