server : skip checkpoints beyond pos_next

[aldehir]

Overview

While working on #24176, I noticed all the Gemma 4 models (12B and 26B in particular) were forgetful when sending a second identical initial message after a series of turns (e.g. the same agentic test case twice). I also noticed the issue did not appear when using cache_prompt = false.

I threw Fable at it, and it found an issue with checkpoints and SWA models. Here is a minimal repro to demonstrate the issue: checkpoint-restore-repro.py

master

=== request 1: first turn, baseline ===
finish_reason=stop content='pineapple'
=== grow turn 1: finish_reason=stop, 123 chars ===
=== grow turn 2: finish_reason=stop, 122 chars ===
=== grow turn 3: finish_reason=length, 108 chars ===
=== request 2: first turn again, diverges from the cached session ===
finish_reason=length content='0000 status=200 latency_ms=17\n2026-06-10T12:00:00Z service-a request_id=a-0000 status=200 latency_ms=17\n2026-06-10T12:01:07Z service-a request_id=a-0001 status=200 latency_ms=20\n2026-06-10T12:02:14Z service-a request_id=a-0002 status=200 latency_ms=23\n2026-06-10T12:03:21Z service-a request_id=a-0003'
FAIL: corrupted response after cache reuse (finish_reason=length, 850 chars)

PR

=== request 1: first turn, baseline ===
finish_reason=stop content='pineapple'
=== grow turn 1: finish_reason=stop, 123 chars ===
=== grow turn 2: finish_reason=stop, 122 chars ===
=== grow turn 3: finish_reason=length, 108 chars ===
=== request 2: first turn again, diverges from the cached session ===
finish_reason=stop content='pineapple'
PASS: secret word returned after cache reuse

Is this a legitimate issue and fix? It appears to have fixed the issues I was seeing.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES

[ggerganov]

This is a regression introduced by #23981. After the change, the checkpoint server information claims to cover a larger position range than it actually does (due to the new masking logic during saving the checkpoint).

I think a better fix would be to reflect the correct position range.

Something like this probably:

diff --git a/tools/server/server-context.cpp b/tools/server/server-context.cpp
index bdfa51718..1cff8ad61 100644
--- a/tools/server/server-context.cpp
+++ b/tools/server/server-context.cpp
@@ -2046,7 +2046,7 @@ private:
 
         auto & cur = slot.prompt.checkpoints.emplace_back();
 
-        cur.update_pos(slot.prompt.n_tokens() - n_tokens_cur, pos_min, pos_max);
+        cur.update_pos(slot.prompt.n_tokens() - n_tokens_cur, pos_max - n_swa, pos_max);
 
         cur.update_tgt(ctx_tgt,       slot.id, LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY);
         cur.update_dft(ctx_dft.get(), slot.id, LLAMA_STATE_SEQ_FLAGS_PARTIAL_ONLY);

[aldehir]

I tested your recommendation and it also fixes the issue. Also looks like it improves cache reuse, as it is selecting a checkpoint closer to the end in my limited testing. That said, I don't fully understand the implications.

[ggerganov]

Yup, I'll follow-up on this. It needs some unit tests and a more robust way to determine the position range of the saved checkpoints.