feat: Snapshot capacity hygiene: session-chain supersede and configurable budget #1146

Part of epic #1148.

Problem / Background

The default snapshot budget is 512 MiB: DEFAULT_SNAPSHOT_CAPACITY_BYTES = DEFAULT_CAPACITY_BYTES / 4 in src/server/prompt_cache/policy.rs. One conversation snapshot for gemma-4-31b-it-4bit is roughly 300-370 MB (observed 307,693,200 and 369,133,200 bytes on 2026-08-14, main at 9c154ff). In a two-turn run the second insert LRU-evicted the first: /v1/cache/stats showed snapshot_inserts = 2, snapshot_entries = 1. Even with prefix matching fixed, two concurrent conversations would thrash each other out of the store.

This affects every snapshot-only family (all supports_snapshot_reuse() models), and snapshot size varies widely with model size: a fixed 512 MiB holds roughly one 31B conversation but dozens of 4B ones, so no single constant is right.

There are two separable defects:

1. Within one session, a newer longer snapshot strictly dominates the older shorter one for future exact-prefix matches, yet both compete for budget until LRU happens to pick the right victim.
2. The budget itself is a fixed constant with no CLI or env override, and 512 MiB is sized without regard to the model's per-token state footprint.

Proposed Solution

(a) Session-chain supersede: in insert_snapshot (src/server/prompt_cache/store.rs ~line 642), when the new entry's token vector extends an existing entry's vector in the same session bucket, remove the superseded shorter entry explicitly instead of relying on LRU.

(b) Configurable, model-aware budget: expose snapshot_capacity_bytes, snapshot_max_entries, and snapshot_ttl through CLI flags and MLXCEL_* env vars, following the existing prompt-cache knobs in src/server/cli_input.rs. Derive the default from the model's per-token state footprint (for example, enough bytes for N conversations of a target token length) instead of the fixed 512 MiB.

Acceptance Criteria

• The observed thrash no longer occurs for a single conversation: repeated turns in one session keep exactly one (the longest) snapshot resident, with the superseded entry removed at insert time
• Two concurrent conversations of moderate length hold snapshots simultaneously under an appropriate budget, counter-verified (snapshot_entries = 2, no eviction between their turns)
• snapshot_capacity_bytes, snapshot_max_entries, and snapshot_ttl are settable via CLI flags and MLXCEL_* env vars, and documented in the server help text
• Supersede only fires when the new vector is a strict extension of the old within the same session bucket; unrelated sessions are untouched
• Unit tests beside the store cover supersede, cross-session isolation, and budget accounting after explicit removal

Technical Considerations

• Supersede must run before the capacity check so the freed bytes count toward admitting the new entry.
• The boundary snapshot (feat: Donate a history-boundary snapshot during prefill for snapshot-only families #1143) and warm-up snapshot (feat: Post-completion incremental warm-up snapshot for the next turn's history prefix #1144) insert more entries per session; supersede is what keeps their steady-state footprint at one chain per conversation.
• Keep llama-server flag-compatibility conventions in mind when naming the new flags (project principle: CLI/config compatibility).

Verification note (implementation PR). The supersede rule, its strict-extension and same-session boundaries, and the byte accounting are implemented in insert_snapshot and covered by five new tests in src/server/prompt_cache/store_tests.rs. The three knobs are wired to CLI flags and MLXCEL_* env vars and confirmed on a real checkpoint (qwen3.5-0.8b-4bit): the server logs and /v1/cache/stats both report the flag-supplied snapshot_capacity_bytes. The end-to-end HTTP demonstration of chain collapse is blocked on the sibling boundary-snapshot and warm-up-snapshot work in this epic: today's donated vector is prompt plus generated tokens and ends with a generation prompt, which the next turn's re-rendered prompt does not reproduce at that offset, so no same-session strict extension reaches the store yet. Measured on that run: conversation A donated vectors of 72, 98, 124, and 151 tokens against following prompts of 96, 122, and 149, so the length precondition held and the divergence is in the token ids at the turn boundary. The same run reproduced the reported thrash under a 40 MB budget (three resident entries, LRU eviction from the fourth turn onward, and a second conversation evicting the first's entries), which is the condition supersede removes once the boundary snapshot lands.