fix(server): prompt-cache ledger counts nominal paged bytes (32 B/block) so the capacity cap never triggers #226

Measurement context

Found during a cross-version server memory-footprint measurement (v0.1.4 vs main, 2026-06-12) using scripts/bench_memory_footprint.py (commit 18c88c1). Method: one fresh mlxcel serve process per (config, scenario), fixed HTTP workloads at temperature 0, sampling the process phys-footprint via /usr/bin/footprint at 2.5 s (RSS does not see MLX Metal buffers at all). Model: llama-3.2-1b-4bit on M1 Ultra, --max-batch-size 8. Scenarios: idle, shared_prefix_burst (8 concurrent requests sharing a ~3.7k-token system prompt), seqshare (same prefix, 8 sequential requests), multiturn (one conversation, 8 turns), churn (32 distinct prompts). Run-to-run noise: idle/multiturn within 0.5 percent, burst/seqshare/churn peaks within 10 to 25 percent.

Problem

Paged entries are accounted in the prompt-cache ledger using the layout's nominal bytes_per_block placeholder (32). Measured: a 10,576-token qwen3-0.6b entry holds 9,268 pool blocks, about 1.2 GiB of real fp16 KV, but the ledger and /v1/cache/stats record 296,576 bytes. The default capacity_bytes of 2 GiB can therefore retain thousands of times its intended budget; real retention is bounded only by TTL and max_entries.

Churn scenario (32 distinct prompts) settle footprint: main default 1538 MiB vs v0.1.4 994 MiB. That growth is real retention the cap should be bounding.

Fix

Account real bytes for paged entries (block count times real per-block KV bytes; reuse the execution::memory_estimate helpers such as paged_block_bytes instead of the nominal placeholder). Surface real bytes in /v1/cache/stats. Keep the nominal value only where the layout/serde layer requires it.

Acceptance criteria

• Stats bytes approximately equal real pool bytes for paged entries.
• With a small --prompt-cache-capacity-bytes, eviction actually triggers (regression test).
• Existing accounting and observability tests updated and green.

Related

Filed together from the same measurement: #224, #225, #227, #228.