You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Disaggregated / decode-node handoff must transfer paged block tables and pool blocks, not dense snapshots, once the unified store is the source of truth.
Tasks
Extend src/distributed/kv_cache_serde/ to serialize PagedSequenceState block tables plus the referenced pool blocks and restore them onto the decode-node pool. (Fp16 pool-backed path; the block table is restored via the metadata path while pool block CONTENTS are transferred and re-materialized on fresh blocks. Turbo sidecars are out of scope for the Fp16-only path.)
Validate a prefill-node to decode-node handoff reconstructs identical KV and identical first-token logits. (Covered by the #[ignore] real-model parity test tests/paged_kv_serialize_parity.rs, run on Metal.)
Confirm refcount and block ownership are correct after restore (no double-free, no leak). (CPU unit tests assert per-block refcount == 1 and live-block-count parity after restore.)
Acceptance criteria
A disaggregated run reproduces single-node output within tolerance using the paged block-transfer path. (Covered by the #[ignore] parity test, run on Metal; the live disaggregated scheduler/transport wiring is the epic capstone End-to-end validation, parity, benchmarks, and docs #126.)
Block accounting after restore matches the originating node. (CPU unit tests assert paged_stats() + live_block_count() parity between origin and decode pools.)
Implementation status
Block-content transfer ("Approach A") is implemented in the kv_cache_serde module's public flow plus the supporting mlxcel-core pool/cache primitives. Wiring serialize/restore into the live disaggregated scheduler and a real transport is deferred to #126. See the PR for details.
Activity
feat: unify radix prompt cache with paged block pool for prefix shari…
fix: correct fp16/bf16 byte round-trip in paged KV cache serde
style: apply rustfmt to issue #125 paged KV serde changes
test: guard fp16/bf16 from_bytes round-trip at FFI and serde layers
feat: serialize paged KV block contents for node handoff (#185)
feat: harden paged KV handoff deserialization boundary (#186)