Repository navigation
test: extend paged KV cache parity to llama3 - #169
Merged
Merged
Conversation
Generalize the scheduler and prefix-share real-model parity tests to cover both pool-backed families — qwen3 and llama3 (Fp16 dense-natural backend) — via shared per-family helpers, completing sub-step (c) of #121's a/b/c decomposition. The scheduler test runs single-sequence and batched (B=2) paged-vs-dense parity per family; the prefix-share DoD test runs the radix-store adopt/share-vs-cold parity per family. All six cases produce byte-identical greedy tokens. Model-owned-state families (gemma3/llama4/qwen3.5) and quantized-KV (Int8/Turbo) paths keep the dense backend and are out of scope for pool parity; the pool-backing gate in cache.rs already documents this, and both test module docs now reinforce it.
8 tasks
be-free-snowboy
pushed a commit
to freestylejs/snowbrain-mlxcel
that referenced
this pull request
Oct 27, 2026
…ablup#169) Generalize the scheduler and prefix-share real-model parity tests to cover both pool-backed families — qwen3 and llama3 (Fp16 dense-natural backend) — via shared per-family helpers, completing sub-step (c) of lablup#121's a/b/c decomposition. The scheduler test runs single-sequence and batched (B=2) paged-vs-dense parity per family; the prefix-share DoD test runs the radix-store adopt/share-vs-cold parity per family. All six cases produce byte-identical greedy tokens. Model-owned-state families (gemma3/llama4/qwen3.5) and quantized-KV (Int8/Turbo) paths keep the dense backend and are out of scope for pool parity; the pool-backing gate in cache.rs already documents this, and both test module docs now reinforce it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Final sub-step (c) of #121, completing its a/b/c decomposition (a: scheduler→pool wiring #167 → b: radix↔pool unification #168 → c: this). Closes #121.
Extends the real-model paged-KV parity coverage from qwen3 to both pool-backed families — qwen3 and llama3 (Fp16 dense-natural backend) — and reinforces the documented pool-backing scope.
Changes
tests/paged_scheduler_parity.rs— generalized to per-family helpers (assert_single_sequence_parity,assert_batched_decode_parity, parameterizedload_or_skip). Runs single-sequence and batched (B=2) paged-vs-dense parity for qwen3 (qwen3-0.6b-4bit) and llama3 (llama-3.2-1b-4bit).tests/paged_prefix_share_parity.rs— generalized to a per-family helper (assert_prefix_share_parity, parameterizedload_or_skip+ cache key). Runs the radix-store adopt/share-vs-cold DoD for qwen3 and llama3.The shared deterministic prompt ids are < 128k, valid for both vocabularies; parity compares identical-input runs, so the (semantically arbitrary) tokens only need to match across backends.
Scope (documented, not changed here)
Pool-backing is gated to Fp16 dense-natural-backend sequences (qwen3/llama3) at
CachePool::allocate_with_layout(cache.rs). Model-owned-state families (gemma3/llama4/qwen3.5) and quantized-KV (Int8/Turbo) paths keep the dense backend and are out of scope for pool parity — their decode path is unchanged by #121. Bringing those onto the pool is future work (overlaps #124). The gate comment already states this; both test module docs now reinforce it.Validation (orchestrator-run, M1 Ultra, metal+accelerate, release)
All six cases pass with byte-identical greedy tokens (stronger than the RMS<5e-3 DoD):
paged_scheduler_parity(4): qwen3/llama3 × {single-sequence, batched B=2} — paged == dense.paged_prefix_share_parity(2): qwen3/llama3 — adopted prefix-share == cold no-cache run.Cold
cargo clippy --testsandcargo fmt --checkclean.