Repository navigation
docs: correct stale "not a server knob" claim in turbo-kv-cache.md - #1571
Merged
Merged
Conversation
MLXCEL_PAGED_ATTENTION_NATIVE gained a second, server-side consumer in issue #899 (resolve_paged_v2_dispatch in src/lib/mlxcel-core/src/layers.rs), which made the fused v2 kernel the production decode path and turned this variable's force-off values into its kill switch. docs/turbo-kv-cache.md still described it as library/bench-only, the same stale claim PR #1119 already corrected in docs/environment-variables.md. The rewritten paragraph names both consumers (the library-only pooled entry point retired by #710, and the server's pool-backed batched paged decode), states the #899 kill-switch and force-on behavior, keeps the #710 retirement as history rather than a current statement, and cross-links docs/environment-variables.md and docs/CONTINUOUS_BATCHING.md instead of duplicating the dispatch policy and log-line prose. The ADR 0001 link is unchanged, and docs/adr/0001-paged-attention-gather-vs-fused-kernel.md and the History clause in docs/environment-variables.md are left untouched since both intentionally describe the past.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/turbo-kv-cache.mdstill describedMLXCEL_PAGED_ATTENTION_NATIVEas a library/bench-only control, "not a server knob." That claim went stale when issue #899 made the fused v2 kernel the production decode path and turned this variable's force-off values into the server's kill switch, and the same claim was already corrected indocs/environment-variables.mdby PR #1119.What changed
### Measured payoffparagraph indocs/turbo-kv-cache.mdto name both current consumers ofMLXCEL_PAGED_ATTENTION_NATIVE: the library-only pooled entry point retired by perf(core): wire the pooled paged-attention decode entry point into the scheduler or retire it #710, and the server's pool-backed batched paged decode (resolve_dispatch_decisionandresolve_paged_v2_dispatchinsrc/lib/mlxcel-core/src/layers.rs).docs/environment-variables.mduses after PR docs: add the six paged-decode v2 variables and correct the NATIVE row #1119.docs/environment-variables.md#paged-decode-v2-variablesanddocs/CONTINUOUS_BATCHING.md#seeing-which-path-raninstead of duplicating the dispatch policy and log-line prose, and kept the existing ADR 0001 link.docs/adr/0001-paged-attention-gather-vs-fused-kernel.mdand theHistory:clause indocs/environment-variables.mduntouched, since both intentionally describe the past.Test plan
grep -rn "not a server" docs README.mdonly shows the ADR 0001 decision record and the twoHistory:clauses (indocs/environment-variables.mdand nowdocs/turbo-kv-cache.md), none asserting current behaviorpython3 scripts/ci/check_cross_repo_refs.pypassesgit diff --statshowsdocs/turbo-kv-cache.mdonly, no code touchedCloses #1139