Repository navigation
metal : optimize argmax and pool_1d - #30862
Merged
Merged
Conversation
Add a two-pass partial reduction for large-row argmax and use cooperative threadgroup reductions for pool_1d. Allocate scratch space for the argmax partials through the Metal buffer allocator. Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)
github-actions
Bot
added
ggml
Add representative POOL_1D max cases for the MoE down-projection amax shapes, reducing the whole intermediate dim. Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)
ggerganov
marked this pull request as ready for review
October 20, 2026 09:07
edwardyoon
pushed a commit
to edwardyoon/focus-llama
that referenced
this pull request
Oct 24, 2026
* metal : optimize argmax and pool_1d Add a two-pass partial reduction for large-row argmax and use cooperative threadgroup reductions for pool_1d. Allocate scratch space for the argmax partials through the Metal buffer allocator. Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * tests : add pool_1d perf cases Add representative POOL_1D max cases for the MoE down-projection amax shapes, reducing the whole intermediate dim. Assisted-by: pi:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * cont : minor fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
rel #30861
Optimize the Metal implementations of
ggml_argmaxandggml_pool_1d.ggml_argmaxon large rows: a part kernel writes per-chunk partial maxima/indices to a scratch buffer, and a final kernel reduces them per row.ggml_pool_1d, so large kernel windows are reduced across up to 256 threads instead of one thread per output.Additional information
POOL_1Dperf cases for the MoE down-projection amax shapes.Performance
Measured with
test-backend-ops perf -b MTL0 -o "ARGMAX,POOL_1D".Apple M2 Ultra
POOL_1D max, full-width reduction:
ARGMAX:
Apple M5 Max
POOL_1D max, full-width reduction:
ARGMAX:
Requirements
pi:llama.cppMac Studio, M2 Ultra, 192GB◷ 3.7h · ⚙ 18.0m · ↑ 574k · ↓ 282k · ≡ 1005 · ⌨ 31