Repository navigation
vulkan: Tune GDN kernel, fix Intel performance - #29476
Conversation
|
Those performance increase metrics look great on the Xe, nice work!! Is this expected to have some improvements on Xe3 as well? I built this PR and benchmarked Qwen3.6-27B Q6_K & Qwen3.8-27B Q8_K_L (UD) and note a very slight PP regression on the Intel B70 running on Windows with this PR applied. I am on the latest driver: 32.0.101.8976 PR 29476 - Qwen 3.6 27B Q6_K
build: b266105b5 (11226) PR 29476 - Qwen 3.8 27B Q8_K_L
build: b266105b5 (11226) Current source code build - Qwen 3.6 27B Q6_K
build: 4da6337 (11223) Current source code build - Qwen 3.8 27B Q8_K_L
build: 4da6337 (11223) |
|
I don't have access to any Xe2 or Xe3 hardware, so I cannot tell you anything about that. I would expect them to benefit too. |
|
What's the |
|
I didn't keep the tests, but it was just additional perf cases, something like this: |
|
On a B65: |
|
Happy to run more tests if you want to suggest a range of |
|
I am happy to run more tests too on my B70 if it helps, I just need to be guided to what commands/tests to do |
* vulkan: tune GDN shader * tune for Intel (cherry picked from commit 5c200e0)
…ix (ggml-org#29476) Exactly the 1,865 llama.cpp files the published llama-cpp-sys-2 0.1.158 crate ships (taken from upstream 26394b4, the llama.cpp it vendors), with upstream commit 5c200e0 ("vulkan: Tune GDN kernel, fix Intel performance", ggml-org#29476) applied to ggml/src/ggml-vulkan/ggml-vulkan.cpp. Nothing else changed. Only the crate's file set, because cargo checks out a git dependency's whole tree, and upstream's tools/ui paths exceed Windows MAX_PATH there (longest path here: 82 chars). Parentless, so the submodule fetch transfers only this tree. Retire once a llama-cpp-2 release vendors a llama.cpp that includes 5c200e0.
Points the llama.cpp submodule at Ethoseen/llama.cpp b8f1d912a70a1a05857beea45738d4b268df0841: the llama.cpp files the published 0.1.158 crate ships plus ggml-org/llama.cpp#29476 only. Everything else is the 0.1.158 release. Sightline uses it through [patch.crates-io]; retire once a llama-cpp-2 release carries the fix.
Overview
I noticed an opportunity for tuning and also that Intel's performance was terrible. This PR fixes that.
Test-backend-ops perf improvements
RTX 3090
AMD Radeon Pro VII was already optimal.
Intel A770
Model benchmark (Qwen 3.6 35B Q2_K)
Prompt processing, t/s
Token generation, t/s
Requirements