Repository navigation
webgpu: add DUP support - #30002
webgpu: add DUP support#30002yomaytk wants to merge 1 commit into
Conversation
|
Unfortunately, I temporarily cannot access my local NVIDIA machine, so I'll check it using CI. I'll mark this as ready once CI passes. Edit: Models Backend Check / webgpu-nvidia passes. |
|
@ggerganov I confirmed that #28075 passes |
|
Closing since this was superseded by #30216. |
|
Yes, I also missed this PR. |
|
I test #30219 because this is the next "Fix CI" to implement, but this time I checked to make sure a PR didn't already exist 😂 |
Ah, it’s OK! I should have used a clearer PR title mentioning the CI fix :) |
|
No worries at all! And it's even better than a CI fix, since it enables the DUP op on the backend. For #30219, I went through the dispatch sites: you moved the whole class of row and element wise ops to the 2D grid. Asking an LLM for a CI fix is a double edged sword: by default it patches the failing site and forgets the other entry points, which is exactly what your PR avoids. Once the format line is fixed and it's rebased, we should get a fully green CI! |
Overview
This PR adds dup support to WebGPU, which fixes the following CI error by the new code of #29622 on my M5 (the same test failed with a Bus error on my M5, but it passes with this change).
https://github.com/ggml-org/llama.cpp/actions/runs/37244334061/job/111559110058
In the following llama-graph.cpp code (line 2488), this error seems to be rooted in the fact that the DUP nodes are assigned to the CPU and WebGPU tries to bind the CPU buffer on the GPU side.
inps[2] = ggml_set_rows(ctx0, ggml_dup(ctx0, inp->mixed_embd), build_tok(inp->mixed_tokens), inp->mixed_slots);Additional information
Requirements