Skip to content

vulkan: don't use std::string in load_shaders, to improve compile time - #15724

Merged
0cc4m merged 2 commits into
ggml-org:masterfrom
jeffbolznv:load_shaders_const_char
Sep 3, 2025
Merged

vulkan: don't use std::string in load_shaders, to improve compile time#15724
0cc4m merged 2 commits into
ggml-org:masterfrom
jeffbolznv:load_shaders_const_char

Conversation

@jeffbolznv

@jeffbolznv jeffbolznv commented Sep 1, 2025

Copy link
Copy Markdown
Contributor

This improves the incremental build time for ggml-vulkan.cpp pretty significantly:

before
msvc: 23s
wsl/gcc: 72s

after
msvc: 10s
wsl/gcc: 29s

@LostRuins I'm curious how much this helps on your system.

@jeffbolznv
jeffbolznv requested a review from 0cc4m as a code owner September 1, 2025 22:39
@github-actions github-actions Bot added Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning labels Sep 1, 2025
@0cc4m

0cc4m commented Sep 2, 2025

Copy link
Copy Markdown
Contributor

Is the problem the concatenation or the index to string conversion? Cause this does make it a little harder to track which shader is being used. Maybe we can find a better way to keep the information that doesn't affect the compile time, like a second string or a simple integer index.

@jeffbolznv

jeffbolznv commented Sep 2, 2025

Copy link
Copy Markdown
Contributor Author

The speedup comes from changing the function parameters in ggml_vk_create_pipeline to const char *. I suspect the compiler is converting a bunch of const char * to std::string at compile time and it's expensive because there are so many.

I've pushed another commit that keeps the full string for those that were using it. Compile time is still good.

@LostRuins

LostRuins commented Sep 3, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Hi @jeffbolznv, yes, this seems to improve my compile times for ggml vulkan as well.
With -O3 it brings me from 1m 17.26s to 0m 39.70s, so it's a pretty good speedup.

This seems in line with your own findings.

0cc4m
0cc4m approved these changes Sep 3, 2025
View reviewed changes

@0cc4m 0cc4m left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't think of something better currently, so it's fine for now.

@0cc4m
0cc4m merged commit 0fce7a1 into ggml-org:master Sep 3, 2025
gabe-l-hart added a commit to gabe-l-hart/llama.cpp that referenced this pull request Sep 4, 2025
@gabe-l-hart
Merge remote-tracking branch 'origin/master' into gabe-l-hart/alora-s…
d8651c8 
…upport

* origin/master: (72 commits)
metal : Add template specialization for mul_mm_id w/ ne20 == 10 (ggml-org#15799)
llama : set n_outputs to 1 to avoid 0 outputs mean-pooling (ggml-org#15791)
CANN: Refactor ND to NZ workspace to be per-device (ggml-org#15763)
server: add exceed_context_size_error type (ggml-org#15780)
Document the new max GPU layers default in help (ggml-org#15771)
ggml: add ops for WAN video model (cuda && cpu) (ggml-org#15669)
CANN: Fix precision issue on 310I DUO multi-devices (ggml-org#15784)
opencl: add hs=40 to FA (ggml-org#15758)
CANN: fix acl_rstd allocation size in ggml_cann_rms_norm (ggml-org#15760)
vulkan: fix mmv subgroup16 selection (ggml-org#15775)
vulkan: don't use std::string in load_shaders, to improve compile time (ggml-org#15724)
vulkan : update ggml_vk_instance_validation_ext_available (ggml-org#15666)
ggml vulkan: add hardsigmoid and hardswish operations (ggml-org#15762)
CUDA: Optimize `rms_norm_f32` kernel and its fused variants, giving 1-6% perf E2E (ggml-org#15715)
model-conversion : fix pyright errors (ggml-org#15770)
sampling : optimize dist sampler (ggml-org#15704)
llama : fix incorrect model type for Gemma 270M (ggml-org#15764)
model-conversion : remove hardcoded /bin/bash shebangs [no ci] (ggml-org#15765)
CANN: Add RoPE contiguous check for 310I DUP device (ggml-org#15735)
ggml-cpu : optimize RVV kernels (ggml-org#15720)
...
gabe-l-hart added a commit to gabe-l-hart/llama.cpp that referenced this pull request Sep 5, 2025
@gabe-l-hart
Merge remote-tracking branch 'origin/master' into gabe-l-hart/thinkin…
fe6b72b 
…g-model-disabled-agent-prefill

* origin/master: (84 commits)
CUDA: fastdiv, launch bounds for mmvq + q8_1 quant (ggml-org#15802)
tests : add --list-ops and --show-coverage options (ggml-org#15745)
gguf: gguf_writer refactor (ggml-org#15691)
kv-cache : fix SWA checks + disable cacheless iSWA (ggml-org#15811)
model-conversion : add --embeddings flag to modelcard.template [no ci] (ggml-org#15801)
chat : fixed crash when Hermes 2 <tool_call> had a newline before it (ggml-org#15639)
chat : nemotron thinking & toolcalling support (ggml-org#15676)
scripts : add Jinja tester PySide6 simple app (ggml-org#15756)
llama : add support for EmbeddingGemma 300m (ggml-org#15798)
metal : Add template specialization for mul_mm_id w/ ne20 == 10 (ggml-org#15799)
llama : set n_outputs to 1 to avoid 0 outputs mean-pooling (ggml-org#15791)
CANN: Refactor ND to NZ workspace to be per-device (ggml-org#15763)
server: add exceed_context_size_error type (ggml-org#15780)
Document the new max GPU layers default in help (ggml-org#15771)
ggml: add ops for WAN video model (cuda && cpu) (ggml-org#15669)
CANN: Fix precision issue on 310I DUO multi-devices (ggml-org#15784)
opencl: add hs=40 to FA (ggml-org#15758)
CANN: fix acl_rstd allocation size in ggml_cann_rms_norm (ggml-org#15760)
vulkan: fix mmv subgroup16 selection (ggml-org#15775)
vulkan: don't use std::string in load_shaders, to improve compile time (ggml-org#15724)
...
walidbr pushed a commit to walidbr/llama.cpp that referenced this pull request Sep 7, 2025
@jeffbolznv Walid Ben Romdhane
vulkan: don't use std::string in load_shaders, to improve compile time (
30238ee 
ggml-org#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
@jeffbolznv shaobo.xie
vulkan: don't use std::string in load_shaders, to improve compile tim…
97ec2e4 
…e (#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@jeffbolznv
vulkan: don't use std::string in load_shaders, to improve compile time (
46af822 
ggml-org#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@jeffbolznv
vulkan: don't use std::string in load_shaders, to improve compile time (
15910ce 
ggml-org#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
@jeffbolznv
vulkan: don't use std::string in load_shaders, to improve compile time (
11d8a6e 
ggml-org#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
MrLordCat referenced this pull request in MrLordCat/llama.cpp-rdna-lab Jul 16, 2026
@jeffbolznv
vulkan: don't use std::string in load_shaders, to improve compile tim…
ce456c6 
…e (#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
AMD-melliott pushed a commit to AMD-melliott/llama.cpp that referenced this pull request Aug 4, 2026
@jeffbolznv @AMD-melliott
vulkan: don't use std::string in load_shaders, to improve compile time (
bcec8ac 
ggml-org#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
zommiommy pushed a commit to zommiommy/llama.cpp that referenced this pull request Aug 18, 2026
@jeffbolznv @zommiommy
vulkan: don't use std::string in load_shaders, to improve compile time (
978a506 
ggml-org#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
frostyautumnleaf pushed a commit to frostyautumnleaf/llama.cpp that referenced this pull request Oct 5, 2026
@jeffbolznv
vulkan: don't use std::string in load_shaders, to improve compile time (
3c1d7d4 
ggml-org#15724)

* vulkan: don't use std::string in load_shaders, to improve compile time

* keep the string version for those calls that use it
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@0cc4m 0cc4m 0cc4m approved these changes
+1 more reviewer
@LostRuins LostRuins LostRuins approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ggml changes relating to the ggml tensor library for machine learning Vulkan Issues specific to the Vulkan backend

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jeffbolznv @0cc4m @LostRuins