Skip to content

[Bug] Vulkan GGML Backend Performance Regression on commit 9b0fceb #1647

Closed
Closed
[Bug] Vulkan GGML Backend Performance Regression on commit 9b0fceb#1647
Labels
bugSomething isn't working

Description

@sz1kormar
sz1kormar
opened on Jun 13, 2026

Git commit

A major performance regression occurs in the Vulkan GGML backend when moving from
commit 19bdfe2 → commit 9b0fceb.
On an AMD RX 580 running Windows 10 22H2 through sd.cpp‑webui, iteration time increases from ~9.5s/it to 22–23s/it, more than doubling generation time on identical workloads.

Operating System & Version

Windows 10 22H2

GGML backends

Vulkan

Command-line arguments used

sd-cli.exe -M img_gen -p "anime, depth of field, 1girl, closed eyes, white shirt, open shirt, cross necklace, nurse cap, pov, 1boy, holding another's arm, lora:anima-turbo-lora-v0.1:1" --sampling-method er_sde --steps 8 -W 1152 -H 896 -b 1 --cfg-scale 1 -s 20514 --clip-skip -1 --embd-dir D:\Programs\sd.cpp-webui\models/embeddings/ --lora-model-dir D:\Programs\sd.cpp-webui\models/loras/ -t 0 --rng std_default --sampler-rng std_default --lora-apply-mode auto -o D:\Programs\sd.cpp-webui\outputs\txt2img\158.png --diffusion-model D:\Programs\sd.cpp-webui\models\unet\anima_baseV10.safetensors --vae D:\Programs\sd.cpp-webui\models\vae\qwenimagevae_v7.safetensors --llm D:\Programs\sd.cpp-webui\models\text_encoders\qwen_3_06b_base.safetensors --type f16 --scheduler sgm_uniform --vae-tile-overlap 0.5 --vae-tile-size 64x64 --cache-mode spectrum --vae-tiling --diffusion-conv-direct --vae-conv-direct --mmap --color

Steps to reproduce

  1. Launch sd.cpp‑webui using build 19bdfe2.

  2. Run any image generation task using the Vulkan backend.

  3. Update to build 9b0fceb.

  4. Run the exact same command and settings.

  5. Compare iteration times.

What you expected to happen

Performance should match or exceed previous Vulkan backend throughput.

What actually happened

BEFORE regression:
sd-master‑19bdfe2‑bin‑win‑vulkan‑x64

Performance: ~9.5 seconds per iteration

AFTER regression:
sd-master‑9b0fceb‑bin‑win‑vulkan‑x64

Performance: 22–23 seconds per iteration

This represents a 2.3× slowdown.

Logs / error messages / stack trace

No response

Additional context / environment details

While testing, it is possible to patch or swap the .exe and .dll files of sd.cpp‑webui during runtime.
As long as the change is made after a generation finishes or is cancelled, the process does not need to be restarted.

Because the models remain fully cached in VRAM and RAM, this allows:

Rapid A/B testing between builds

Switching between 19bdfe2 and 9b0fceb without reloading models

Eliminating model‑loading time as a variable

Ensuring the regression is isolated to runtime performance, not initialization

This confirms the slowdown is not caused by model reloads, cache state, or frontend overhead — it is directly tied to the Vulkan backend behavior between the two commits.

VRAM usage is stable and even slightly lower on the new commit; the regression is purely in iteration speed, not memory usage.

Activity

  1. sz1kormar
    added
    bugSomething isn't working
    on Jun 13, 2026

  2. ThePixelDiffusionPirate commented on Jun 14, 2026

    @ThePixelDiffusionPirate
    ThePixelDiffusionPirate
    on Jun 14, 2026

    Unfortunately, there’s often something wrong with the Windows Vulkan releases. I’m currently using a two-week-old release with my 5700 XT.

  3. sz1kormar commented on Jun 15, 2026

    @sz1kormar
    sz1kormar
    on Jun 15, 2026
    Author

    Unfortunately, there’s often something wrong with the Windows Vulkan releases. I’m currently using a two-week-old release with my 5700 XT.

    I stick to older builds too and mainly use Anima Base 1.0 — it’s lighter than Z Image and Qwen while still outperforming classic Stable Diffusion for what I need.

  4. JohnLoveJoy commented on Jun 17, 2026

    @JohnLoveJoy
    JohnLoveJoy
    on Jun 17, 2026

    You can make it run a bit faster by using the TAESD, It works well with Anima. It's also worth trying to run it on KoboldCPP, as it uses a more recent version of GGML.

    https://github.com/LostRuins/koboldcpp/releases

    And don't activate FA, I tested it and it caused this:

    Image

  5. wbruna commented on Jun 17, 2026

    @wbruna
    wbruna
    on Jun 17, 2026
    Contributor

    @sz1kormar , there are a few other revisions between those; could you check if e.g. 3a54597 (master-690-3a54597) works as expected?

    It's also worth trying to run it on KoboldCPP, as it uses a more recent version of GGML.

    Not that different right now; it was updated three days ago.

    And don't activate FA, I tested it and it caused this:

    That seems to be specific to the Vukan backend:

    Vulkan ROCm
    Image Image

    I don't think we have an open issue for it?

  6. wbruna commented on Jun 17, 2026

    @wbruna
    wbruna
    on Jun 17, 2026
    Contributor

    And don't activate FA, I tested it and it caused this:

    @JohnLoveJoy , could you give #1678 a try?

  7. sz1kormar commented on Jun 19, 2026

    @sz1kormar
    sz1kormar
    on Jun 19, 2026
    Author

    @wbruna i checked that commit version 3a54597 and it works with the same speed for me as 19bdfe2.

    Looks like I found a newer version that works fine here without the slowdown.

  8. fszontagh commented on Jun 21, 2026

    @fszontagh
    fszontagh
    on Jun 21, 2026
    Contributor

    @sz1kormar could you share the per-step iteration times rather than just the average? Something like:

    [step 1/8] X.Xs/it
[step 2/8] X.Xs/it
...
[step 8/8] X.Xs/it

    on the regressed commit (9b0fceb). Curious whether the slowdown is evenly distributed across steps or concentrated in early ones.

  9. sz1kormar commented on Jun 21, 2026

    @sz1kormar
    sz1kormar
    on Jun 21, 2026
    Author

    @fszontagh here the results for per-step iteration times:

    before:
    [step 1/8] 9.58s/it
    [step 2/8] 9.52s/it
    [step 3/8] 9.50s/it
    [step 4/8] 9.49s/it
    [step 5/8] 9.49s/it
    [step 6/8] 9.48s/it
    [step 7/8] 9.48s/it
    [step 8/8] 9.48s/it

    after:
    [step 1/8] 29.32s/it
    [step 2/8] 23.71s/it
    [step 3/8] 23.81s/it
    [step 4/8] 23.45s/it
    [step 5/8] 23.69s/it
    [step 6/8] 23.74s/it
    [step 7/8] 23.67s/it
    [step 8/8] 23.49s/it

  10. jeffbolznv commented on Jun 22, 2026

    @jeffbolznv
    jeffbolznv
    on Jun 22, 2026

    Please try running before/after with these env vars set: GGML_VK_PERF_LOGGER=1 GGML_VK_PERF_LOGGER_FREQUENCY=99999. This may help identify which ops have gotten slower.

  11. sz1kormar commented on Jun 22, 2026

    @sz1kormar
    sz1kormar
    on Jun 22, 2026
    Author

    BEFORE PERFORMANCE BUG

    Vulkan Timings:
    ADD: 2598 x 546.238 us = 1.41913e+06 us
    CONCAT: 12 x 145.56 us = 1746.72 us
    CONT: 5332 x 871.759 us = 4.64822e+06 us
    CONV_2D M=Cout=1152, K=CinKWKH=384, N=NOWOH=4096: 6 x 1752.32 us = 10513.9 us (2065.35 GFLOPS/s)
    CONV_2D M=Cout=192, K=CinKWKH=3456, N=NOWOH=16384: 6 x 11276.5 us = 67659 us (1927.91 GFLOPS/s)
    CONV_2D M=Cout=192, K=CinKWKH=3456, N=NOWOH=65536: 6 x 42388.7 us = 254332 us (2051.5 GFLOPS/s)
    CONV_2D M=Cout=384, K=CinKWKH=384, N=NOWOH=4096: 6 x 631.173 us = 3787.04 us (1911.34 GFLOPS/s)
    CONV_2D M=Cout=96, K=CinKWKH=1728, N=NOWOH=262144: 6 x 41521.6 us = 249130 us (2094.04 GFLOPS/s)
    GELU: 204 x 1245.97 us = 254179 us
    GET_ROWS: 7 x 71.68 us = 501.76 us
    IM2COL_3D: 192 x 35577.7 us = 6.83092e+06 us
    MUL: 2188 x 551.512 us = 1.20671e+06 us
    MUL_MAT bf16 m=2048 n=4032 k=68: 6 x 1241.47 us = 7448.8 us (897.944 GFLOPS/s)
    MUL_MAT bf16 m=64 n=4032 k=2048: 6 x 2711.2 us = 16267.2 us (389.756 GFLOPS/s)
    MUL_MAT f16 m=1024 n=52 k=1024: 128 x 470.736 us = 60254.2 us (231.549 GFLOPS/s)
    MUL_MAT f16 m=1024 n=52 k=2048: 28 x 909.725 us = 25472.3 us (239.688 GFLOPS/s)
    MUL_MAT f16 m=1024 n=52 k=3072: 28 x 1287.51 us = 36050.2 us (254.058 GFLOPS/s)
    MUL_MAT f16 m=1024 n=55 k=1024: 222 x 370.35 us = 82217.8 us (311.292 GFLOPS/s)
    MUL_MAT f16 m=1024 n=55 k=4096: 36 x 1173.43 us = 42243.5 us (393.135 GFLOPS/s)
    MUL_MAT f16 m=16384 n=384 k=10368: 30 x 57763.5 us = 1.73291e+06 us (2258.4 GFLOPS/s)
    MUL_MAT f16 m=16384 n=384 k=192: 6 x 1270.83 us = 7624.96 us (1896.11 GFLOPS/s)
    MUL_MAT f16 m=16384 n=384 k=5184: 6 x 28627.8 us = 171767 us (2278.33 GFLOPS/s)
    MUL_MAT f16 m=2048 n=4032 k=2048: 1008 x 14174.3 us = 1.42877e+07 us (2385.63 GFLOPS/s)
    MUL_MAT f16 m=2048 n=4032 k=8192: 168 x 59583.5 us = 1.001e+07 us (2270.48 GFLOPS/s)
    MUL_MAT f16 m=2048 n=512 k=1024: 336 x 1219.38 us = 409712 us (1760.26 GFLOPS/s)
    MUL_MAT f16 m=2048 n=52 k=1024: 28 x 854.588 us = 23928.5 us (255.09 GFLOPS/s)
    MUL_MAT f16 m=262144 n=96 k=2592: 36 x 99539.3 us = 3.58341e+06 us (1310.38 GFLOPS/s)
    MUL_MAT f16 m=3072 n=52 k=1024: 56 x 1010.23 us = 56573 us (323.684 GFLOPS/s)
    MUL_MAT f16 m=4096 n=16 k=16: 6 x 131.946 us = 791.68 us (15.3973 GFLOPS/s)
    MUL_MAT f16 m=4096 n=384 k=10368: 60 x 16037 us = 962220 us (2033.63 GFLOPS/s)
    MUL_MAT f16 m=4096 n=384 k=432: 6 x 1111.6 us = 6669.6 us (1221.11 GFLOPS/s)
    MUL_MAT f16 m=4096 n=55 k=1024: 36 x 570.653 us = 20543.5 us (808.106 GFLOPS/s)
    MUL_MAT f16 m=65536 n=192 k=5184: 36 x 56672.6 us = 2.04021e+06 us (2301.77 GFLOPS/s)
    MUL_MAT f16 m=8192 n=4032 k=2048: 168 x 54977.3 us = 9.23618e+06 us (2460.26 GFLOPS/s)
    MUL_MAT f32 m=128 n=4032 k=4032 batch=16: 168 x 25212.3 us = 4.23566e+06 us (2640.8 GFLOPS/s)
    MUL_MAT f32 m=128 n=4032 k=512 batch=16: 168 x 3629 us = 609671 us (2327.77 GFLOPS/s)
    MUL_MAT f32 m=128 n=52 k=52 batch=16: 28 x 178.817 us = 5006.88 us (61.3425 GFLOPS/s)
    MUL_MAT f32 m=384 n=4096 k=4096: 6 x 7245.41 us = 43472.5 us (1778.14 GFLOPS/s)
    MUL_MAT f32 m=4032 n=4032 k=128 batch=16: 168 x 29227.3 us = 4.91019e+06 us (2269.41 GFLOPS/s)
    MUL_MAT f32 m=4096 n=4096 k=384: 6 x 5581.28 us = 33487.7 us (2305.59 GFLOPS/s)
    MUL_MAT f32 m=512 n=4032 k=128 batch=16: 168 x 3763.48 us = 632265 us (2238 GFLOPS/s)
    MUL_MAT f32 m=52 n=52 k=128 batch=16: 28 x 194.8 us = 5454.4 us (56.6341 GFLOPS/s)
    MUL_MAT f32 m=52 n=55 k=64 batch=16: 36 x 133.213 us = 4795.68 us (43.6257 GFLOPS/s)
    MUL_MAT f32 m=55 n=55 k=64 batch=16: 36 x 133.24 us = 4796.64 us (46.1333 GFLOPS/s)
    MUL_MAT f32 m=64 n=55 k=52 batch=16: 36 x 147.395 us = 5306.24 us (39.3564 GFLOPS/s)
    MUL_MAT f32 m=64 n=55 k=55 batch=16: 36 x 151.515 us = 5454.56 us (40.5165 GFLOPS/s)
    MUL_MAT_ADD MUL_MAT_VEC f16 m=6144 n=1 k=256: 498 x 123.33 us = 61418.6 us (25.4567 GFLOPS/s)
    MUL_MAT_VEC bf16 m=2048 n=1 k=2048: 6 x 112.373 us = 674.24 us (74.6312 GFLOPS/s)
    MUL_MAT_VEC bf16 m=256 n=1 k=2048: 6 x 124.08 us = 744.48 us (8.44874 GFLOPS/s)
    MUL_MAT_VEC bf16 m=4096 n=1 k=256: 6 x 112.586 us = 675.52 us (18.5906 GFLOPS/s)
    MUL_MAT_VEC bf16 m=6144 n=1 k=2048: 6 x 259.893 us = 1559.36 us (96.8077 GFLOPS/s)
    MUL_MAT_VEC f16 m=256 n=1 k=2048: 504 x 124.286 us = 62640.6 us (8.43467 GFLOPS/s)
    MUL_MAT_VEC f16 m=262144 n=3 k=2592: 6 x 9681.79 us = 58090.7 us (421.005 GFLOPS/s)
    MUL_MAT_VEC f16 m=6144 n=1 k=256: 6 x 121.653 us = 729.92 us (25.8076 GFLOPS/s)
    NORM: 510 x 498.682 us = 254328 us
    PAD: 180 x 2196.99 us = 395458 us
    REPEAT: 978 x 770.41 us = 753461 us
    RMS_NORM(192,128,128,1): 6 x 417.866 us = 2507.2 us
    RMS_NORM(192,256,256,1): 36 x 1361.61 us = 49017.9 us
    RMS_NORM(384,128,128,1): 30 x 460.709 us = 13821.3 us
    RMS_NORM(384,64,64,1): 66 x 197.163 us = 13012.8 us
    RMS_NORM(96,512,512,1): 42 x 5111.39 us = 214679 us
    RMS_NORM_MUL RMS_NORM(1024,52,1,1): 57 x 109.695 us = 6252.64 us
    RMS_NORM_MUL RMS_NORM(1024,55,1,1): 114 x 115.131 us = 13125 us
    RMS_NORM_MUL RMS_NORM(128,16,4032,1): 504 x 1678.19 us = 845808 us
    RMS_NORM_MUL RMS_NORM(128,16,512,1): 168 x 291.015 us = 48890.6 us
    RMS_NORM_MUL RMS_NORM(128,16,52,1): 28 x 143.828 us = 4027.2 us
    RMS_NORM_MUL RMS_NORM(128,8,52,1): 28 x 124.177 us = 3476.96 us
    RMS_NORM_MUL RMS_NORM(2048,1,1,1): 6 x 120.426 us = 722.56 us
    RMS_NORM_MUL RMS_NORM(64,16,52,1): 36 x 128.16 us = 4613.76 us
    RMS_NORM_MUL RMS_NORM(64,16,55,1): 108 x 126.534 us = 13665.8 us
    ROPE: 56 x 144.994 us = 8119.68 us
    SCALE: 460 x 4587.53 us = 2.11027e+06 us
    SILU: 718 x 184.29 us = 132321 us
    SOFT_MAX: 442 x 7673.54 us = 3.3917e+06 us
    TIMESTEP_EMBEDDING: 6 x 111.306 us = 667.84 us
    UPSCALE: 18 x 2740.89 us = 49336 us
    Total time: 7.67784e+07 us.

  12. sz1kormar commented on Jun 22, 2026

    @sz1kormar
    sz1kormar
    on Jun 22, 2026
    Author

    AFTER PERFORMANCE BUG:

    Vulkan Timings:
    ADD: 2598 x 4230.26 us = 1.09902e+07 us
    CONCAT: 12 x 352.066 us = 4224.8 us
    CONT: 5332 x 2081.53 us = 1.10987e+07 us
    CONV_2D M=Cout=1152, K=CinKWKH=384, N=NOWOH=4096: 6 x 1775.92 us = 10655.5 us (2037.91 GFLOPS/s)
    CONV_2D M=Cout=192, K=CinKWKH=3456, N=NOWOH=16384: 6 x 11472.6 us = 68835.5 us (1894.96 GFLOPS/s)
    CONV_2D M=Cout=192, K=CinKWKH=3456, N=NOWOH=65536: 6 x 42866.6 us = 257200 us (2028.63 GFLOPS/s)
    CONV_2D M=Cout=384, K=CinKWKH=384, N=NOWOH=4096: 6 x 630.373 us = 3782.24 us (1913.77 GFLOPS/s)
    CONV_2D M=Cout=96, K=CinKWKH=1728, N=NOWOH=262144: 6 x 41952.1 us = 251712 us (2072.55 GFLOPS/s)
    GELU: 204 x 15484.4 us = 3.15882e+06 us
    GET_ROWS: 7 x 165.142 us = 1156 us
    IM2COL_3D: 192 x 35963.3 us = 6.90496e+06 us
    MUL: 2188 x 3817.04 us = 8.35168e+06 us
    MUL_MAT bf16 m=2048 n=4032 k=68: 6 x 5849.2 us = 35095.2 us (190.585 GFLOPS/s)
    MUL_MAT bf16 m=64 n=4032 k=2048: 6 x 7152.75 us = 42916.5 us (147.734 GFLOPS/s)
    MUL_MAT f16 m=1024 n=52 k=1024: 128 x 499.745 us = 63967.4 us (218.109 GFLOPS/s)
    MUL_MAT f16 m=1024 n=52 k=2048: 28 x 1019.74 us = 28552.8 us (213.829 GFLOPS/s)
    MUL_MAT f16 m=1024 n=52 k=3072: 28 x 1428.45 us = 39996.6 us (228.991 GFLOPS/s)
    MUL_MAT f16 m=1024 n=55 k=1024: 222 x 405.246 us = 89964.8 us (284.486 GFLOPS/s)
    MUL_MAT f16 m=1024 n=55 k=4096: 36 x 1355.4 us = 48794.6 us (340.354 GFLOPS/s)
    MUL_MAT f16 m=16384 n=384 k=10368: 30 x 58203.3 us = 1.7461e+06 us (2241.34 GFLOPS/s)
    MUL_MAT f16 m=16384 n=384 k=192: 6 x 1322.48 us = 7934.88 us (1822.05 GFLOPS/s)
    MUL_MAT f16 m=16384 n=384 k=5184: 6 x 28966.4 us = 173799 us (2251.69 GFLOPS/s)
    MUL_MAT f16 m=2048 n=4032 k=2048: 1008 x 16718.9 us = 1.68526e+07 us (2022.54 GFLOPS/s)
    MUL_MAT f16 m=2048 n=4032 k=8192: 168 x 212787 us = 3.57482e+07 us (635.769 GFLOPS/s)
    MUL_MAT f16 m=2048 n=512 k=1024: 336 x 1392.68 us = 467941 us (1541.22 GFLOPS/s)
    MUL_MAT f16 m=2048 n=52 k=1024: 28 x 926.497 us = 25941.9 us (235.292 GFLOPS/s)
    MUL_MAT f16 m=262144 n=96 k=2592: 36 x 100803 us = 3.62892e+06 us (1293.95 GFLOPS/s)
    MUL_MAT f16 m=3072 n=52 k=1024: 56 x 1006.38 us = 56357.1 us (324.924 GFLOPS/s)
    MUL_MAT f16 m=4096 n=16 k=16: 6 x 184.853 us = 1109.12 us (10.9904 GFLOPS/s)
    MUL_MAT f16 m=4096 n=384 k=10368: 60 x 16071.4 us = 964282 us (2029.28 GFLOPS/s)
    MUL_MAT f16 m=4096 n=384 k=432: 6 x 1086.8 us = 6520.8 us (1248.97 GFLOPS/s)
    MUL_MAT f16 m=4096 n=55 k=1024: 36 x 563.2 us = 20275.2 us (818.8 GFLOPS/s)
    MUL_MAT f16 m=65536 n=192 k=5184: 36 x 57160.9 us = 2.05779e+06 us (2282.1 GFLOPS/s)
    MUL_MAT f16 m=8192 n=4032 k=2048: 168 x 57480.7 us = 9.65675e+06 us (2353.11 GFLOPS/s)
    MUL_MAT f32 m=128 n=4032 k=4032 batch=16: 168 x 83236.6 us = 1.39838e+07 us (799.894 GFLOPS/s)
    MUL_MAT f32 m=128 n=4032 k=512 batch=16: 168 x 5336.52 us = 896535 us (1582.95 GFLOPS/s)
    MUL_MAT f32 m=128 n=52 k=52 batch=16: 28 x 187.954 us = 5262.72 us (58.3604 GFLOPS/s)
    MUL_MAT f32 m=384 n=4096 k=4096: 6 x 7375.79 us = 44254.7 us (1746.71 GFLOPS/s)
    MUL_MAT f32 m=4032 n=4032 k=128 batch=16: 168 x 108079 us = 1.81573e+07 us (613.704 GFLOPS/s)
    MUL_MAT f32 m=4096 n=4096 k=384: 6 x 5637.68 us = 33826.1 us (2282.52 GFLOPS/s)
    MUL_MAT f32 m=512 n=4032 k=128 batch=16: 168 x 4454.39 us = 748338 us (1890.87 GFLOPS/s)
    MUL_MAT f32 m=52 n=52 k=128 batch=16: 28 x 216.022 us = 6048.64 us (51.0702 GFLOPS/s)
    MUL_MAT f32 m=52 n=55 k=64 batch=16: 36 x 185.395 us = 6674.24 us (31.3466 GFLOPS/s)
    MUL_MAT f32 m=55 n=55 k=64 batch=16: 36 x 185.946 us = 6694.08 us (33.0568 GFLOPS/s)
    MUL_MAT f32 m=64 n=55 k=52 batch=16: 36 x 193.56 us = 6968.16 us (29.9698 GFLOPS/s)
    MUL_MAT f32 m=64 n=55 k=55 batch=16: 36 x 205.453 us = 7396.32 us (29.8797 GFLOPS/s)
    MUL_MAT_ADD MUL_MAT_VEC f16 m=6144 n=1 k=256: 498 x 145.356 us = 72387.4 us (21.5993 GFLOPS/s)
    MUL_MAT_VEC bf16 m=2048 n=1 k=2048: 6 x 287.28 us = 1723.68 us (29.193 GFLOPS/s)
    MUL_MAT_VEC bf16 m=256 n=1 k=2048: 6 x 156.586 us = 939.52 us (6.69482 GFLOPS/s)
    MUL_MAT_VEC bf16 m=4096 n=1 k=256: 6 x 238.773 us = 1432.64 us (8.76587 GFLOPS/s)
    MUL_MAT_VEC bf16 m=6144 n=1 k=2048: 6 x 719.2 us = 4315.2 us (34.9829 GFLOPS/s)
    MUL_MAT_VEC f16 m=256 n=1 k=2048: 504 x 138.183 us = 69644.5 us (7.58643 GFLOPS/s)
    MUL_MAT_VEC f16 m=262144 n=3 k=2592: 6 x 9784.88 us = 58709.3 us (416.569 GFLOPS/s)
    MUL_MAT_VEC f16 m=6144 n=1 k=256: 6 x 176.32 us = 1057.92 us (17.8062 GFLOPS/s)
    NORM: 510 x 5126.78 us = 2.61466e+06 us
    PAD: 180 x 2227.12 us = 400882 us
    REPEAT: 978 x 2043.83 us = 1.99886e+06 us
    RMS_NORM(192,128,128,1): 6 x 423.653 us = 2541.92 us
    RMS_NORM(192,256,256,1): 36 x 1379.56 us = 49664 us
    RMS_NORM(384,128,128,1): 30 x 486.896 us = 14606.9 us
    RMS_NORM(384,64,64,1): 66 x 204.375 us = 13488.8 us
    RMS_NORM(96,512,512,1): 42 x 5165.47 us = 216950 us
    RMS_NORM_MUL RMS_NORM(1024,52,1,1): 57 x 123.326 us = 7029.6 us
    RMS_NORM_MUL RMS_NORM(1024,55,1,1): 114 x 129.493 us = 14762.2 us
    RMS_NORM_MUL RMS_NORM(128,16,4032,1): 504 x 4860.24 us = 2.44956e+06 us
    RMS_NORM_MUL RMS_NORM(128,16,512,1): 168 x 706.43 us = 118680 us
    RMS_NORM_MUL RMS_NORM(128,16,52,1): 28 x 145.782 us = 4081.92 us
    RMS_NORM_MUL RMS_NORM(128,8,52,1): 28 x 159.382 us = 4462.72 us
    RMS_NORM_MUL RMS_NORM(2048,1,1,1): 6 x 130.186 us = 781.12 us
    RMS_NORM_MUL RMS_NORM(64,16,52,1): 36 x 132.355 us = 4764.8 us
    RMS_NORM_MUL RMS_NORM(64,16,55,1): 108 x 133.694 us = 14439 us
    ROPE: 56 x 167.522 us = 9381.28 us
    SCALE: 460 x 12702.2 us = 5.84302e+06 us
    SILU: 718 x 223.266 us = 160305 us
    SOFT_MAX: 442 x 7783.88 us = 3.44048e+06 us
    TIMESTEP_EMBEDDING: 6 x 119.52 us = 717.12 us
    UPSCALE: 18 x 2740.44 us = 49328 us
    Total time: 1.64412e+08 us.

  13. jeffbolznv commented on Jun 22, 2026

    @jeffbolznv
    jeffbolznv
    on Jun 22, 2026

    The higher times spent in add/cont remind me of some recent llama.cpp issues that went away with RADV_PERFTEST=nogttspill. But if this is on windows, I guess that experiment won't work, but it could still be something related to spilling to system memory. I guess the next step might be to bisect the ggml changes.

  14. sz1kormar
    closed this as completedon Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions