img2img 'weakened' by diffusion-fa

[wbruna]

Since ab835f7 , img2img generations need higher strength values to apply changes to the input images.

This is an SDXL test on Vulkan, with the prompt ./sd -m ./cyberrealisticPony_semiRealV35a.safetensors -s 1 -p 'silver car' -i ./base.png , leaving the strength at the default (0.75) for the first gens:

base image	1896b28	1896b28 --diffusion-fa	ab835f7	ab835f7 --diffusion-fa	ab835f7 --diffusion-fa --strength 0.85

I first noticed this issue on current master 5900ef6 . All binaries built with current ggml 7dee1d6 .

@Green-Sky

[Green-Sky]

Hmm, interesting. Maybe we should really just remove the padding changes again.
#736

[wbruna]

The same test on 5900ef6 with:

diff --git a/ggml_extend.hpp b/ggml_extend.hpp
index 22dd88c..f408b79 100644
--- a/ggml_extend.hpp
+++ b/ggml_extend.hpp
@@ -880,7 +880,7 @@ __STATIC_INLINE__ struct ggml_tensor* ggml_nn_attention_ext(struct ggml_context*
 #else
     if (can_use_flash_attn && L_k % 256 != 0) {
         // TODO(Green-Sky): might be worth just padding by default
-        if (L_k == 77 || L_k == 4208 || L_k == 3952) {
+        if (L_k == 4208 || L_k == 3952) {
             kv_pad = GGML_PAD(L_k, 256) - L_k;
         } else {
             can_use_flash_attn = false;

[Green-Sky]

The same test on 5900ef6 with:

diff --git a/ggml_extend.hpp b/ggml_extend.hpp
index 22dd88c..f408b79 100644
--- a/ggml_extend.hpp
+++ b/ggml_extend.hpp
@@ -880,7 +880,7 @@ __STATIC_INLINE__ struct ggml_tensor* ggml_nn_attention_ext(struct ggml_context*
 #else
     if (can_use_flash_attn && L_k % 256 != 0) {
         // TODO(Green-Sky): might be worth just padding by default
-        if (L_k == 77 || L_k == 4208 || L_k == 3952) {
+        if (L_k == 4208 || L_k == 3952) {
             kv_pad = GGML_PAD(L_k, 256) - L_k;
         } else {
             can_use_flash_attn = false;

Yeah that just disables the most common padding case.
The problem with PAD is that it add zeros, and if a zero is not the neutral value, calculations will go off.

[wbruna]

Ah, I see. So that's essentially equivalent to switching the #if 0 right above.

[wbruna]

For completeness sake, with the change I've mentioned on #774 (4c8b774), I get:

[Green-Sky]

@wbruna is this still an issue?

[wbruna]

Oh, thanks for reminding me of this one. It was fixed by #778 .