vulkan: tune KHR cooperative matrix support for Adreno GPUs - #29328
Conversation
Sync with mainline
Sync with main
Sync with main
Sync with main
Sync with main
Sync with main
github-actions
Bot
added
Vulkan
| device->mul_mat_l[i] = false; | ||
| device->mul_mat_m[i] = true; | ||
| device->mul_mat_s[i] = true; | ||
| device->mul_mat_s[i] = false; |
There was a problem hiding this comment.
Is it correct to disable these for non-coopmat qcom devices?
There was a problem hiding this comment.
for non-coopmat Adreno there's no hardware minimum-tile constraint forcing s off, so disabling it there is unjustified. It should be !device->coopmat_support
|
Is the PR title accurate? From the code it looks like it deactivates coopmat for non-Adreno Qualcomm devices instead, and it adds tuning for Adreno. |
|
@0cc4m Coopmat should be enabled only for Adreno GPU's having matrix cores in hardware. For rest qualcomm devices it should be disabled. |
|
I don't disagree, but that's not what the title says. I assume currently coopmat is running on all Qualcomm devices, assuming they report coopmat support, but it's not running well? |
Raman-Raje
changed the title
|
Updated the title. Manage is the correct word. Do you find any better alternative..? |
0cc4m
changed the title
| return true; | ||
| } | ||
| case VK_VENDOR_ID_QUALCOMM: | ||
| return false; |
There was a problem hiding this comment.
This optimization needs a deeper analysis. For now return false to avoid calling the wrong shaders and breakdowns
| } | ||
| } | ||
| } else if(props.vendorID == VK_VENDOR_ID_QUALCOMM){ | ||
| VK_LOG_DEBUG("ggml_vulkan: [DEBUG] Qualcomm Adreno GPU device=\""<< props.deviceName << "\")"); |
There was a problem hiding this comment.
I don't think this is necessary. If you want a debug output for architectures it should be generic, not Qualcomm-only.
There was a problem hiding this comment.
You are right. Forgot to remove this
3 participants
Overview
This PR enables
VK_KHR_cooperative_matrix(coopmat) support in the Vulkan backend for Qualcomm Adreno GPUs that have hardware matrix cores. The initial target is Snapdragon 8 Elite Gen 6.Coopmat is only enabled when the device exposes both
VK_KHR_cooperative_matrixandVK_QCOM_cooperative_matrix_conversion. Adreno devices without these extensions keep their current behaviour and do not use coopmat.Additional information
vk_device_architecture::QUALCOMM_ADRENOvalue.QUALCOMM_ADRENO. Other Adreno GPUs are excluded.m_warptile,m_warptile_mmq) with 64-wide workgroups when coopmat is available.Requirements