[webgpu] Optimize Conv by im2col-matmul - #26603
Conversation
|
Lunar Lake Operation
sd-turbo
|
Jiajia Qin (qjia7)
left a comment
Jiajia Qin (qjia7)
left a comment
There was a problem hiding this comment.
Jianhui Dai (@daijh) Are you ok to submit to commit to remove the device limitation so that the CI can test your new path? If the CI is happy, you can revert it again? Or add a test config which can also go to this path so the the CI can test it? Guenther Schmuelling (@guschmue) Yulong Wang (@fs-eire) How's opinions on this?
|
Certainly. |
601e7c1 to
c9b3fdd
Compare
|
Fix |
c9b3fdd to
24cb42e
Compare
|
Suppresses the unused variable warning, please help to re-trigger. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
24cb42e to
8eb8ecc
Compare
|
Thanks. Revert previous change to prevent performance regression on untested devices. |
|
/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
|
The failure in the CI is not relevant to this PR's changes. |
…pacted on un-tested devices
…ively impacted on un-tested devices" This reverts commit 8eb8ecc.
aa7fa8e to
528bf05
Compare
|
/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
8f8069d
into
microsoft:main
### Description This PR optimizes the `Conv` operation by implementing two new compute shaders: `oihw_to_ohwi` and `im2col-matmul`. `oihw_to_ohwi`: Improves performance over the default Transpose shader by utilizing workgroup memory to ensure continuous memory read/write patterns. `im2col-matmul`: - Employs a workgroup size of 64. - Dynamically selects tile sizes (32x64 or 16x64) based on the source/weight shape. - Each invocation handles a dedicated weight element. - Uses subgroupShuffle to efficiently access the source tile, leveraging k_vec4 vectorization for better memory throughput. Testing on Lunar Lake demonstrated **up to an 87%** performance improvement in Conv_2D operations. ### Motivation and Context See above.
### Description This PR optimizes the `Conv` operation by implementing two new compute shaders: `oihw_to_ohwi` and `im2col-matmul`. `oihw_to_ohwi`: Improves performance over the default Transpose shader by utilizing workgroup memory to ensure continuous memory read/write patterns. `im2col-matmul`: - Employs a workgroup size of 64. - Dynamically selects tile sizes (32x64 or 16x64) based on the source/weight shape. - Each invocation handles a dedicated weight element. - Uses subgroupShuffle to efficiently access the source tile, leveraging k_vec4 vectorization for better memory throughput. Testing on Lunar Lake demonstrated **up to an 87%** performance improvement in Conv_2D operations. ### Motivation and Context See above.
5 participants
Description
This PR optimizes the
Convoperation by implementing two new compute shaders:oihw_to_ohwiandim2col-matmul.oihw_to_ohwi:Improves performance over the default Transpose shader by utilizing workgroup memory to ensure continuous memory read/write patterns.
im2col-matmul:Testing on Lunar Lake demonstrated up to an 87% performance improvement in Conv_2D operations.
Motivation and Context
See above.