[WebGPU] Fuse eight more activations into Conv - #32117
Merged
Hariharan Seshadri (hariharans29) merged 10 commits intoAug 30, 2026
Merged
[WebGPU] Fuse eight more activations into Conv#32117Hariharan Seshadri (hariharans29) merged 10 commits into
Hariharan Seshadri (hariharans29) merged 10 commits into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Extend the WebGPU Conv+activation fusion allowlist to QuickGelu, HardSwish, Elu, Gelu, Gelu(tanh), Softplus, ThresholdedRelu and Erf, with matching WGSL snippets for the generated-shader path and the im2col template. QuickGelu at alpha == 1 drops the multiply, so it carries its own cache-key term. Elu was missing from the ONNX layout-transpose handler map, which kept the transpose wedged between an internal-NHWC Conv and Elu; add it so the ONNX op reaches the fusion. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
com.microsoft.QuickGelu declares alpha with a default of 1.702 (contrib_defs.cc), the GELU approximation. The fusion fell back to 1.0, which is SiLU/Swish - a different function. Elu and ThresholdedRelu sit directly below with the same shape of code and both legitimately default to 1.0, so this reads as copy-paste. The standalone WebGPU QuickGelu kernel already uses 1.702f. Scope: Graph::Resolve() materializes schema attribute defaults onto every node before any transformer runs, so GetNodeAttribute never actually returned null here and no shipped model was miscompiled. The constant was wrong but unreachable. It is fixed because the fallback is the documented contract and nothing guarantees the resolve-time fill stays in place. Adds two tests: one pinning that a bare QuickGelu fuses with 1.702, and one that strips the attribute after Resolve() so the fallback constant itself is exercised. Only the second fails without this change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
std::setprecision is declared in <iomanip>; the file relied on it arriving transitively through <sstream>, which is not guaranteed. size_t is used unqualified, so add <cstddef> as well. Every other standard symbol in the file already has its header: numeric_limits/<limits>, ostringstream/<sstream>, string/<string>, vector/<vector>, isdigit and isalnum/<cctype>, strtof/<cstdlib>. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The transpose optimizer is shared cross-EP infrastructure, not WebGPU code. A one-line change to onnx_transpose_optimization.cc buried in a 34-file WebGPU PR gets reviewed by WebGPU people rather than the owners of that file, so it moves to a dedicated follow-up alongside the three contrib Gelu-family handlers it belongs with. Consequence, disclosed in the PR description: until that follow-up lands, Elu, com.microsoft.QuickGelu, com.microsoft.Gelu and com.microsoft.FastGelu do not fuse end-to-end, because layout transform leaves a Transpose between the NHWC Conv and the activation and ConvActivationSelector requires the Conv's sole consumer to be the activation. These activations still execute and produce correct results, just unfused. This is an unrealized optimization, not a correctness issue. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
An absent optional input may be serialized as an entry with an empty name rather than omitted, so FastGelu(X, "") still leaves two entries in InputDefs(). The count-based check rejected the Conv fusion for that form even though it is semantically bias-free. Use NodeArg::Exists(), which is already the idiom in this file (HasElementDataType), so both spellings of "no bias" fuse. This also removes a contradiction with the HandleFastGelu transpose handler landing in the stacked PR, which treats the empty-name form as bias-free and pushes Transposes through it: the layout churn would be paid and the fusion then refused. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
std::ostringstream is imbued with the global locale. An embedding application that calls std::locale::global() with a comma-decimal locale (common for European desktop software, and ORT is a library embedded into such applications) would make FloatLiteral emit "0,3275911". That string reaches WGSL shader source through GetActivationDeclaration and GetActivationSnippet, where f32(0,3275911) parses as a two-argument constructor call and fails shader compilation. Format with the EP's OStringStream (FastOStringStream) instead. Its floating-point operator<< uses std::to_chars, which by specification does not consult the locale, and which emits the shortest representation that round-trips exactly. This is not a new dependency, and shortest round-trip is not a new convention. BatchNormalization already streams a raw float straight into WGSL shader source through the same class: batch_norm.cc:86: << "inverseSqrt(input_var + " << epsilon_ << ")" with `float epsilon_;` at batch_norm.h:22, and it puts the same float in the program cache key at batch_norm.cc:126. So the floating-point std::to_chars path is not merely present in a header, it is already instantiated and load-bearing in shipping shader generation, and shortest round-trip is already this EP's de facto rendering for floats embedded in WGSL. FloatLiteral's max_digits10 was the outlier; this makes it consistent. (The repo also pins C++20, so every supported toolchain has floating-point std::to_chars regardless.) Shortest round-trip also shortens the generated source: 0.1f now renders as "0.1" rather than the "0.100000001" that max_digits10 forced. ConstantsAreEmittedAtFullPrecision measured digit counts against max_digits10, which no longer describes the emitter, so it is replaced by ConstantsUseShortestRoundTripForm, and the exact-value coverage it used to provide is folded into KnownConstantsAreNotTruncated by pinning all six erf coefficients. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Responds to review feedback on microsoft#32048: comments should state the invariant being enforced and why it matters locally, rather than narrating implementation history or every way a future change could invalidate a test. Deletes one comment that only described a hypothetical regression, and compresses six others that either duplicated the rationale already present at the implementation site or enumerated failure modes the assertion itself makes obvious. Comments only; no code changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This end-to-end parity test requires the Elu transpose handler, which 02791ec moved out of this PR. Without it the transpose optimizer cannot push the Transpose out from between the NHWC Conv and the Elu, so the Conv never absorbs the activation and the test fails: Conv did not absorb Elu, so the fused kernel was never executed. Graph was: .Elu .Transpose .Transpose com.ms.internal.nhwc.Conv Elu is the only activation these parity tests use that is absent from the transpose optimizer's handler map (onnx_transpose_optimization.cc:2759); Relu, LeakyRelu, HardSigmoid, Clip, HardSwish, Softplus, Gelu, ThresholdedRelu, Erf, Mul and Sigmoid are all present, which is why only the Elu case regressed. Following the rule already used on this stack, a test that needs two layers lives at the top layer, so this moves to the PR that owns the handler rather than dragging the handler back down here. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Ananya Anand (4n4ny4)
force-pushed
the
webgpu-conv-activation-eight-more
branch
from
August 23, 2026 06:25
b61ad23 to
1a9d2c9
Compare
Contributor
Author
|
/azp run |
|
No pipelines are associated with this pull request. |
cpplint's build/include_order rule wants C++ system headers before other project headers. <cstddef> was added after op_kernel_info.h and string_macros.h, which reviewdog flagged on the Optional Lint C++ job.
|
Azure Pipelines: No pipelines were found matching this branch/path. |
Removed redundant comment about QuickGelu alpha default value.
Jiajia Qin (qjia7)
approved these changes
Aug 30, 2026
Hariharan Seshadri (hariharans29)
approved these changes
Aug 30, 2026
Hariharan Seshadri (hariharans29)
merged commit Aug 30, 2026
76b1d96
into
microsoft:main
90 checks passed
Ananya Anand (4n4ny4)
added a commit
that referenced
this pull request
Aug 31, 2026
…U variants (#32118) ### Description `OrtExtendedHandlers()` had no entries for `com.microsoft.Gelu`, `FastGelu` and `QuickGelu`, and the ONNX map was missing `Elu`, so layout propagation stopped at the activation and left a transpose wedged between the NHWC Conv and the activation where `ConvActivationFusion` never saw them adjacent. Results stayed correct, the fusion just silently did not fire. `FastGelu` needs a bespoke handler rather than a table entry because its optional bias is pinned to the last dimension, and `broadcast_node_handler` would rank-normalize it into a rank-4 bias that fails `bias_gelu_helper::CheckInputs` outright, so the handler pushes the transpose only when there is no bias. `Elu` lives here rather than in #32117 because the transpose optimizer is shared cross-EP infrastructure, not WebGPU code. This also registers `FastGeluFusion` for WebGPU, which was cpu, cuda and dml only, since `GeluFusion` and `BiasGeluFusion` already landed in #32053. Covered by 6 fusion tests and 5 transpose-optimizer tests, including a negative test that `FastGelu` with a bias is left alone. ### Motivation and Context Models often already contain contrib GELU nodes, since `QuickGeluFusion` and `GeluFusion` run before layout transformation, so without these handlers the Conv activation fusion never fires on exactly the models most likely to benefit. #32117 adds the WebGPU side of the same work and needs these handlers to fuse end to end. --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Ananya Anand <t-anaanand@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Extends the WebGPU Conv+activation fusion allowlist by eight kinds: QuickGelu, HardSwish, Elu, Gelu, Gelu(tanh), Softplus, ThresholdedRelu and Erf, each with a WGSL snippet for the generated-shader path and a matching branch in the im2col template. QuickGelu at alpha == 1 is a distinct shader because the multiply drops out entirely, so it carries its own
QuickGeluUnitAlphacache-key term to stop the two variants colliding in the pipeline cache. This also fixes the QuickGelu alpha fallback, which was1.0frather than the schema default1.702f: attributes are materialized onto nodes duringGraph::Resolve(), so the fallback was unreachable and could not change model output, but it was still wrong on paper and inconsistent with the standalone WebGPU QuickGelu kernel. Four of the eight (Eluand the three contrib GELU variants) also need a transpose-optimizer handler to fuse end to end, and since that map is shared cross-EP infrastructure rather than WebGPU code it lives in #32118; until that lands those four still execute correctly, just unfused. Covered by 35 new tests, including negative controls for Selu and the CPU EP, execution parity against unfused results, and one that strips QuickGelu's alpha afterResolve()so it actually fails without the fix.Motivation and Context
These activations are common after convolutions. QuickGelu is how SiLU/Swish reaches the graph and HardSwish appears throughout MobileNet-class models, but each one previously forced a separate dispatch and a round trip through global memory.