[WebGPU] Add transpose-optimizer handlers for Elu and the contrib GELU variants

[Ananya Anand (4n4ny4)]

Description

OrtExtendedHandlers() had no entries for com.microsoft.Gelu, FastGelu and QuickGelu, and the ONNX map was missing Elu, so layout propagation stopped at the activation and left a transpose wedged between the NHWC Conv and the activation where ConvActivationFusion never saw them adjacent. Results stayed correct, the fusion just silently did not fire. FastGelu needs a bespoke handler rather than a table entry because its optional bias is pinned to the last dimension, and broadcast_node_handler would rank-normalize it into a rank-4 bias that fails bias_gelu_helper::CheckInputs outright, so the handler pushes the transpose only when there is no bias. Elu lives here rather than in #32117 because the transpose optimizer is shared cross-EP infrastructure, not WebGPU code. This also registers FastGeluFusion for WebGPU, which was cpu, cuda and dml only, since GeluFusion and BiasGeluFusion already landed in #32053. Covered by 6 fusion tests and 5 transpose-optimizer tests, including a negative test that FastGelu with a bias is left alone.

Motivation and Context

Models often already contain contrib GELU nodes, since QuickGeluFusion and GeluFusion run before layout transformation, so without these handlers the Conv activation fusion never fires on exactly the models most likely to benefit. #32117 adds the WebGPU side of the same work and needs these handlers to fuse end to end.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Copilot reviewed 27 out of 34 changed files in this pull request and generated no new comments.

Suppressed comments (4)

onnxruntime/core/optimizer/conv_activation_fusion.cc:210

• QuickGelu's schema default is alpha = 1.702f (core/graph/contrib_ops/contrib_defs.cc:624), but an omitted attribute is encoded here as 1.0. A valid com.microsoft.QuickGelu node without an explicit alpha will therefore be fused as SiLU and produce different results.

      activation_params.push_back(alpha_attr == nullptr ? 1.0f : alpha_attr->f());

onnxruntime/core/optimizer/conv_activation_fusion.cc:108

• An explicitly empty optional bias input is semantically absent, but it still leaves two entries in InputDefs(). This check rejects FastGelu(X, "") even though the new transpose handler correctly treats that form as elementwise, so layout propagation succeeds but the intended Conv fusion still does not happen. Check Exists() for the optional slot, as other optimizer code does.

This issue also appears on line 210 of the same file.

        return activation_node.InputDefs().size() == 1;

onnxruntime/test/providers/webgpu/activation_snippet_test.cc:6

• std::setprecision is declared by <iomanip>, which this new test does not include. Depending on transitive includes from gtest makes this translation unit non-portable and can fail to compile when those headers change.

#include <limits>

onnxruntime/core/providers/webgpu/nn/fuse_utils.cc:22

• std::ostringstream honors the process locale, so after an application installs a locale with a comma decimal separator this can emit WGSL such as 0,707..., causing shader compilation to fail. Shader literals must be locale-independent; imbue the classic locale before formatting.

std::string FloatLiteral(float value) {
  std::ostringstream oss;
  oss << std::setprecision(std::numeric_limits<float>::max_digits10) << value;
  return oss.str();

[Hariharan Seshadri (hariharans29)]

Can you please rebase with main ? For some reason, the 2 CI checks are not queueing. Hopefully a rebase fixes that.

[Ananya Anand (4n4ny4)]

Rebased onto main. #32117 merged in the meantime, so the eight commits this was stacked on are gone. The push dismissed your approval, could you re-approve? Thank you!