[WebGPU] Enable GeluFusion and BiasGeluFusion for the WebGPU EP

[Fanchen Kong (fanchenkong1)]

Description

Add a new EP allowlist that includes WebGPU
(cpu_acl_cuda_dml_webgpu_eps), and use it for the Level-2 GeluFusion and BiasGeluFusion transformers. This allows the erf-based GELU pattern (Div -> Erf -> Add -> Mul -> Mul, optionally preceded by a bias Add) to fuse into the WebGPU EP's kMSDomain Gelu and BiasGelu kernels instead of running as separate elementwise dispatches.

This covers models using older opsets that the opset >= 20 Level-1 GeluFusion transformer does not handle.

Performance Impact

In native WebGPU build, the vision submodel of zero-shot image classification achieved a 1.11x wall-clock speedup on Panther Lake and a 1.15x speedup on Wildcat Lake.

Platform	Latency reduction	Speedup
Intel Wildcat Lake (WCL)	−13.0%	1.15×
Intel Panther Lake (PTL)	−9.9%	1.11×

This PR addresses the Gelu and BiasGelu items listed in
#29841.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Fanchen Kong (fanchenkong1)]

Jiajia Qin (@qjia7) , this change is ready for review. PTAL, thanks!

[Jiajia Qin (qjia7)]

Fanchen Kong (@fanchenkong1) The changes look good to me. Please also link #29841 into your description. Please also check whether there are responding cases cover this new path for webgpu.

BTW, would you like to take #29841 to see anything else need to be done as a follow-up?

[Fanchen Kong (fanchenkong1)]

Jiajia Qin (@qjia7), Thanks for reviewing this change! For test coverage, I added a new Gelu_half to validate FP16 Gelu on WebGPU EP. The graph transformations are already covered by the existing GraphTransformationTests.

As a follow-up, I will check the list at #29841. And enable the ones that show a worthwhile speedup.

[Jiajia Qin (qjia7)]

Review frame

• Problem/feature validity: Validated. For models below ONNX opset 20, Level-1 GeluFusion cannot emit the official ONNX Gelu, so the erf decomposition remains assigned as separate WebGPU elementwise nodes. Level 2 can legally replace that assigned subgraph with com.microsoft::Gelu; WebGPU registers both com.microsoft::Gelu and com.microsoft::BiasGelu for float/float16.
• Risk/scope: Deep because this changes post-partition graph-rewrite eligibility and therefore node ownership and execution on WebGPU. The source change is narrow: it adds WebGPU to the Level-2 allowlists for GeluFusion and BiasGeluFusion, plus a standalone FP16 Gelu kernel test.
• Direction gate: Pass. The owner solution is the same bounded allowlist change: preserve Level-1 behavior for opset >= 20, then enable the existing Level-2 transformers only for an EP that registers the fused contrib kernels. The PR does this without changing schemas, kernels, or general fusion matching.

Confirmed findings

T1: Add a regression test for the WebGPU-owned transformation path

The added ActivationOpTest.Gelu_half constructs com.microsoft::Gelu directly (activation_op_test.cc:63-76). It proves the existing FP16 WebGPU kernel can execute, but it cannot fail if either changed allowlist entry in GenerateTransformers() is removed.

The cited GraphTransformationTests also do not cover this change. The legacy-opset tests at graph_transform_test.cc:7448-7639 instantiate GeluFusion and BiasGeluFusion directly with an empty compatible-provider set, and their graphs have no WebGPU ownership. The current-opset tests at graph_transform_test.cc:7185-7330 likewise instantiate unrestricted transformers directly. None obtains the production Level-2 transformers from GenerateTransformers(), assigns the decomposition to kWebGpuExecutionProvider, or checks that the fused node retains WebGPU ownership.

As a result, a later removal or typo in either new allowlist entry would leave all of these tests green while silently restoring the extra WebGPU dispatches this PR is intended to eliminate. Please add a regression test that exercises the production Level-2 registration on WebGPU-owned nodes. A pre-opset-20 Add(bias) -> Div -> Erf -> Add -> Mul -> Mul graph can cover both changed entries in one case: after applying the generated Level-2 transformers, assert that the graph contains one com.microsoft::BiasGelu, no decomposition nodes, and that the fused node is assigned to kWebGpuExecutionProvider. A no-bias case may be used separately if that is clearer.

Clarifications

None.

Test coverage

• Fused kernel correctness: covered for float/float16 BiasGelu by existing element_wise_ops_test.cc cases; this PR adds direct float16 Gelu coverage.
• Fusion pattern matching: covered by existing provider-unrestricted graph-transformer tests.
• Production WebGPU allowlist and provider-preservation path: missing; T1 requests this regression coverage.
• CI at head 634499a03a08cfa63d0624bbe7594bc6702d094f: native WebGPU build/test jobs passed. The CUDA plugin failure was a self-hosted runner communication loss, and the iPhone simulator failure was an HTTP 500 while downloading FXdiv; neither is attributable to these two changed files.

Verdict

The feature is valid and the design direction is appropriate: the fused WebGPU contrib kernels already exist, and enabling the existing post-partition transformers is the correct abstraction boundary. T1 is the merge blocker because the current tests do not exercise the behavior changed by this PR, despite the discussion claiming that they do. There are no remaining clarification requests and no cleanup-only comments.

[Copilot]

Pull request overview

Enables Level-2 GELU and BiasGELU fusion for the WebGPU execution provider.

Changes:

• Adds WebGPU to GELU fusion allowlists.
• Adds WebGPU fusion regression tests.
• Adds FP16 GELU kernel coverage.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
onnxruntime/core/optimizer/graph_transformer_utils.cc	Enables WebGPU GELU fusions.
onnxruntime/test/optimizer/graph_transform_test.cc	Tests WebGPU fusion registration.
onnxruntime/test/contrib_ops/activation_op_test.cc	Tests FP16 GELU execution.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Fanchen Kong (fanchenkong1)]

Hi Jiajia Qin (@qjia7), two regression tests are added for WebGPU fusion registration. PTAL, thanks!

[Fanchen Kong (fanchenkong1)]

WebCI failed with 'Cannot start ChromeHeadless', which appears to be an infrastructure-related issue.

[Hariharan Seshadri (hariharans29)]

WebCI failed with 'Cannot start ChromeHeadless', which appears to be an infrastructure-related issue.

Thanks. Relevant folks are aware and looking at this. As soon as there is a fix, I ll ping this thread for a rebase or I ll retry if it needs no commit from main.

[Hariharan Seshadri (hariharans29)]

For some reason, it isn't queueing the CUDA Plugin builds. Could you rebase with main please ?

EDIT: Seems like it all passed.