Update optimizer opset version checks for latest ONNX opset 26

[Dmitri Smirnov (yuslepukhin)]

This pull request expands support for additional ONNX opset versions in the attention fusion optimization code, making the optimizer compatible with newer and more diverse ONNX models. The changes primarily update the accepted opset versions for various operators such as Transpose, Reshape, Squeeze, Unsqueeze, Shape, and others across multiple functions. This ensures broader model compatibility and improves the robustness of the fusion logic.

Expanded opset version support for attention fusion:

• Updated accepted opset versions for key operators (Transpose, Reshape, Squeeze, Unsqueeze, Shape, Add, Mul, Sub, Div, Cast, etc.) in the main attention fusion logic (attention_fusion.cc), allowing matching and fusion of newer ONNX models using these operators at opsets up to 25. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

Helper and mask subgraph matching improvements:

• Broadened opset version checks for subgraph matching in helper functions, including those for Gemm subgraphs, unidirectional mask subgraphs, input mask subgraphs, and past subgraph matching, to support additional opset versions and operator variants. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

These changes collectively future-proof the attention fusion optimizer for a wider range of ONNX models and operator versions, reducing the likelihood of unsupported patterns during optimization.

[Copilot]

Pull request overview

This PR expands ONNX Runtime optimizer pattern matching and unit tests to recognize newer ONNX operator schema versions (opset 23–25), aiming to keep attention fusions and reshape fusion behavior compatible with opset 25 models.

Changes:

• Broadened supported operator-version allowlists in optimizer fusions (e.g., Transpose/Reshape/Squeeze/Unsqueeze/Shape) to include newer schema versions up to opset 25.
• Added opset 25 coverage for MobileCLIP attention fusion and GroupQueryAttentionPreNorm fusion unit tests.
• Extended ReshapeFusionOpsetTest to iterate additional opsets (19/21/23/24/25).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
onnxruntime/core/optimizer/attention_fusion.cc	Updates MobileCLIP attention fusion pattern version checks for newer ONNX schemas.
onnxruntime/core/optimizer/attention_fusion_helper.h	Extends supported Transpose versions in GPT attention helper logic.
onnxruntime/core/optimizer/group_query_attention_pre_norm_fusion.cc	Expands supported Reshape versions in the GQA pre-norm fusion matcher.
onnxruntime/core/optimizer/reshape_fusion.cc	Updates Shape/Unsqueeze schema version handling in reshape fusion logic.
onnxruntime/test/optimizer/graph_transform_test.cc	Adds opset coverage (incl. 25) for attention and reshape fusion tests.
onnxruntime/test/optimizer/group_query_attention_pre_norm_fusion_test.cc	Adds opset 25 test for Qwen GQA pre-norm fusion.

Comments suppressed due to low confidence (1)

onnxruntime/test/optimizer/graph_transform_test.cc:8241

• ReshapeFusionOpsetTest now iterates opsets 19/21/23/24/25, but the shape_test_for_opset15 flag is mutated inside build_test_case and then reused across iterations. After the first opset>=15 run, subsequent iterations build a Shape with start=1,end=2 and also switch to the (pre,pre) checker branch, so the newly added opsets are not actually validating the fusion path this test is meant to cover.

  const std::vector<int> opsets{11, 12, 13, 14, 15, 18, 19, 21, 23, 24, 25};
  bool shape_test_for_opset15 = false;

  for (auto& opset : opsets) {
    auto build_test_case = [&](ModelTestBuilder& builder) {

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

onnxruntime/core/optimizer/reshape_fusion.cc:181

• The Shape(start/end) guard rejects any explicit end attribute, even if it is set to the default "no slicing" value (e.g., INT64_MAX). That can unnecessarily block reshape-fusion for graphs that redundantly set end to the default. Consider treating an end attribute with a very large value (i.e., equivalent to full-shape) as acceptable, and only rejecting when start/end imply an actual slice.

    // Opset 15+ added start/end attributes to Shape. Reject partial-shape queries.
    if (shape.SinceVersion() >= 15) {
      const ONNX_NAMESPACE::AttributeProto* start_attr = graph_utils::GetNodeAttribute(shape, "start");
      const ONNX_NAMESPACE::AttributeProto* end_attr = graph_utils::GetNodeAttribute(shape, "end");
      if (!((!start_attr || static_cast<int>(start_attr->i()) == 0) && (!end_attr))) {
        return false;
      }

[Hariharan Seshadri (hariharans29)]

Verdict: Approve, but attention_fusion_helper.h has an internal inconsistency worth fixing before merge

The mechanical opset-list expansions are fine, and two of the changes are actually meaningful correctness fixes hiding inside the "version bump" framing. One change in attention_fusion_helper.h is half-done in a way that defeats its own purpose — Copilot's second comment is accurate and worth acting on.

---

The two real fixes hiding in here

1. Shape SinceVersion check in reshape_fusion.cc — correctness fix

Pre-PR:

if (graph_utils::MatchesOpSinceVersion(shape, {15})) {
  // check start/end attributes that, if set, would block fusion
  ...
}

MatchesOpSinceVersion(shape, {15}) returns true only when shape.SinceVersion() == 15. For a model at opset 19 or 21 (where Shape has SinceVersion 19/21 respectively), this returned false, skipped the start/end attribute check, and would fuse a partial-Shape pattern that should not be fused. This is a latent correctness bug being silently fixed.

Post-PR:

if (shape.SinceVersion() >= 15) { ... }

is the right shape — once the schema added the attribute, every later version inherits it. This deserves a callout in the PR description because the framing "extend opset version checks" undersells it; "fix partial-Shape detection on opset ≥ 19" would be more accurate.

The test side does pick this up — ReshapeFusionOpsetTest was previously contorted into a one-shot state machine (shape_test_for_opset15) so that the partial-shape path ran exactly once across all opsets, and the new test now runs the partial-shape negative case for every opset ≥ 15. That's the right refactor.

2. Unsqueeze axes-from-input in reshape_fusion.cc — same class of fix

Pre-PR:

} else if (graph_utils::MatchesOpSinceVersion(unsqueeze, {13})) {
  const NodeArg* axes_node_arg = unsqueeze.InputDefs()[1];
  ...
}

Same issue: only matched SinceVersion() == 13. For opset 21+ where Unsqueeze has SinceVersion 21 (or whichever version it was bumped to), this returned false → reshape fusion failed silently on those models. The new structural check InputDefs().size() > 1 fixes it.

Copilot suggested gating on SinceVersion() >= 13 instead. Two reasonable opinions:

• Pro SinceVersion() >= 13: schema-aligned, matches the comment, defends against a hypothetical malformed Unsqueeze that has the wrong arity for its version.
• Pro InputDefs().size() > 1: future-proof against any later opset bump (no need to revisit), and the "malformed Unsqueeze with wrong arity" case would fail schema validation before reaching this code anyway.

I'd take the structural check as you have it. The defensiveness Copilot is asking for is downstream of schema validation, and you'd otherwise need to update this site on every Unsqueeze bump going forward. Author's call — not blocking either way.

---

The one thing worth addressing before merge

FuseGptAttention in attention_fusion_helper.h is internally inconsistent

The PR updates one line in this function:

// line 1450
if (graph_utils::IsSupportedOptypeVersionAndDomain(*k_concat, "Transpose",
                                                   {1, 13, 21, 23, 24, 25}, kOnnxDomain)) {
  transpose_optimized_pattern = true;
  ...
}

But everything downstream of that gate is still locked to the old opsets:

// ~line 1468
if (!graph_utils::IsSupportedOptypeVersionAndDomain(*k_concat, "Concat",
                                                    {4, 11, 13}, kOnnxDomain)) {
  return false;
}

// ~line 1474
std::vector<graph_utils::EdgeEndToMatch> k_path{
    {0, 1, "Transpose", {1, 13},     kOnnxDomain},
    {0, 0, "Reshape",   {5, 13},     kOnnxDomain},
    {1, 0, "Split",     {2, 11, 13}, kOnnxDomain}};

Consequence: on an opset 23/24/25 GPT model the precheck succeeds, then FindPath fails because the Transpose has SinceVersion 21 (or whichever) and isn't in {1, 13}. Net result of the one-line change in this function: nothing. Either:

• (a) Update the q/k/v path matchers, Reshape {5, 13} → {5, 13, 14, 19, 21, 23}, Split {2, 11, 13} → {2, 11, 13, 18, ...}, Concat {4, 11, 13} → matching set, and the inner Transpose {1, 13} → {1, 13, 21, 23, 24, 25} — consistent with the stated PR intent. Plus an opset-25 test for the GPT path the same way you did for MobileCLIP and the GQA pre-norm fusion.
• (b) Or revert the line 1450 change and explicitly scope the PR to "opset 25 for MobileCLIP / GQA pre-norm / reshape fusion" only, since FuseGptAttention won't actually work end-to-end at opset 25 without the rest.

Either is fine. The current state is the one option that doesn't make sense.

Copilot's second comment captured this. Acting on it would close the gap.

---

Pattern-level observations on the version list extensions

Adding 24 and 25 to every list

Reshape {5, 13, 14, 19, 21, 23, 24, 25}, Transpose {1, 13, 21, 23, 24, 25}, etc.

IsSupportedOptypeVersionAndDomain checks SinceVersion(), which is the version the operator's schema was last changed, not the model's opset. So the entries that actually do anything are the versions where the operator was bumped. If Reshape was last bumped at v23, then {24, 25} in its list are no-ops (a model declared at opset 25 will still report Reshape.SinceVersion() == 23).

This is harmless future-proofing, and consistent with how similar extensions have been done in the repo before — I'd just flag in the PR description what was actually bumped at 24/25 vs. what was added defensively. Helps the next person doing the same exercise understand which entries are load-bearing.

Hot-spot to harmonize as follow-up (out of scope here)

This file (attention_fusion.cc) has the same set of operator version lists repeated 9 times for Reshape and 6 times for Transpose. Every opset bump now triggers a search-and-replace across the file, with the FuseGptAttention mistake above being a natural consequence. A small refactor into named constants (kReshapeOpsetVersions, kTransposeOpsetVersions) would have made this PR a 4-line change and made the next one trivial. Not for this PR — file as a cleanup.

---

Tests look right

• AttentionFusionMobileClipMhaOpset25Test is the parallel form of the existing AttentionFusionMobileClipMhaTest (opset 14 → 25), reusing the same helper and checker. Minimal and correct.
• GroupQueryAttentionPreNormFusionFusesQwenPatternOpset25 likewise parallels the existing test.
• ReshapeFusionOpsetTest refactor (drop the state machine, run the positive case always and the partial-Shape negative case for every opset ≥ 15) is cleaner and increases coverage. Good.

One small note: ReshapeFusionOpsetTest now iterates {11, 12, 13, 14, 15, 18, 19, 21, 23, 24, 25} but skips 16, 17, 20, 22. If those gaps are intentional (Reshape unchanged at those opsets so they collapse to the previous SinceVersion), fine. If not, adding them is one character each. Minor.

---

Bottom line

The two reshape_fusion.cc corrections are nice quiet wins. The mechanical version-list extensions in attention_fusion.cc are fine. The attention_fusion_helper.h change is currently a no-op due to the unchanged downstream matchers — either complete it (preferred, with a matching test) or drop it. Recommend addressing that one point and then this is good to land.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

onnxruntime/core/optimizer/reshape_fusion.cc:181

• The new Shape start/end-attribute guard rejects any node that has an "end" attribute, even when end is the default full-range value. ORT’s Shape kernel treats end==std::numeric_limits<int64_t>::max() as the default (full shape), so this check can incorrectly block ReshapeFusion matching for models/exporters that explicitly set end to INT64_MAX.

    // Opset 15+ added start/end attributes to Shape. Reject partial-shape queries.
    if (shape.SinceVersion() >= 15) {
      const ONNX_NAMESPACE::AttributeProto* start_attr = graph_utils::GetNodeAttribute(shape, "start");
      const ONNX_NAMESPACE::AttributeProto* end_attr = graph_utils::GetNodeAttribute(shape, "end");
      if (!((!start_attr || static_cast<int>(start_attr->i()) == 0) && (!end_attr))) {
        return false;
      }