[CALCITE-7822] AssertionError "type mismatch" in SqlToRelConverter for OVER () in a GROUP BY query by kalayciburak · Pull Request #5293 · apache/calcite

kalayciburak · 2026-09-25T08:48:14Z

Jira Link

Changes Proposed

A RANGE window with no ORDER BY does not need a sort key. Every row is a peer, so an empty order list is valid. That is true whether or not the FROM item has a monotonic column.

If the FROM item does have one (a column declared sorted, or a column sorted on in a subquery), the key is kept only when its input refs still name the same fields. After GROUP BY that column may be gone. The old code still built a RexInputRef from the old ordinal, so the ref type did not match the aggregate field and Project.isValid failed. The key is dropped in that case. It is not rebuilt by column name.

Tests

The queries are in core/src/test/resources/sql/winagg.iq:

select deptno, count(*) over () as c from emp group by deptno returns deptno 10, 20 and 30, each with c = 3
the same shape on a FROM item with no monotonic column (select deptno, sal from emp)
the same shape with an explicit RANGE frame
select job, max(max(sal)) over () from emp group by job
ROWS framing, an explicit ORDER BY, and GROUP BY of empno itself are unchanged

Ran sql/winagg.iq through CoreQuidemTest (JAVA_HOME=21). Passed. :core:checkstyleMain passed.

mihaibudiu · 2026-10-02T03:08:37Z

A RANGE window without ORDER BY takes an implicit sort key from the FROM item's monotonic column.

What is a monotonic column?

select deptno, count(*) over () as c from emp group by deptno on the scott schema now returns DEPTNO=10/20/30 with C=3. The same shape with an explicit RANGE frame and max(max(sal)) over () grouped by job also converts. ROWS framing, an explicit ORDER BY, and GROUP BY of the monotonic column itself are unchanged.

I am not sure I understand this

mihaibudiu · 2026-10-02T03:11:39Z

+   * <a href="https://issues.apache.org/jira/browse/CALCITE-7822">[CALCITE-7822]
+   * AssertionError "type mismatch" in SqlToRelConverter for OVER () in a
+   * GROUP BY query</a>. */
+  @Test void testEmptyOverInAggregateQuery() {


would these queries work as quidem tests in an .iq file?
These tests are much more readable

yep, they do. moved them to winagg.iq and dropped the java assertions.

mihaibudiu · 2026-10-02T03:14:50Z

+  }
+
+  /** Rebinds a simple column order key onto the current input, or null. */
+  private @Nullable RexNode rebindImplicitOrderKey(SqlNameMatcher nameMatcher,


what does "rebind" mean?

it meant look the column up by name on the current row, if GROUP BY didn't remove it. renamed the helper so it says that.

mihaibudiu · 2026-10-02T03:15:18Z

+    }
+  }
+
+  /** Whether every input ref in {@code orderKey} matches the current input. */


in what sense does it match the input?

the ref index is in range, and its type equals the field at that index (nullability aside). if the types differ, the ordinal now names a different column.

kalayciburak · 2026-10-02T05:27:07Z

monotonic column is a column the table marks as sorted. on scott.EMP that is EMPNO. a RANGE window with no ORDER BY uses it as the logical order.

after GROUP BY deptno, EMPNO is gone. the old code still built a ref from EMPNO's old ordinal (0), which is now DEPTNO, so the types don't match (SMALLINT vs TINYINT). that's the assertion.

rewrote that prose in the description, and moved the queries to winagg.iq.

mihaibudiu · 2026-10-02T21:07:02Z

-              RelFieldCollation.NullDirection.UNSPECIFIED,
-              bb::sortToRex));
+    if (orderList.isEmpty() && !rows) {
+      // RANGE with no ORDER BY still needs a logical order. That order is


still -> always

mihaibudiu · 2026-10-02T21:07:23Z

+    if (orderList.isEmpty() && !rows) {
+      // RANGE with no ORDER BY still needs a logical order. That order is
+      // the first monotonic column of the FROM item (a column the table
+      // declares as sorted, for example EMPNO). After GROUP BY that column


or a column that one has sorted on in a subquery

mihaibudiu · 2026-10-02T21:14:58Z

+   * Adds RANGE order keys taken from the FROM item when the window has no
+   * ORDER BY. Keeps a key whose input refs still name the same fields.
+   * Otherwise looks the column up by name on the current input, or drops
+   * it if GROUP BY removed it (CALCITE-7822).


in general you should not refer to issues in javadoc comments that are not test code.

mihaibudiu · 2026-10-02T21:15:41Z

+    if (orderList.isEmpty() && !rows) {
+      // RANGE with no ORDER BY still needs a logical order. That order is
+      // the first monotonic column of the FROM item (a column the table
+      // declares as sorted, for example EMPNO). After GROUP BY that column


do not refer to a column name in compiler code. empno is test data, the compiled does not depend on test data.

mihaibudiu · 2026-10-02T21:16:45Z

+      // the first monotonic column of the FROM item (a column the table
+      // declares as sorted, for example EMPNO). After GROUP BY that column
+      // may be gone. Its old ordinal then names a different field, and
+      // Project.isValid fails (CALCITE-7822).


remove reference to issue

mihaibudiu · 2026-10-02T21:29:22Z

+      // may be gone. Its old ordinal then names a different field, and
+      // Project.isValid fails (CALCITE-7822).
+      final SqlNodeList implicit = bb.scope.getOrderList();
+      if (implicit == null) {


I think this is a completely separate issue, which has nothing to do with the bug you are solving.
This is not about implicit sorting.
Can we focus on solving the issue even if the table does not have implicit sorting? This query should work for that case.

yeah, this shouldn't be about preserving the implicit sort. dropped the name lookup. if the refs don't match the current input the key is left off, so over () works with or without a monotonic column. added the no-monotonic case to winagg.iq.

also cleaned the comments, no test column name and no issue id in the compiler notes.

mihaibudiu · 2026-10-02T22:42:52Z

-              RelFieldCollation.Direction.ASCENDING,
-              RelFieldCollation.NullDirection.UNSPECIFIED,
-              bb::sortToRex));
+    if (orderList.isEmpty() && !rows) {


I thought we agreed to leave this part unchanged.
The fix should have nothing to do with the implicit order.

…r OVER () in a GROUP BY query A RANGE window without ORDER BY used the FROM item's monotonic column as an implicit sort key. After GROUP BY that column is no longer an input, but its original ordinal still became a RexInputRef, so Project.isValid failed (SMALLINT NOT NULL vs TINYINT). Keep the converted key when its input refs match the current input. Otherwise rebind a simple column by name, or drop the key if GROUP BY removed it. Unbounded RANGE already allows an empty order list.

The Java assertions were hard to read. Same queries now live in winagg.iq. Dropped the word rebind: a converted order key is kept when its refs still name the same fields, otherwise the column is looked up by name.

A window with no ORDER BY should run even when the FROM item has no monotonic column. Keep an implicit key only when its input refs still name the same fields. Do not rebuild it by column name.

sonarqubecloud · 2026-10-07T04:12:35Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
87.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

mihaibudiu reviewed Oct 2, 2026

View reviewed changes

kalayciburak added 3 commits October 6, 2026 01:23

[CALCITE-7822] Move OVER () GROUP BY tests to quidem

00c26b5

The Java assertions were hard to read. Same queries now live in winagg.iq. Dropped the word rebind: a converted order key is kept when its refs still name the same fields, otherwise the column is looked up by name.

[CALCITE-7822] Drop a RANGE order key that no longer matches the input

f11e99c

A window with no ORDER BY should run even when the FROM item has no monotonic column. Keep an implicit key only when its input refs still name the same fields. Do not rebuild it by column name.

kalayciburak force-pushed the calcite-7822-empty-over-groupby branch from 346561b to f11e99c Compare October 5, 2026 22:35

Conversation

kalayciburak commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Jira Link

Changes Proposed

Tests

Uh oh!

mihaibudiu commented Oct 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kalayciburak commented Oct 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Oct 7, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kalayciburak commented Sep 25, 2026 •

edited

Loading