Backend
Velox (the window parts also apply to Bolt and ClickHouse, see below).
Bug description
Two kinds of RANGE window frames fail the query instead of falling back to vanilla Spark.
1. A date order key with an interval bound fails planning on Velox and Bolt. PullOutPreProject pre-computes RANGE frame boundaries for date keys by turning the key into an integer (datediff from epoch) and adding the bound to it. That only works for an integer bound. With an interval bound it builds Add(int, interval), which never resolves, and planning fails:
select l_commitdate,
count(*) over (order by l_commitdate range between interval '1' month preceding and current row)
from lineitem
org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to dataType on unresolved object
range between 1 preceding and current row on the same date key works.
2. A non-integral bound fails the query with NumberFormatException on ClickHouse. WindowFunctionNode.setBound reads a literal frame bound with Long.valueOf(bound.eval(null).toString()). Spark casts a RANGE bound to the order key type, so on a decimal key both 1.5 PRECEDING and 10 PRECEDING arrive as -1.50 / -10.00. The resulting NumberFormatException is not one of the exceptions validation turns into a fallback, so the query fails. ClickHouse's supportWindowExec does not check frame bounds, so a query like this reaches it:
select order_id, count(*) over (partition by category order by price
range between 10 preceding and 10 following)
from orders -- price DECIMAL(18,2)
Velox and Bolt already reject non-integral sort keys before this point. The ClickHouse path was traced by reading the code; I could not run the ClickHouse backend.
Gluten version
main (2681cc9)
Spark version
Spark 3.5
This issue was written with the assistance of AI (Claude Opus).
Backend
Velox (the window parts also apply to Bolt and ClickHouse, see below).
Bug description
Two kinds of RANGE window frames fail the query instead of falling back to vanilla Spark.
1. A date order key with an interval bound fails planning on Velox and Bolt.
PullOutPreProjectpre-computes RANGE frame boundaries for date keys by turning the key into an integer (datedifffrom epoch) and adding the bound to it. That only works for an integer bound. With an interval bound it buildsAdd(int, interval), which never resolves, and planning fails:range between 1 preceding and current rowon the same date key works.2. A non-integral bound fails the query with
NumberFormatExceptionon ClickHouse.WindowFunctionNode.setBoundreads a literal frame bound withLong.valueOf(bound.eval(null).toString()). Spark casts a RANGE bound to the order key type, so on a decimal key both1.5 PRECEDINGand10 PRECEDINGarrive as-1.50/-10.00. The resultingNumberFormatExceptionis not one of the exceptions validation turns into a fallback, so the query fails. ClickHouse'ssupportWindowExecdoes not check frame bounds, so a query like this reaches it:Velox and Bolt already reject non-integral sort keys before this point. The ClickHouse path was traced by reading the code; I could not run the ClickHouse backend.
Gluten version
main (2681cc9)
Spark version
Spark 3.5
This issue was written with the assistance of AI (Claude Opus).