Repository navigation
[VL] Spark 4.1: Add TimeType (TIME data type) support (SPARK-51162) #11919
Description
Backend
VL (Velox)
Gluten version: main branch
Description
Spark 4.1 introduced the TIME data type (SPARK-51162 SPIP, 101 sub-tasks). DataTypeTestUtils.atomicTypes now includes TimeType, causing all tests that iterate over atomicTypes to fail when encountering the unsupported type.
Feature toggle: spark.sql.timeType.enabled (default true in tests).
Spark 4.1 only.
Parent issue: #11910 ([VL] Spark 4.x: Tracking new feature support)
Impact
| Suite | Failures | How affected |
|---|---|---|
| GlutenRandomDataGeneratorSuite | 232 | atomicTypes enumeration |
| GlutenOrderingSuite | 2 | TimeType(0) / TimeType(6) |
| GlutenHiveResultSuite | 1 | New test (SPARK-51517) |
| GlutenCastWithAnsiOnSuite (partial) | ~6 | TimeType cast tests |
Note: GlutenCastWithAnsiOnSuite also has 4 ANSI Cast failures — those belong to #10134, not this issue.
Additionally, some SQL query test files may be blocked by TimeType (cast.sql, literals.sql, nonansi/literals.sql).
References
- Apache Spark JIRA: SPARK-51162
Activity
malinjawi commentedon May 9, 2026
Thanks for raising @baibaichen. Will check this out as I also have a patch for ANSI Time cast: facebookincubator/velox#16123