Umbrella tracking issue for re-enabling the Spark 4.2 unit tests that are currently disabled to keep the gluten-ut/spark42 CI green.
Added in PR #13163 (GLUTEN-12569). 150 UTs are disabled across 7 root-cause themes. Each theme has its own sub-issue with the exact disabled tests, the disable mechanism, and the files updated. As the underlying Gluten/Velox support lands, remove the corresponding .exclude(...) / allowlist / @Ignore / ignore(...) entries and close the sub-issue.
Sub-issues (7)
Summary
| # |
Theme |
Items |
Sub-issue |
| 1 |
Spark 4.2 Storage-Partitioned Join (SPJ) support in Gluten |
29 |
#13174 |
| 2 |
Spark 4.2 DSv2 row-level writes (DELETE/MERGE/UPDATE/INSERT) on Gluten path |
39 |
#13175 |
| 3 |
Floating-point collection semantics: -0.0/NaN normalization & RESPECT NULLS |
12 |
#13176 |
| 4 |
Golden-file drift on Spark 4.2 (TPC-DS plan-stability + SQLQueryTest .sql) |
46 |
#13177 |
| 5 |
Velox expression & operator gaps (math/TIME/date/broadcast/hash-agg) |
6 |
#13178 |
| 6 |
Spark 4.2 framework/API changes breaking Gluten tests (non-velox) |
7 |
#13179 |
| 7 |
File-source & Hive scan (Parquet/ORC ignoreMissingFiles/NullType, bucketed-scan AE, ADD JAR) |
11 |
#13180 |
|
Total |
150 |
|
How these were disabled
.exclude(...) in gluten-ut/spark42/.../velox/VeloxTestSettings.scala (most suites, honoring BackendTestSettings.shouldRun)
- allowlist removal in
gluten-ut/spark42/.../velox/VeloxSQLQueryTestSettings.scala (GlutenSQLQueryTestSuite .sql golden tests — that suite ignores the exclude registry)
- class-level
@org.scalatest.Ignore for whole-suite aborts (GlutenBloomFilterFallbackSuite, MiscOperatorSuite, GlutenWholeStageCodegenSuite)
test(...)->ignore(...) for one standalone test (GlutenAutoAdjustStageResourceProfileSuite)
All disables are CI-confirmed (local-only false-positives were removed via a validation run with excludes reverted).
Related: PR #13163, GLUTEN-12569.
Umbrella tracking issue for re-enabling the Spark 4.2 unit tests that are currently disabled to keep the
gluten-ut/spark42CI green.Added in PR #13163 (GLUTEN-12569). 150 UTs are disabled across 7 root-cause themes. Each theme has its own sub-issue with the exact disabled tests, the disable mechanism, and the files updated. As the underlying Gluten/Velox support lands, remove the corresponding
.exclude(...)/ allowlist /@Ignore/ignore(...)entries and close the sub-issue.Sub-issues (7)
Summary
How these were disabled
.exclude(...)ingluten-ut/spark42/.../velox/VeloxTestSettings.scala(most suites, honoringBackendTestSettings.shouldRun)gluten-ut/spark42/.../velox/VeloxSQLQueryTestSettings.scala(GlutenSQLQueryTestSuite.sqlgolden tests — that suite ignores the exclude registry)@org.scalatest.Ignorefor whole-suite aborts (GlutenBloomFilterFallbackSuite,MiscOperatorSuite,GlutenWholeStageCodegenSuite)test(...)->ignore(...)for one standalone test (GlutenAutoAdjustStageResourceProfileSuite)All disables are CI-confirmed (local-only false-positives were removed via a validation run with excludes reverted).
Related: PR #13163, GLUTEN-12569.