Skip to content

chore: move CometCollationSuite to spark-4.1+ test shim - #4864

Merged
andygrove merged 4 commits into
apache:mainfrom
0lai0:chore-4770-comet-collation-suite-4.1+
Oct 8, 2026
Merged

andygrove merged 4 commits into
apache:mainfrom
0lai0:chore-4770-comet-collation-suite-4.1+

Conversation

@0lai0

@0lai0 0lai0 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #4770

Rationale for this change

CometCollationSuite currently lives in the spark-4.1 test shim, but the same tests apply to Spark 4.2+. Moving it to spark-4.1+ and enabling that test shim avoids duplicating the suite per Spark minor version.

What changes are included in this PR?

  • spark/pom.xml: add src/test/${shims.minorPlusVerSrc} to the add-test-source execution so the spark-4.1+ test shim is compiled.
  • Move CometCollationSuite.scala from spark/src/test/spark-4.1/ to spark/src/test/spark-4.1+/ (content unchanged).

How are these changes tested?

./mvnw test -Pspark-4.1 -Dtest=none -Dsuites=org.apache.spark.sql.CometCollationSuite
./mvnw compile test-compile -Pspark-4.2 -DskipTests

@andygrove andygrove left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending CI. Thanks @0lai0

@andygrove

Copy link
Copy Markdown
Member

CI didn't run ....

@0lai0

0lai0 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @andygrove

@andygrove andygrove added enhancement New feature or request test Testing related labels Sep 6, 2026

@rich7420 rich7420 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

@sunchao

sunchao commented Sep 10, 2026

Copy link
Copy Markdown
Member

@0lai0 could you rebase this PR?

@0lai0
0lai0 force-pushed the chore-4770-comet-collation-suite-4.1+ branch from 2fd4856 to 74e2a13 Compare September 10, 2026 03:26
@0lai0

0lai0 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto latest main

@dwsmith1983

Copy link
Copy Markdown
Contributor

@sunchao's #6495 also deduplicates CometCollationSuite, and goes a step further by moving it to spark-4.x, so the Spark 4.0 copy is folded in too. The two conflict, so could you settle on one between you? If #6495 goes in, this one and #4770 can be closed.

@andygrove
andygrove enabled auto-merge October 8, 2026 15:42
@andygrove
andygrove added this pull request to the merge queue Oct 8, 2026
Merged via the queue into apache:main with commit 9a6241f Oct 8, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request test Testing related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move CometCollationSuite into the spark-4.1+ test shim to avoid per-version duplication

5 participants