Part of #5572.
map_sort falls the whole projection back to Spark for any non-scalar map key:
if (!supportedScalarSortElementType(keyType)) {
Unsupported(Some(s"MapSort on map with key type $keyType is not supported"))
}spark/src/main/spark-4.x/org/apache/comet/serde/CometMapSort.scala:40-41
supportedScalarSortElementType (serde/QueryPlanSerde.scala:1238) admits the scalar types only, and excludes collated strings, so map_sort on a map keyed by a struct, array or collated string fails the operator.
MapSort.doGenCode sorts with Spark's interpreted ordering, which handles all of those, and MapType is supported recursively by CometBatchKernelCodegen.isSupportedDataType, so the dispatcher covers the gap.
Fix: mix CodegenDispatchFallback into CometMapSort.
The serde's other arm — Incompatible under spark.comet.exec.strictFloatingPoint=true for a floating-point key (:42-46) — would be routed by the same mixin, and that is the right outcome: strict mode is asking for Spark-exact behavior, which is precisely what the dispatcher gives.
This is a Spark 4.x-only serde, so the test belongs with the other 4.x-gated suites.
Part of #5572.
map_sortfalls the whole projection back to Spark for any non-scalar map key:spark/src/main/spark-4.x/org/apache/comet/serde/CometMapSort.scala:40-41supportedScalarSortElementType(serde/QueryPlanSerde.scala:1238) admits the scalar types only, and excludes collated strings, somap_sorton a map keyed by a struct, array or collated string fails the operator.MapSort.doGenCodesorts with Spark's interpreted ordering, which handles all of those, andMapTypeis supported recursively byCometBatchKernelCodegen.isSupportedDataType, so the dispatcher covers the gap.Fix: mix
CodegenDispatchFallbackintoCometMapSort.The serde's other arm —
Incompatibleunderspark.comet.exec.strictFloatingPoint=truefor a floating-point key (:42-46) — would be routed by the same mixin, and that is the right outcome: strict mode is asking for Spark-exact behavior, which is precisely what the dispatcher gives.This is a Spark 4.x-only serde, so the test belongs with the other 4.x-gated suites.