Part of #5572.
lpad and rpad decline two shapes, and both fall the whole projection back to Spark:
if (expr.str.isInstanceOf[Literal]) {
return Unsupported(Some(PadReasons.literalStrReason))
}
if (!expr.pad.isInstanceOf[Literal]) {
return Unsupported(Some(PadReasons.nonLiteralPadReason))
}
spark/src/main/scala/org/apache/comet/serde/strings.scala:413-419 (CometStringRPad) and :441-447 (CometStringLPad)
The non-literal pad restriction is the one that matters. lpad(name, 10, pad_col) — padding with a value that varies per row — is a perfectly ordinary query shape, and there is nothing about it that Spark's generated code cannot do. StringRPad and StringLPad are plain codegen expressions over StringType and IntegerType, all of which CometBatchKernelCodegen.isSupportedDataType admits.
The literal-str restriction is lower value (ConstantFolding handles most of it) but costs nothing extra to cover.
Fix: mix CodegenDispatchFallback into both serdes.
Worth checking while in here: docs/source/user-guide/latest/expressions.md renders lpad and rpad with an em-dash in the Implementation column, meaning GenerateDocs found no serde registered under those function names. Either the registry name differs from the function name or the rows need fixing — either way the doc currently says nothing useful about how these execute.
Part of #5572.
lpadandrpaddecline two shapes, and both fall the whole projection back to Spark:spark/src/main/scala/org/apache/comet/serde/strings.scala:413-419(CometStringRPad) and:441-447(CometStringLPad)The non-literal
padrestriction is the one that matters.lpad(name, 10, pad_col)— padding with a value that varies per row — is a perfectly ordinary query shape, and there is nothing about it that Spark's generated code cannot do.StringRPadandStringLPadare plain codegen expressions overStringTypeandIntegerType, all of whichCometBatchKernelCodegen.isSupportedDataTypeadmits.The literal-
strrestriction is lower value (ConstantFoldinghandles most of it) but costs nothing extra to cover.Fix: mix
CodegenDispatchFallbackinto both serdes.Worth checking while in here:
docs/source/user-guide/latest/expressions.mdrenderslpadandrpadwith an em-dash in the Implementation column, meaningGenerateDocsfound no serde registered under those function names. Either the registry name differs from the function name or the rows need fixing — either way the doc currently says nothing useful about how these execute.