What is the problem the feature request solves?
Note: This issue was generated with AI assistance. The specification details have been extracted from Spark documentation and may need verification.
Comet does not currently support the Spark try_make_interval function, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.
The TryMakeInterval expression creates an interval from separate components (years, months, weeks, days, hours, minutes, seconds). Unlike MakeInterval, this expression returns NULL instead of throwing an exception when an error occurs during interval creation. It is implemented as a RuntimeReplaceable expression that delegates to MakeInterval with failOnError = false.
Supporting this expression would allow more Spark workloads to benefit from Comet's native acceleration.
Describe the potential solution
Spark Specification
Syntax:
try_make_interval([years [, months [, weeks [, days [, hours [, mins [, secs]]]]]]])
Arguments:
| Argument |
Type |
Description |
| years |
Expression |
Number of years (optional, defaults to 0) |
| months |
Expression |
Number of months (optional, defaults to 0) |
| weeks |
Expression |
Number of weeks (optional, defaults to 0) |
| days |
Expression |
Number of days (optional, defaults to 0) |
| hours |
Expression |
Number of hours (optional, defaults to 0) |
| mins |
Expression |
Number of minutes (optional, defaults to 0) |
| secs |
Expression |
Number of seconds as Decimal (optional, defaults to 0) |
Return Type: Returns INTERVAL data type or NULL if an error occurs during interval construction.
Supported Data Types:
- Numeric types: All numeric expressions are supported for input parameters
- Decimal: The seconds parameter specifically uses Decimal type with
MAX_LONG_DIGITS precision and scale of 6
- Integer literals: Default values use integer literals (0) for most parameters
Edge Cases:
- Null handling: Returns
NULL if any input parameter is NULL
- Invalid intervals: Returns
NULL instead of throwing exceptions for invalid interval combinations
- Overflow conditions: Returns
NULL when interval components exceed valid ranges
- Missing parameters: Automatically fills missing parameters with appropriate zero values (Literal(0) for most, Decimal(0) for seconds)
- Empty constructor: Intentionally not supported in try version as it would never overflow
Examples:
-- Create interval with all components
SELECT try_make_interval(1, 2, 3, 4, 5, 6, 7.5);
-- Create interval with partial components
SELECT try_make_interval(2, 3);
-- Handle invalid input gracefully (returns NULL)
SELECT try_make_interval(9999999, 9999999, 9999999);
-- Compare with make_interval behavior
SELECT
try_make_interval(1, NULL, 3) as safe_result, -- Returns NULL
make_interval(1, NULL, 3) as unsafe_result; -- Throws exception
// DataFrame API usage
import org.apache.spark.sql.functions._
// Create interval column
df.select(try_make_interval(col("years"), col("months"), col("days")))
// With literal values
df.select(try_make_interval(lit(1), lit(6), lit(0), lit(15)))
// Handle potentially invalid data
df.select(try_make_interval(col("user_years"), col("user_months")))
.filter(col("interval_col").isNotNull)
Implementation Approach
See the Comet guide on adding new expressions for detailed instructions.
- Scala Serde: Add expression handler in
spark/src/main/scala/org/apache/comet/serde/
- Register: Add to appropriate map in
QueryPlanSerde.scala
- Protobuf: Add message type in
native/proto/src/proto/expr.proto if needed
- Rust: Implement in
native/spark-expr/src/ (check if DataFusion has built-in support first)
Additional context
Difficulty: Medium
Spark Expression Class: org.apache.spark.sql.catalyst.expressions.TryMakeInterval
Related:
MakeInterval - The non-safe version that throws exceptions on errors
INTERVAL literal syntax - Direct interval creation
- Date/time arithmetic functions for interval operations
This issue was auto-generated from Spark reference documentation.
What is the problem the feature request solves?
Comet does not currently support the Spark
try_make_intervalfunction, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.The
TryMakeIntervalexpression creates an interval from separate components (years, months, weeks, days, hours, minutes, seconds). UnlikeMakeInterval, this expression returnsNULLinstead of throwing an exception when an error occurs during interval creation. It is implemented as aRuntimeReplaceableexpression that delegates toMakeIntervalwithfailOnError = false.Supporting this expression would allow more Spark workloads to benefit from Comet's native acceleration.
Describe the potential solution
Spark Specification
Syntax:
Arguments:
Return Type: Returns
INTERVALdata type orNULLif an error occurs during interval construction.Supported Data Types:
MAX_LONG_DIGITSprecision and scale of 6Edge Cases:
NULLif any input parameter isNULLNULLinstead of throwing exceptions for invalid interval combinationsNULLwhen interval components exceed valid rangesExamples:
Implementation Approach
See the Comet guide on adding new expressions for detailed instructions.
spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scalanative/proto/src/proto/expr.protoif needednative/spark-expr/src/(check if DataFusion has built-in support first)Additional context
Difficulty: Medium
Spark Expression Class:
org.apache.spark.sql.catalyst.expressions.TryMakeIntervalRelated:
MakeInterval- The non-safe version that throws exceptions on errorsINTERVALliteral syntax - Direct interval creationThis issue was auto-generated from Spark reference documentation.