You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Native string to float/double and decimal casts accept strings Spark rejects #6839
The native string-to-floating-point and string-to-decimal casts accept some strings that Spark rejects. Spark returns NULL for them (or raises CAST_INVALID_INPUT under ANSI), while Comet returns a value. Found while reviewing #6826.
String to FLOAT/DOUBLE. Spark parses with Double.parseDouble / Float.parseFloat and falls back to Cast.processFloatingPointSpecialLiterals. Java only allows a d/D/f/F suffix after a decimal number, and accepts a signed NaN only in the exact case +NaN / -NaN. Comet's parse_string_to_float strips the suffix first and hands the rest to Rust's parser, which accepts nan, inf and infinity in any case, so 'NaNF', 'NaND', 'InfinityD', 'infF', '-nan' and '+nan' all cast to NaN or infinity.
String to DECIMAL. Spark parses with new java.math.BigDecimal(str), which rejects a scale (fractional digits minus exponent) outside the int range. Comet's parse_decimal_str / parse_string_to_decimal compute the scale in i32, which wraps in release builds (overflow checks are off), so '1e-2147483648', '1.0e-2147483647' and '0e-2147483648' cast to 0.00. Debug builds panic with attempt to subtract with overflow instead.
Steps to reproduce
CREATETABLEt(s string) USING parquet;
INSERT INTO t VALUES ('NaNF'), ('InfinityD'), ('-nan'), ('1e-2147483648');
SELECT s, try_cast(s AS DOUBLE), try_cast(s ASDECIMAL(10,2)) FROM t;
Expected behavior
Spark returns NULL for every cast above. Under ANSI, CAST(s AS DOUBLE) and CAST(s AS DECIMAL(10,2)) raise CAST_INVALID_INPUT.
Activity