You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Native cast string to date returns NULL for years outside chrono's NaiveDate range #6842
Comet's native CAST(string AS DATE) returns NULL for any year outside chrono's NaiveDate range (-262143 to 262142). Spark parses these strings to valid dates. Spark's stringToDate accepts a year of up to 7 digits and only rejects a date whose epoch day overflows Int, so it accepts years up to about +/-5.88 million. This happens with CAST, TRY_CAST and to_date, and in every eval mode, including ANSI, where Comet returns NULL without raising an error.
Steps to reproduce
CREATETABLEt(s string) USING parquet;
INSERT INTO t VALUES ('999999-01-01'), ('294248-01-01'), ('-290309-01-01'), ('1000000-01-01');
SELECT s, unix_date(try_cast(s ASDATE)) FROM t;
Spark returns 364522607, 106752347, -106752712 and 364522972. Comet returns NULL for all four rows. Collected java.sql.Date values display truncated for years past 9999, so unix_date makes the difference visible.
Expected behavior
Comet should return the same epoch day as Spark for any date whose epoch day fits in Int (-5877641-06-23 to 5881580-07-11). Outside that range, and for invalid calendar dates, Comet should return NULL in LEGACY/TRY mode and raise CAST_INVALID_INPUT in ANSI mode.
Additional context
date_parser already computes the epoch day without chrono via ymd_to_epoch_day, but then applies an extra chrono year bound. String-to-timestamp is not affected: Spark limits timestamp years to 6 digits, and Comet's timestamp parsers already match Spark at the Long micros boundaries. Native make_date had the same limitation and was fixed in #5443 (#5208).
Activity