Skip to content

Native cast string to date returns NULL for years outside chrono's NaiveDate range #6842

Open

Description

@viirya

Describe the bug

Comet's native CAST(string AS DATE) returns NULL for any year outside chrono's NaiveDate range (-262143 to 262142). Spark parses these strings to valid dates. Spark's stringToDate accepts a year of up to 7 digits and only rejects a date whose epoch day overflows Int, so it accepts years up to about +/-5.88 million. This happens with CAST, TRY_CAST and to_date, and in every eval mode, including ANSI, where Comet returns NULL without raising an error.

Steps to reproduce

CREATE TABLE t(s string) USING parquet;
INSERT INTO t VALUES ('999999-01-01'), ('294248-01-01'), ('-290309-01-01'), ('1000000-01-01');
SELECT s, unix_date(try_cast(s AS DATE)) FROM t;

Spark returns 364522607, 106752347, -106752712 and 364522972. Comet returns NULL for all four rows. Collected java.sql.Date values display truncated for years past 9999, so unix_date makes the difference visible.

Expected behavior

Comet should return the same epoch day as Spark for any date whose epoch day fits in Int (-5877641-06-23 to 5881580-07-11). Outside that range, and for invalid calendar dates, Comet should return NULL in LEGACY/TRY mode and raise CAST_INVALID_INPUT in ANSI mode.

Additional context

date_parser already computes the epoch day without chrono via ymd_to_epoch_day, but then applies an extra chrono year bound. String-to-timestamp is not affected: Spark limits timestamp years to 6 digits, and Comet's timestamp parsers already match Spark at the Long micros boundaries. Native make_date had the same limitation and was fixed in #5443 (#5208).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:expressionsExpression evaluationbugSomething isn't workingpriority:mediumFunctional bugs, performance regressions, broken features

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions