You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Check the timezone label of every native expression that returns a timestamp #6849
Inside a native plan, every TimestampType value has to carry the Arrow label "UTC", and every TimestampNTZType value no label at all (see "The invariant" in docs/source/contributor-guide/timezones.md). A native expression that returns another label passes any test that only projects its result, because the scan and shuffle boundaries cast each column back to its declared type. The mistake only shows up when the result is compared with another timestamp or fed to another native expression. #6328 (timestamp_seconds) and #6330 (date_trunc) were both this bug. The timezone audit in #6335 found them, and each fix added a test for its own expression, but nothing checks the others.
What to add
A SQL file test, or one per expression, that for each native expression returning a timestamp:
compares the result with a timestamp column (= and <), and feeds it to hour and to a cast to STRING
runs in UTC, Etc/UTC and a zone with DST such as America/Los_Angeles, through ConfigMatrix
uses expect_native(<name>), because a dispatched expression returns Spark's answer from inside a Comet operator
The expressions that run natively today and return TIMESTAMP or TIMESTAMP_NTZ are timestamp_seconds, date_trunc, casts to TIMESTAMP and TIMESTAMP_NTZ, and, with allowIncompatible, from_utc_timestamp, to_utc_timestamp and convert_timezone. Expressions that run through the codegen dispatcher get their label from the dispatcher's output schema, so one dispatched case is enough to cover those.
Activity
LinSimon-901101 commentedon Oct 12, 2026
take