Skip to content

Check the timezone label of every native expression that returns a timestamp #6849

Open

Description

@andygrove

What / Why

Inside a native plan, every TimestampType value has to carry the Arrow label "UTC", and every TimestampNTZType value no label at all (see "The invariant" in docs/source/contributor-guide/timezones.md). A native expression that returns another label passes any test that only projects its result, because the scan and shuffle boundaries cast each column back to its declared type. The mistake only shows up when the result is compared with another timestamp or fed to another native expression. #6328 (timestamp_seconds) and #6330 (date_trunc) were both this bug. The timezone audit in #6335 found them, and each fix added a test for its own expression, but nothing checks the others.

What to add

A SQL file test, or one per expression, that for each native expression returning a timestamp:

  • compares the result with a timestamp column (= and <), and feeds it to hour and to a cast to STRING
  • runs in UTC, Etc/UTC and a zone with DST such as America/Los_Angeles, through ConfigMatrix
  • uses expect_native(<name>), because a dispatched expression returns Spark's answer from inside a Comet operator

The expressions that run natively today and return TIMESTAMP or TIMESTAMP_NTZ are timestamp_seconds, date_trunc, casts to TIMESTAMP and TIMESTAMP_NTZ, and, with allowIncompatible, from_utc_timestamp, to_utc_timestamp and convert_timezone. Expressions that run through the codegen dispatcher get their label from the dispatcher's output schema, so one dispatched case is enough to cover those.

Follows up on #6335.

Activity

  1. LinSimon-901101 commented on Oct 12, 2026

    @LinSimon-901101
    Contributor

    take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area:expressionsExpression evaluationenhancementNew feature or requestpriority:lowMinor issues, test failures, tooling, cosmetictestTesting related

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions