Skip to content

[GLUTEN-10134][CI] Remove Delta known-failures fixed by cast mode preservation - #12815

Merged
philo-he merged 1 commit into
apache:mainfrom
felipepessoto:delta-known-failures-remove-fixed-cast-modes
Aug 18, 2026
Merged

philo-he merged 1 commit into
apache:mainfrom
felipepessoto:delta-known-failures-remove-fixed-cast-modes

Conversation

@felipepessoto

@felipepessoto felipepessoto commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Removes 4 entries from the Delta-on-Gluten baseline (known-failures.txt) that now pass after #12051.

#12051 ("[GLUTEN-10134][VL] Preserve store assignment cast modes") makes CastTransformer forward the per-expression EvalMode (LEGACY / ANSI / TRY) to Velox through Substrait, instead of letting Velox derive cast behaviour from the session-level spark.sql.ansi.enabled. Previously EvalMode.LEGACY and EvalMode.ANSI serialized to identical Substrait nodes, so a LEGACY cast wrongly inherited ANSI overflow semantics whenever the session flag was on.

The Delta implicit-casting tests below run with storeAssignmentPolicy: LEGACY and ansiEnabled: true, i.e. Delta builds a Cast node with evalMode = LEGACY while the session flag is true. They expect the cast to return NULL rather than throw, so they now pass and must be dropped from the baseline:

org.apache.spark.sql.delta.ImplicitMergeCastingSuite#MERGE overflow in WHEN MATCHED THEN UPDATE SET t.value = s.value sourceType: BIGINT, targetType: DECIMAL(7,2) followAnsiEnabled: false, ansiEnabled: true, storeAssignmentPolicy: LEGACY
org.apache.spark.sql.delta.ImplicitMergeCastingSuite#MERGE overflow in WHEN MATCHED THEN UPDATE SET t.value = s.value sourceType: STRING, targetType: INT followAnsiEnabled: false, ansiEnabled: true, storeAssignmentPolicy: LEGACY
org.apache.spark.sql.delta.ImplicitStreamingMergeCastingSuite#Streaming MERGE overflow sourceType: BIGINT, targetType: DECIMAL(7,2) followAnsiEnabled: false, ansiEnabled: true, storeAssignmentPolicy: LEGACY
org.apache.spark.sql.delta.ImplicitStreamingMergeCastingSuite#Streaming MERGE overflow sourceType: STRING, targetType: INT followAnsiEnabled: false, ansiEnabled: true, storeAssignmentPolicy: LEGACY

This is required, not cosmetic: the gate runs with --fail-on-fixed true, so a baseline entry that starts passing turns the Delta workflow red until it is removed.

How was this patch tested?

Evidence comes from the scheduled Delta Spark UT (Gluten) run for commit 8439151, which contains the fix (95f280f is an ancestor of the run's head SHA):

https://github.com/apache/gluten/actions/runs/32101727188

  • Shard 0 failed with NOW-PASSING (remove from baseline) for the two ImplicitStreamingMergeCastingSuite tests.
  • Shard 3 failed with NOW-PASSING (remove from baseline) for the two ImplicitMergeCastingSuite tests.
  • The delta-spark-aggregate job reports Now-passing (global) = 4 (exactly the lines removed here), alongside Regressions = 0, Stale = 0, and Skipped this run = 0.

Since the run reports no regressions and no stale entries, these 4 deletions are the only baseline change needed; the remaining 60 Implicit*MergeCastingSuite entries (other type/config combinations that still fail) are untouched.

Diff is 4 deletions and 0 additions; the file's sorted order and header are preserved.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GitHub Copilot CLI

Related issue: #10134

…servation

apache#12051 makes `CastTransformer` forward the per-expression `EvalMode`
(`LEGACY` / `ANSI` / `TRY`) to Velox through Substrait instead of letting
Velox derive cast behaviour from the session-level `spark.sql.ansi.enabled`.

As a result a `Cast` node whose own `evalMode` is `LEGACY` no longer inherits
ANSI overflow semantics, so the Delta implicit-casting tests that run with
`storeAssignmentPolicy: LEGACY` and `ansiEnabled: true` now pass and must be
removed from the baseline. The gate fails on now-passing baseline entries
(`--fail-on-fixed true`) to keep the list honest.

The four tests were reported as now-passing by the scheduled Delta Spark UT
run for commit 8439151, which includes the fix:
https://github.com/apache/gluten/actions/runs/32101727188

That run reported 0 regressions, 0 stale entries and exactly these 4
now-passing tests:

  ImplicitMergeCastingSuite
    MERGE overflow ... sourceType: BIGINT, targetType: DECIMAL(7,2)
    MERGE overflow ... sourceType: STRING, targetType: INT
  ImplicitStreamingMergeCastingSuite
    Streaming MERGE overflow sourceType: BIGINT, targetType: DECIMAL(7,2)
    Streaming MERGE overflow sourceType: STRING, targetType: INT

(all with followAnsiEnabled: false, ansiEnabled: true, storeAssignmentPolicy: LEGACY)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 18, 2026 18:48
@github-actions github-actions Bot added the INFRA label Aug 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes four Delta-on-Gluten baseline entries from known-failures.txt that are now passing after #12051 preserved per-expression cast EvalMode semantics (avoiding incorrect ANSI overflow behavior when storeAssignmentPolicy=LEGACY with spark.sql.ansi.enabled=true).

Changes:

  • Drop 2 ImplicitMergeCastingSuite baseline lines for storeAssignmentPolicy: LEGACY + ansiEnabled: true overflow cases.
  • Drop 2 ImplicitStreamingMergeCastingSuite baseline lines for the same configuration/cases.
  • Keep the remaining baseline intact (ordering and surrounding entries preserved).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@ReemaAlzaid

Copy link
Copy Markdown
Contributor

Thanks @felipepessoto! Confirmed these 4 tests are fixed by #12051 (LEGACY casts under ansiEnabled=true that previously ran as ANSI casts and threw on overflow)

cc:@philo-he

@philo-he philo-he left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@philo-he
philo-he merged commit 545dbbe into apache:main Aug 18, 2026
14 checks passed
@felipepessoto
felipepessoto deleted the delta-known-failures-remove-fixed-cast-modes branch August 18, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants