Skip to content

Support Spark 4.2 insert-only MERGE execution #6612

Open

Description

@unikdahal

Problem

Spark 4.2 introduces InsertOnlyMergeExec for insert-only MERGE plans. These plans can bypass the normal MergeRowsExec path, so native MergeRows support alone does not cover all MERGE shapes.

Proposed solution

Support Spark 4.2 insert-only MERGE while preserving Spark's V2 MergeSummary contract:

  • support multiple NOT MATCHED clauses through native MergeRows where applicable
  • keep InsertOnlyMergeExec responsible for its writer summary semantics
  • preserve insert-only behavior without cardinality validation
  • cover the single-clause optimized path that has no MergeRowsExec child

Add result and MergeSummary parity tests against Spark.

Related: #5318, #5122

Activity

  1. unikdahal commented on Oct 4, 2026

    @unikdahal
    ContributorAuthor

    take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions