Repository navigation

feat: execute scalar Arrow Python UDFs in native Comet - #6130

Merged

andygrove merged 18 commits into

apache:mainfrom

viirya:codex/native-arrow-python-udf

Oct 23, 2026

Merged

feat: execute scalar Arrow Python UDFs in native Comet#6130
andygrove merged 18 commits into
apache:mainfrom
viirya:codex/native-arrow-python-udf

Commits on Sep 23, 2026
- feat: execute scalar Arrow Python UDFs in native Comet
  viirya
  committed
  dc54a31
- docs: clarify scalar Arrow UDF support starts in Spark 4.1
  viirya
  committed
  27871a5
- fix: align native Arrow UDF semantics and CI coverage
  viirya
  committed
  fabcae3
- fix: avoid Arrow UDF polling spin and narrow schema support
  viirya
  committed
  e337202
- Fix embedded Python symbols and verify Arrow input types
  viirya
  committed
  6cef96b
- Document embedded Python build and deployment requirements
  viirya
  committed
  ac0f784
Commits on Sep 24, 2026
- Merge upstream main into native Arrow UDF branch
  viirya
  committed
  b8cc409
Commits on Sep 25, 2026
- Fix native Arrow UDF environment and batch semantics
  viirya
  committed
  4830cee
- Honor Arrow Python UDF byte batch limit
  viirya
  committed
  c73d8b5
Commits on Sep 28, 2026
- Skip empty Arrow UDF batches and protect plan metadata
  viirya
  committed
  67a9f77
- Merge upstream main into native Arrow UDF branch
  viirya
  committed
  de71a8e
Commits on Sep 29, 2026
- Fix native Arrow UDF subqueries, file fallback, and exchange reuse
  viirya
  committed
  b0dab2f
Commits on Oct 6, 2026
- Merge upstream main and guard Arrow UDF accumulators
  viirya
  committed
  d6588a6
Commits on Oct 9, 2026
- Address native Arrow UDF review: archives, results, logging, metrics
  viirya
  committed
  794b276
Commits on Oct 10, 2026
- Fix Arrow UDF tracebacks before Python 3.12 and serialize imports
  viirya
  committed
  9391442
Commits on Oct 12, 2026
- Start embedded Python with unbuffered stdio
  viirya
  committed
  23f08a8
Commits on Oct 18, 2026
- Merge upstream main into native Arrow UDF branch
  
  viirya
  and
  Isaac
  committed
  eb23556
Commits on Oct 20, 2026
- Stop native Arrow UDF stream after its first error
  viirya
  committed
  4345da3

Commits

Commits on Sep 23, 2026

Commits on Sep 24, 2026

Commits on Sep 25, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Oct 6, 2026

Commits on Oct 9, 2026

Commits on Oct 10, 2026

Commits on Oct 12, 2026

Commits on Oct 18, 2026

Commits on Oct 20, 2026