Skip to content

fix: revert Apache datafusion-iceberg adoption (#742) - #749

Merged
alexanderbianchi merged 1 commit into
datafusion-contrib:mainfrom
alexanderbianchi:fix/revert-742
Sep 24, 2026
Merged

alexanderbianchi merged 1 commit into
datafusion-contrib:mainfrom
alexanderbianchi:fix/revert-742

Conversation

@alexanderbianchi

Copy link
Copy Markdown
Collaborator

Summary

Revert #742 until Apache datafusion-iceberg uses an iceberg-rust revision containing FileScanTask serialization support.

#742 aligned our Iceberg dependencies with upstream's older 665c64e revision, dropping that support. Distributed scans panic in FileScanTaskWorkUnit::serialized() with Serialization not implemented for this field, and executes_with_estimated_scan_tasks then hangs. Reproduced on the merge commit.

This is an exact revert: restore the previous 4d83bc7 dependency pins and local predicate conversion implementation, without changing task-count or transport behavior.

Fixes #748.

Validation

  • cargo test --locked -p datafusion-distributed-iceberg --features integration — all 84 unit/integration tests and 1 doctest pass, including the previously hanging test.
  • cargo fmt --all -- --check
  • git diff --check HEAD^ HEAD

@alexanderbianchi
alexanderbianchi merged commit 588a07f into datafusion-contrib:main Sep 24, 2026
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deadlock in Iceberg integration test

2 participants