Add experimental UniForm Iceberg enablement for migrated Delta tables#4855
Open
FastLee wants to merge 3 commits into
Open
Add experimental UniForm Iceberg enablement for migrated Delta tables#4855FastLee wants to merge 3 commits into
FastLee wants to merge 3 commits into
Conversation
|
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4855 +/- ##
==========================================
+ Coverage 87.79% 87.81% +0.01%
==========================================
Files 123 123
Lines 17595 17605 +10
Branches 3717 3718 +1
==========================================
+ Hits 15448 15459 +11
Misses 1458 1458
+ Partials 689 688 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
FastLee
force-pushed
the
feature/enable-uniform-iceberg
branch
3 times, most recently
from
4be25e3 to
43dd261
Compare
|
✅ 48/48 passed, 5 skipped, 6h39m44s total Running from acceptance #9037 |
The databricks CLI's 'labs install' fetches the release list and source zipball from api.github.com anonymously (it never reads GITHUB_TOKEN), so it routinely hits GitHub's 60 req/hr unauthenticated rate limit on shared runner IPs and fails with '403 Forbidden'. PR #4857's GITHUB_TOKEN env was a no-op because the CLI ignores that variable. Pre-fetch the release list and source archive with authenticated 'gh' (5000 req/hr), seed the CLI's local cache and lib dir, then install with --offline so no GitHub API calls are made. Co-authored-by: Isaac
…grated tables Adds `enable_uniform_iceberg` config option (default: false) that enables Delta UniForm Iceberg compatibility on all migrated Delta tables post-migration. Sets delta.columnMapping.mode, delta.universalFormat.enabledFormats, and delta.enableIcebergCompatV2 via ALTER TABLE SET TBLPROPERTIES. Closes #4401 Co-authored-by: Isaac
Co-authored-by: Isaac
FastLee
force-pushed
the
feature/enable-uniform-iceberg
branch
from
fecfdb5 to
f87e2ad
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
enable_uniform_icebergexperimental config flag (default:false) toWorkspaceConfigdelta.columnMapping.mode = 'name',delta.universalFormat.enabledFormats = 'iceberg', anddelta.enableIcebergCompatV2 = 'true'on all migrated Delta tables via ALTER TABLE SET TBLPROPERTIESCloses #4401
Test plan
This pull request and its description were written by Isaac.