Skip to content

Whitelisting Onelake API & Workspace PL FQDNs - #711

Merged
alamb merged 2 commits into
apache:mainfrom
SmritiAgrawal04:main
Jun 11, 2026
Merged

alamb merged 2 commits into
apache:mainfrom
SmritiAgrawal04:main

Conversation

@SmritiAgrawal04

@SmritiAgrawal04 SmritiAgrawal04 commented May 5, 2026 •

Copy link
Copy Markdown
Contributor

Rationale for this change

  1. With the release of the WS PL FQDNs with workspace private links, we need to add support for the new workspace FQDNs (“{wsid}.z{xy}.dfs.fabric.microsoft.com), so users can continue to use Delta-RS with OneLake and workspace private links.

What changes are included in this PR?

  1. Valid WS-PL FQDN egs:
  1. Invalid WS-PL FQDN egs:

Testing Details

  1. Unit Tests
  • parse_url() correctly extracts account_name, endpoint, container_name, and sets use_fabric_endpoint = true for
    WS-PL hosts
  • Validates both abfss:// and https:// schemes with WS-PL FQDN patterns like
    {workspaceid}.z{xy}.msit-dfs.fabric.microsoft.com
  1. Integration Testing
  1. Delta-RS Local Patch
  • Patched latest delta-rs to use local object_store via [patch.crates-io].
  • Successfully ran Delta-RS integration test against a real ABFSS based (as https scheme is not supported by Delta-RS) WSPL link:
    TABLE_URI = '' abfss://7c09f7b8-d05f-4a0d-9534-c3cad6e25525@7c09f7b8d05f4a0d9534c3cad6e25525.z7c.msit-blob.fabric.microsoft.com/3f49a318-25a0-4100-948d-d08229390322/Tables/dbo/wspltesttable" cargo run -p deltalake --example basic_operations --features 'azure,datafusion'
image

Are there any user-facing changes?

Yes. The documentaion is as below:

  1. https://learn.microsoft.com/en-us/fabric/security/security-workspace-level-private-links-overview#connecting-to-workspaces

SmritiAgrawal04 and others added 2 commits May 5, 2026 22:33
* Whitelisting Onelake API & Workspace PL FQDNs

* Addressing comments to whitelist api-onelake fqdns and add UTs

* Reverting the mistakenly modified UT

* Adding validation for xy in WS-PL URL & case insensitive regex

* Making regex an optional dependency

* Eliminating the use of regex for ws-pl fqdn matching

* Adding a separte method 'azure_test_workspace_private_link' for ws-pl tests

* Organising private link UTs

* Adding integration test for WSPL & rebasing with unpstream

* Whitelisting Onelake API & Workspace PL FQDNs

* Addressing comments to whitelist api-onelake fqdns and add UTs

* Reverting the mistakenly modified UT

* Adding validation for xy in WS-PL URL & case insensitive regex

* Making regex an optional dependency

* Eliminating the use of regex for ws-pl fqdn matching

* Adding a separte method 'azure_test_workspace_private_link' for ws-pl tests

* Organising private link UTs

* Fixing cargo fmt

* Fixing Clippy

* Fixing Clippy

* Fixing Clippy

* Modified Integration test to pass workspace & artifact id within URL only

* Fixing Clippy

* Adding WSPL support for ABFSS scheme as well

* Adding comments for readability

* Adding comments for readability

---------

Co-authored-by: Smriti Agrawal <smagrawal@microsoft.com>

@kevinjqliu kevinjqliu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I have been working with @SmritiAgrawal04 on this. We ran integration tests and validated this against WSPL test environment. We also wired this local change with delta-rs to validate that delta-rs can read abfss:// uri string for WSPL

@kevinjqliu

Copy link
Copy Markdown
Contributor

@tustvold could you take a look when you get a chance?

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/azure/mod.rs
}
}

#[ignore = "Used for manual testing against a real Workspace Private Link Endpoint."]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@crepererum and I were talking the other day and it might be a good idea to get access to a real azure bucket to test against -- perhaps @kevinjqliu could help facilitate this somehow (I am not sure the best way to integrate real integration tests)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants