Describe the bug
The native Iceberg write gate admits s3 / s3a data locations without checking whether the S3 settings in effect can be honoured by the native writer. Settings it cannot honour are dropped silently.
What reaches the native writer:
- From the Hadoop configuration, only six
fs.s3a.* settings are translated: access key, secret key, session token, endpoint, region and path-style access (NativeConfig.scala:62-68, via hadoopToIcebergS3Properties at CometIcebergNativeWrite.scala:684-687).
- FileIO properties are forwarded as-is (
CometIcebergNativeWrite.scala:594-595), and load_file_io passes on every s3. / client. key (iceberg_common.rs:39, :122). iceberg-rust ignores keys it does not know.
- Credentials otherwise come from Comet's own credential-provider bridge when one is configured, or from opendal's default chain.
Settings that are silently lost include:
- Hadoop S3A:
fs.s3a.aws.credentials.provider, fs.s3a.encryption.algorithm / fs.s3a.encryption.key (SSE-KMS, SSE-C), fs.s3a.assumed.role.*, and EMRFS settings for s3://
- S3FileIO:
client.factory, client.credentials-provider, s3.remote-signing-enabled (REST catalogs), s3.acl, s3.write.tags.*, s3.write.storage-class, access-point mappings
Two outcomes, depending on the setting:
- A write that works with iceberg-java fails with an access error, because the native writer uses different credentials.
- The write succeeds, but the objects are missing the encryption, ACL, tags or storage class the table owner configured. Nothing reports this.
The gs:// path already fails closed for this reason (#5637). S3 does not.
Steps to reproduce
Configure an S3A write that needs a non-default credentials provider or SSE-KMS through fs.s3a.encryption.*, enable spark.comet.iceberg.write.enabled, and insert into an Iceberg table on s3a://. The plan shows CometIcebergWriteExec, and the resulting objects are written without the configured encryption (or the write fails on credentials).
Expected behavior
Fail closed. The gate should decline the native write, with a reason, when any Hadoop fs.s3a.* or FileIO s3.* / client.* setting outside an allow-list of settings the native writer honours is set. Settings could then be added to the allow-list one at a time as the native writer gains support.
Additional context
Found in an audit of the native Iceberg write path before enabling it by default. Part of #5649. Related: #5643 (keep-or-lift per eligibility restriction), #5637.
Describe the bug
The native Iceberg write gate admits
s3/s3adata locations without checking whether the S3 settings in effect can be honoured by the native writer. Settings it cannot honour are dropped silently.What reaches the native writer:
fs.s3a.*settings are translated: access key, secret key, session token, endpoint, region and path-style access (NativeConfig.scala:62-68, viahadoopToIcebergS3PropertiesatCometIcebergNativeWrite.scala:684-687).CometIcebergNativeWrite.scala:594-595), andload_file_iopasses on everys3./client.key (iceberg_common.rs:39,:122). iceberg-rust ignores keys it does not know.Settings that are silently lost include:
fs.s3a.aws.credentials.provider,fs.s3a.encryption.algorithm/fs.s3a.encryption.key(SSE-KMS, SSE-C),fs.s3a.assumed.role.*, and EMRFS settings fors3://client.factory,client.credentials-provider,s3.remote-signing-enabled(REST catalogs),s3.acl,s3.write.tags.*,s3.write.storage-class, access-point mappingsTwo outcomes, depending on the setting:
The
gs://path already fails closed for this reason (#5637). S3 does not.Steps to reproduce
Configure an S3A write that needs a non-default credentials provider or SSE-KMS through
fs.s3a.encryption.*, enablespark.comet.iceberg.write.enabled, and insert into an Iceberg table ons3a://. The plan showsCometIcebergWriteExec, and the resulting objects are written without the configured encryption (or the write fails on credentials).Expected behavior
Fail closed. The gate should decline the native write, with a reason, when any Hadoop
fs.s3a.*or FileIOs3.*/client.*setting outside an allow-list of settings the native writer honours is set. Settings could then be added to the allow-list one at a time as the native writer gains support.Additional context
Found in an audit of the native Iceberg write path before enabling it by default. Part of #5649. Related: #5643 (keep-or-lift per eligibility restriction), #5637.