Conversation
## Which issue does this PR close? - Closes apache#2711. ## What changes are included in this PR? Reject DataFusion `InsertOp::Overwrite` and `InsertOp::Replace` before constructing an Iceberg write plan. Previously, `IcebergTableProvider::insert_into` ignored the requested insert operation while the commit executor always used `fast_append`. Unsupported overwrite or replace requests could therefore be silently executed with append semantics. This patch keeps the existing append path unchanged and returns `DataFusionError::NotImplemented` for unsupported insert operations. It preserves the existing `_insert_op` parameter name to avoid changing the generated public API snapshot. ## Are these changes tested? Yes. Added unit tests verifying that: - `InsertOp::Overwrite` returns `DataFusionError::NotImplemented`. - `InsertOp::Replace` returns `DataFusionError::NotImplemented`. - Existing append-insert behavior remains unchanged. Local verification: - `cargo test -p iceberg-datafusion test_catalog_backed_provider_rejects -- --nocapture` - `cargo fmt --all -- --check` - `git diff --check`
…ble_properties (apache#2561) ## Which issue does this PR close? N/A — no tracking issue. ## What changes are included in this PR? Only the DataFusion `INSERT INTO` path honored `write.parquet.*` table properties, and only the content-defined-chunking (CDC) keys, via a hand-rolled translation inlined in `IcebergWriteExec`. Anything writing through the writer stack directly (`DataFileWriter` → `ParquetWriterBuilder`) silently fell back to parquet-rs defaults. This PR moves that translation behind a reusable constructor: - Add `ParquetWriterBuilder::from_table_properties(&TableProperties, schema)`, the single place that maps `write.parquet.*` into `WriterProperties`. It currently translates the CDC keys (`write.parquet.content-defined-chunking.*`); other keys still fall back to parquet-rs defaults and can be added here later. - Add a chainable `with_match_mode` setter so the field match mode can be overridden — DataFusion needs name-based matching since its Arrow batches carry no field-id metadata. - Refactor the DataFusion `insert_into` writer to build via `from_table_properties`, reusing the `TableProperties` it already parses and dropping the inline CDC translation. Additive only: `new` and `new_with_match_mode` are unchanged; no breaking changes. ## Are these changes tested? - Unit tests in `parquet_writer.rs`: CDC is off by default, and CDC properties propagate through `build()` to the writer's `WriterProperties` — asserted on the getter rather than by re-reading a written file, so future `write.parquet.*` options only need an assertion on their corresponding getter. - Existing `test_insert_into*` DataFusion integration tests cover the refactored path, which is behaviorally unchanged.
…nencrypted manifest writer (apache#2666) ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - working towards apache#2034 Follow up for apache#2628 ## What changes are included in this PR? Now that we have two constructors for `ManifestWriterBuilder`, `new` and `new_from_encrypted`, it should never be the case that you want to present `key_metadata` to an unencrypted `output`. <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
Bumps [zeroize](https://github.com/RustCrypto/utils) from 1.8.2 to 1.9.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/RustCrypto/utils/commit/0b715735a660a8566ccd240bf42489fe2ed98efb"><code>0b71573</code></a> zeroize v1.9.0 (<a href="https://redirect.github.com/RustCrypto/utils/issues/1494">#1494</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/3e3f18c273ec48c99ffca514ffd8bc91e1352a9a"><code>3e3f18c</code></a> zeroize: always enable AVX-512 support (<a href="https://redirect.github.com/RustCrypto/utils/issues/1493">#1493</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/1ea42bb68f560a2909c856b20aa3c62901f6305c"><code>1ea42bb</code></a> zeroize_derive v1.5.0 (<a href="https://redirect.github.com/RustCrypto/utils/issues/1492">#1492</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/8d785d1f9ddc6f8caeef1f4744fb93647f00a56d"><code>8d785d1</code></a> zeroize: rustdoc improvements (<a href="https://redirect.github.com/RustCrypto/utils/issues/1491">#1491</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/d844f3659d74b1461f9527ac31251d9800f49e55"><code>d844f36</code></a> zeroize: incorporate README.md into rustdoc (<a href="https://redirect.github.com/RustCrypto/utils/issues/1490">#1490</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/c65c09d0a1ca7ba1c27e568712934cb6893e2297"><code>c65c09d</code></a> ctutils: use <code>reason</code> instead of comment in <code>forbid</code> attribute (<a href="https://redirect.github.com/RustCrypto/utils/issues/1489">#1489</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/cbd0963c685df025c42bed31f78c50d1bada3805"><code>cbd0963</code></a> Release block-buffer v0.12.1 (<a href="https://redirect.github.com/RustCrypto/utils/issues/1488">#1488</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/9aa541d55770f47bb1c50c1d476d167977e3de2a"><code>9aa541d</code></a> block-buffer: fix exception safety (<a href="https://redirect.github.com/RustCrypto/utils/issues/1487">#1487</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/5c7e4f9bb31af81bf766360e836b6d633b84dbff"><code>5c7e4f9</code></a> cmov v0.5.4 (<a href="https://redirect.github.com/RustCrypto/utils/issues/1485">#1485</a>)</li> <li><a href="https://github.com/RustCrypto/utils/commit/87cadbce34655ac3c78efa7290f37d942d551b2c"><code>87cadbc</code></a> cmov: fix clippy (<a href="https://redirect.github.com/RustCrypto/utils/issues/1484">#1484</a>)</li> <li>Additional commits viewable in <a href="https://github.com/RustCrypto/utils/compare/zeroize-v1.8.2...zeroize-v1.9.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ile` in `ManifestWriterBuilder` (apache#2628) ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> In apache#2568 we introduced `new_from_encrypted` but the api isn't very ergonomic because `EncryptedOutputFile` has `StandardKeyMetadata` on it already so we don't need to pass in `key_metadata` also. working towards apache#2034 ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? - Closes apache#2758. - Closes apache#2759. ## What changes are included in this PR? Add audit ignore to resolve RUSTSEC-2026-0194 RUSTSEC-2026-0195, since we can't do anything now. ## Are these changes tested? CI
Feature branches rarely need their own CI runs: the code is already tested when a pull request is opened against a release branch. If the push trigger has no branch restriction and pull_request is also configured, every push to a branch with an open PR runs the workflow twice: once for the push and once for the PR synchronisation. Always give the push trigger an explicit list of branches: this stops branches created from a release branch from inheriting its workflow runs. see https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=430408443#GitHubActionsRecommendedPractices-Restrictthepushtriggertospecificbranches ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Signed-off-by: Aurélien Pupier <apupier@ibm.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com>
…2760) ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? - Updated verify_rc.sh to verify tag against tarball on Apache dev repo <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? Manually tested against v0.10.0-rc.2, see results in the comment <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.12.28 to 0.13.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/seanmonstar/reqwest/releases">reqwest's releases</a>.</em></p> <blockquote> <h2>v0.13.3</h2> <h2>tl;dr</h2> <ul> <li>Fix CertificateRevocationList parsing of PEM values.</li> <li>Fix logging in resolver to only show host, not full URL.</li> <li>Fix hickory-dns to fallback to a default if <code>/etc/resolv.conf</code> fails.</li> <li>Fix HTTP/3 to handle <code>STOP_SENDING</code> as not an error.</li> <li>Fix HTTP/3 pool to remove timed out QUIC connections.</li> <li>Fix HTTP/3 connection establishment picking IPv4 and IPv6.</li> <li>Upgrade rustls-platform-verifier.</li> <li>(wasm) Only use wasm-bindgen on unknown-* targets.</li> </ul> <h2>What's Changed</h2> <ul> <li>Update docs.rs Features by <a href="https://github.com/JamesWiresmith"><code>@JamesWiresmith</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2961">seanmonstar/reqwest#2961</a></li> <li>fix: fallback to hickory_resolver's default config if reading /etc/resolv.conf fails by <a href="https://github.com/monosans"><code>@monosans</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2797">seanmonstar/reqwest#2797</a></li> <li>fix: remove timeout con by <a href="https://github.com/cuiweixie"><code>@cuiweixie</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2967">seanmonstar/reqwest#2967</a></li> <li>http3: handle stop_sending without error by <a href="https://github.com/anuraaga"><code>@anuraaga</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2978">seanmonstar/reqwest#2978</a></li> <li>resolve: debug log to change only host by <a href="https://github.com/lms0806"><code>@lms0806</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2992">seanmonstar/reqwest#2992</a></li> <li>Edit reference link by <a href="https://github.com/lms0806"><code>@lms0806</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2996">seanmonstar/reqwest#2996</a></li> <li>docs: more accurate about default HTTP2 window sizes by <a href="https://github.com/seanmonstar"><code>@seanmonstar</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3007">seanmonstar/reqwest#3007</a></li> <li>[HTTP/3] Optimize IPv6 fallback and enforce HTTPS scheme <a href="https://redirect.github.com/seanmonstar/reqwest/issues/2911">#2911</a> by <a href="https://github.com/lyuzichong"><code>@lyuzichong</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3006">seanmonstar/reqwest#3006</a></li> <li>Upgrade rustls-platform-verifier by <a href="https://github.com/jplatte"><code>@jplatte</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3010">seanmonstar/reqwest#3010</a></li> <li>use wasm-bindgen ecosystem only for wasm32-unknown-* target by <a href="https://github.com/Ludea"><code>@Ludea</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3000">seanmonstar/reqwest#3000</a></li> <li>fix rustls crl pem parsing by <a href="https://github.com/Threated"><code>@Threated</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3013">seanmonstar/reqwest#3013</a></li> <li>docs(retry): include ReqRep in docsrs by <a href="https://github.com/seanmonstar"><code>@seanmonstar</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3020">seanmonstar/reqwest#3020</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/JamesWiresmith"><code>@JamesWiresmith</code></a> made their first contribution in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2961">seanmonstar/reqwest#2961</a></li> <li><a href="https://github.com/monosans"><code>@monosans</code></a> made their first contribution in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2797">seanmonstar/reqwest#2797</a></li> <li><a href="https://github.com/cuiweixie"><code>@cuiweixie</code></a> made their first contribution in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2967">seanmonstar/reqwest#2967</a></li> <li><a href="https://github.com/anuraaga"><code>@anuraaga</code></a> made their first contribution in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2978">seanmonstar/reqwest#2978</a></li> <li><a href="https://github.com/lms0806"><code>@lms0806</code></a> made their first contribution in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2992">seanmonstar/reqwest#2992</a></li> <li><a href="https://github.com/lyuzichong"><code>@lyuzichong</code></a> made their first contribution in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3006">seanmonstar/reqwest#3006</a></li> <li><a href="https://github.com/Ludea"><code>@Ludea</code></a> made their first contribution in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/3000">seanmonstar/reqwest#3000</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/seanmonstar/reqwest/compare/v0.13.2...v0.13.3">https://github.com/seanmonstar/reqwest/compare/v0.13.2...v0.13.3</a></p> <h2>v0.13.2</h2> <h2>tl;dr</h2> <ul> <li>Fix HTTP/2 and native-tls ALPN feature combinations.</li> <li>Fix HTTP/3 to send h3 ALPN.</li> <li>(wasm) fix <code>RequestBuilder::json()</code> from override previously set content-type.</li> </ul> <h2>What's Changed</h2> <ul> <li>chore(deps): bump actions/checkout from 5 to 6 by <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2921">seanmonstar/reqwest#2921</a></li> <li>Update readme for 0.13 by <a href="https://github.com/VojtaStanek"><code>@VojtaStanek</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2926">seanmonstar/reqwest#2926</a></li> <li>fix http2 feature is not enabled for "native-tls" by <a href="https://github.com/fox0"><code>@fox0</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2927">seanmonstar/reqwest#2927</a></li> <li>chore(deps): remove unused webpki-roots and rustls-native-certs by <a href="https://github.com/seanmonstar"><code>@seanmonstar</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2932">seanmonstar/reqwest#2932</a></li> <li>docs: native-tls-alpn has changed to native-tls-no-alpn by <a href="https://github.com/seanmonstar"><code>@seanmonstar</code></a> in <a href="https://redirect.github.com/seanmonstar/reqwest/pull/2940">seanmonstar/reqwest#2940</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md">reqwest's changelog</a>.</em></p> <blockquote> <h2>v0.13.3</h2> <ul> <li>Fix CertificateRevocationList parsing of PEM values.</li> <li>Fix logging in resolver to only show host, not full URL.</li> <li>Fix hickory-dns to fallback to a default if <code>/etc/resolv.conf</code> fails.</li> <li>Fix HTTP/3 to handle <code>STOP_SENDING</code> as not an error.</li> <li>Fix HTTP/3 pool to remove timed out QUIC connections.</li> <li>Fix HTTP/3 connection establishment picking IPv4 and IPv6.</li> <li>Upgrade rustls-platform-verifier.</li> <li>(wasm) Only use wasm-bindgen on unknown-* targets.</li> </ul> <h2>v0.13.2</h2> <ul> <li>Fix HTTP/2 and native-tls ALPN feature combinations.</li> <li>Fix HTTP/3 to send h3 ALPN.</li> <li>(wasm) fix <code>RequestBuilder::json()</code> from override previously set content-type.</li> </ul> <h2>v0.13.1</h2> <ul> <li>Fixes compiling with rustls on Android targets.</li> </ul> <h1>v0.13.0</h1> <ul> <li><strong>Breaking changes</strong>: <ul> <li><code>rustls</code> is now the default TLS backend, instead of <code>native-tls</code>.</li> <li><code>rustls</code> crypto provider defaults to aws-lc instead of <em>ring</em>. (<code>rustls-no-provider</code> exists if you want a different crypto provider)</li> <li><code>rustls-tls</code> has been renamed to <code>rustls</code>.</li> <li>rustls roots features removed, <code>rustls-platform-verifier</code> is used by default. <ul> <li>To use different roots, call <code>tls_certs_only(your_roots)</code>.</li> </ul> </li> <li><code>native-tls</code> now includes ALPN. To disable, use <code>native-tls-no-alpn</code>.</li> <li><code>query</code> and <code>form</code> are now crate features, disabled by default.</li> <li>Long-deprecated methods and crate features have been removed (such as <code>trust-dns</code>, which was renamed <code>hickory-dns</code> a while ago).</li> </ul> </li> <li>Many TLS-related methods renamed to improve autocompletion and discovery, but previous name left in place with a "soft" deprecation. (just documented, no warnings) <ul> <li>For example, prefer <code>tls_backend_rustls()</code> over <code>use_rustls_tls()</code>.</li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/seanmonstar/reqwest/commit/a9a88c4ee00a61b801f4f8e8cb643cdfb9a05b2b"><code>a9a88c4</code></a> v0.13.3</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/f3f6d9df166f447657d502fbaa9590e108a02d4b"><code>f3f6d9d</code></a> docs(retry): include ReqRep in docsrs (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/3020">#3020</a>)</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/5f9c231502d827bdd19864277187b133bb746f2f"><code>5f9c231</code></a> fix rustls CRL PEM parsing (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/3013">#3013</a>)</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/11d835dcad9171f614343c714377f0fcc6638205"><code>11d835d</code></a> use wasm-bindgen ecosystem only for wasm32-unknown-* target (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/3000">#3000</a>)</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/1f72916f5cdc30f6cb6c63038c89063795294d50"><code>1f72916</code></a> Upgrade rustls-platform-verifier (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/3010">#3010</a>)</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/5d5bf355744b181d31533501133ad9fbf99e8849"><code>5d5bf35</code></a> [HTTP/3] Optimize IPv6 fallback and enforce HTTPS scheme <a href="https://redirect.github.com/seanmonstar/reqwest/issues/2911">#2911</a> (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/3006">#3006</a>)</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/93dc1b2dc4b3649d9b79b563cf4d3b41448fda0d"><code>93dc1b2</code></a> docs: more accurate about default HTTP2 window sizes (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/3007">#3007</a>)</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/c5e50f004de3ac0914369a08f3e7fd33c3ebc17e"><code>c5e50f0</code></a> docs: update outdated link in comments</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/b25611f7c976651e8c156809f781b939d1ef2b52"><code>b25611f</code></a> resolve: debug log to change only host (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/2992">#2992</a>)</li> <li><a href="https://github.com/seanmonstar/reqwest/commit/ca1f479ab373c074e5fab5b35736de0c9cc46732"><code>ca1f479</code></a> http3: handle stop_sending without error (<a href="https://redirect.github.com/seanmonstar/reqwest/issues/2978">#2978</a>)</li> <li>Additional commits viewable in <a href="https://github.com/seanmonstar/reqwest/compare/v0.12.28...v0.13.3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [enum-ordinalize](https://github.com/magiclen/enum-ordinalize) from 4.3.2 to 4.4.1. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/magiclen/enum-ordinalize/commit/7cccbc89fd2be1e1c27e7ba77c1aa58389557040"><code>7cccbc8</code></a> fix min-dependencies</li> <li><a href="https://github.com/magiclen/enum-ordinalize/commit/fcb3109863bd58c60ad46cdaecbc71d82bccecdd"><code>fcb3109</code></a> improve tests</li> <li><a href="https://github.com/magiclen/enum-ordinalize/commit/6e439f87e8c5118bf1a84556f7c783c119c546af"><code>6e439f8</code></a> fix test warnings</li> <li><a href="https://github.com/magiclen/enum-ordinalize/commit/970a6901744a1e95bb0cb5a3442056829cabb273"><code>970a690</code></a> improve code</li> <li><a href="https://github.com/magiclen/enum-ordinalize/commit/9185e0265ac715725b5a7b724523d6bfb4458867"><code>9185e02</code></a> use Rust 2024</li> <li><a href="https://github.com/magiclen/enum-ordinalize/commit/18f0caf395f5b06d364e1ee316fcd6e2d99a272b"><code>18f0caf</code></a> bump actions/checkout from 5 to 6</li> <li>See full diff in <a href="https://github.com/magiclen/enum-ordinalize/compare/v4.3.2...v4.4.1">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.2.0 to 6.3.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/setup-python/releases">actions/setup-python's releases</a>.</em></p> <blockquote> <h2>v6.3.0</h2> <h2>What's Changed</h2> <h3>Enhancement</h3> <ul> <li>Add RHEL support and include Linux distro in cache keys by <a href="https://github.com/priyagupta108"><code>@priyagupta108</code></a> in <a href="https://redirect.github.com/actions/setup-python/pull/1323">actions/setup-python#1323</a></li> <li>Fix pip cache error handling on Windows by <a href="https://github.com/priyagupta108"><code>@priyagupta108</code></a> in <a href="https://redirect.github.com/actions/setup-python/pull/1040">actions/setup-python#1040</a></li> </ul> <h3>Dependency update</h3> <ul> <li>Upgrade minimatch from 3.1.2 to 3.1.5 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/setup-python/pull/1281">actions/setup-python#1281</a></li> <li>Upgrade actions dependencies by <a href="https://github.com/gowridurgad"><code>@gowridurgad</code></a> with <a href="https://github.com/Copilot"><code>@Copilot</code></a> in <a href="https://redirect.github.com/actions/setup-python/pull/1303">actions/setup-python#1303</a></li> <li>Upgrade <code>@actions/cache</code> to 5.1.0, log cache write denied by <a href="https://github.com/jasongin"><code>@jasongin</code></a> in <a href="https://redirect.github.com/actions/setup-python/pull/1324">actions/setup-python#1324</a></li> <li>Upgrade dependency versions and test workflow configuration by <a href="https://github.com/HarithaVattikuti"><code>@HarithaVattikuti</code></a> in <a href="https://redirect.github.com/actions/setup-python/pull/1322">actions/setup-python#1322</a></li> </ul> <h3>Documentation</h3> <ul> <li>Update advanced-usage.md by <a href="https://github.com/Dunky-Z"><code>@Dunky-Z</code></a> in <a href="https://redirect.github.com/actions/setup-python/pull/811">actions/setup-python#811</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/gowridurgad"><code>@gowridurgad</code></a> with <a href="https://github.com/Copilot"><code>@Copilot</code></a> made their first contribution in <a href="https://redirect.github.com/actions/setup-python/pull/1303">actions/setup-python#1303</a></li> <li><a href="https://github.com/jasongin"><code>@jasongin</code></a> made their first contribution in <a href="https://redirect.github.com/actions/setup-python/pull/1324">actions/setup-python#1324</a></li> <li><a href="https://github.com/Dunky-Z"><code>@Dunky-Z</code></a> made their first contribution in <a href="https://redirect.github.com/actions/setup-python/pull/811">actions/setup-python#811</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/setup-python/compare/v6...v6.3.0">https://github.com/actions/setup-python/compare/v6...v6.3.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/actions/setup-python/commit/ece7cb06caefa5fff74198d8649806c4678c61a1"><code>ece7cb0</code></a> Fix pip cache error handling on Windows. (<a href="https://redirect.github.com/actions/setup-python/issues/1040">#1040</a>)</li> <li><a href="https://github.com/actions/setup-python/commit/1d18d7af5f767c1259ede05a0a5bcc30f3dcf1cf"><code>1d18d7a</code></a> Update advanced-usage.md (<a href="https://redirect.github.com/actions/setup-python/issues/811">#811</a>)</li> <li><a href="https://github.com/actions/setup-python/commit/d2b357a6a3a3687dd6781a416c0d24fcfd68660e"><code>d2b357a</code></a> Update dependency versions and test workflow configuration (<a href="https://redirect.github.com/actions/setup-python/issues/1322">#1322</a>)</li> <li><a href="https://github.com/actions/setup-python/commit/8f639b1e75c1048640734b2bb46e22cecf136982"><code>8f639b1</code></a> Merge pull request <a href="https://redirect.github.com/actions/setup-python/issues/1324">#1324</a> from jasongin/update-actions-cache-5.1.0</li> <li><a href="https://github.com/actions/setup-python/commit/6731c2ba87f530c26324d128c8fdd53499a4d4b0"><code>6731c2b</code></a> Resolve high-severity audit issues</li> <li><a href="https://github.com/actions/setup-python/commit/0cb1a84326b90186fcd211036c65b42819794c87"><code>0cb1a84</code></a> Add RHEL support and include Linux distro in cache keys (<a href="https://redirect.github.com/actions/setup-python/issues/1323">#1323</a>)</li> <li><a href="https://github.com/actions/setup-python/commit/dc6eab6194394e0119523369788b507096f923e2"><code>dc6eab6</code></a> Update dist</li> <li><a href="https://github.com/actions/setup-python/commit/6f4b74bfa2f520a380a620de3615c0dac427f4d3"><code>6f4b74b</code></a> Strict equality</li> <li><a href="https://github.com/actions/setup-python/commit/fa8bde1a9cc6347d06948d66bcd68c598b79eaea"><code>fa8bde1</code></a> Bump <code>@actions/cache</code> to 5.1.0, log cache write denied</li> <li><a href="https://github.com/actions/setup-python/commit/c8813ba1bc76ebf779b911ad8ffccbf2e449cb48"><code>c8813ba</code></a> Upgrade <a href="https://github.com/actions"><code>@actions</code></a> dependencies and update licenses (<a href="https://redirect.github.com/actions/setup-python/issues/1303">#1303</a>)</li> <li>Additional commits viewable in <a href="https://github.com/actions/setup-python/compare/a309ff8b426b58ec0e2a45f0f869d46889d02405...ece7cb06caefa5fff74198d8649806c4678c61a1">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…e#2767) Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.82.2 to 2.82.5. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/taiki-e/install-action/releases">taiki-e/install-action's releases</a>.</em></p> <blockquote> <h2>2.82.5</h2> <ul> <li> <p>Update <code>wasmtime@latest</code> to 46.0.1.</p> </li> <li> <p>Update <code>wasm-bindgen@latest</code> to 0.2.126.</p> </li> <li> <p>Update <code>vacuum@latest</code> to 0.29.6.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.6.14.</p> </li> <li> <p>Update <code>cargo-rdme@latest</code> to 2.1.0.</p> </li> </ul> <h2>2.82.4</h2> <ul> <li> <p>Update <code>uv@latest</code> to 0.11.24.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.6.13.</p> </li> <li> <p>Update <code>just@latest</code> to 1.54.0.</p> </li> <li> <p>Update <code>biome@latest</code> to 2.5.1.</p> </li> </ul> <h2>2.82.3</h2> <ul> <li> <p>Update <code>zizmor@latest</code> to 1.26.1.</p> </li> <li> <p>Update <code>wasmtime@latest</code> to 46.0.0.</p> </li> <li> <p>Update <code>tombi@latest</code> to 1.1.5.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.6.12.</p> </li> <li> <p>Update <code>kingfisher@latest</code> to 1.104.0.</p> </li> <li> <p>Update <code>cargo-tarpaulin@latest</code> to 0.35.5.</p> </li> <li> <p>Update <code>cargo-nextest@latest</code> to 0.9.138.</p> </li> <li> <p>Update <code>cargo-crap@latest</code> to 0.3.0.</p> </li> <li> <p>Update <code>cargo-binstall@latest</code> to 1.20.1.</p> </li> <li> <p>Update <code>cargo-rdme@latest</code> to 2.0.1.</p> </li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md">taiki-e/install-action's changelog</a>.</em></p> <blockquote> <h1>Changelog</h1> <p>All notable changes to this project will be documented in this file.</p> <p>This project adheres to <a href="https://semver.org">Semantic Versioning</a>.</p> <!-- raw HTML omitted --> <h2>[Unreleased]</h2> <h2>[2.82.8] - 2026-07-03</h2> <ul> <li> <p>Update <code>vacuum@latest</code> to 0.29.8.</p> </li> <li> <p>Update <code>uv@latest</code> to 0.11.26.</p> </li> <li> <p>Update <code>typos@latest</code> to 1.48.0.</p> </li> <li> <p>Update <code>trivy@latest</code> to 0.72.0.</p> </li> <li> <p>Update <code>tombi@latest</code> to 1.1.7.</p> </li> <li> <p>Update <code>prek@latest</code> to 0.4.6.</p> </li> <li> <p>Update <code>mise@latest</code> to 2026.7.0.</p> </li> <li> <p>Update <code>just@latest</code> to 1.55.1.</p> </li> <li> <p>Update <code>biome@latest</code> to 2.5.2.</p> </li> </ul> <h2>[2.82.7] - 2026-06-30</h2> <ul> <li> <p>Update <code>tombi@latest</code> to 1.1.6.</p> </li> <li> <p>Update <code>kingfisher@latest</code> to 1.105.0.</p> </li> <li> <p>Update <code>gungraun-runner@latest</code> to 0.19.3.</p> </li> <li> <p>Update <code>editorconfig-checker@latest</code> to 3.8.0.</p> </li> <li> <p>Update <code>dprint@latest</code> to 0.55.1.</p> </li> <li> <p>Update <code>cargo-tarpaulin@latest</code> to 0.36.0.</p> </li> </ul> <h2>[2.82.6] - 2026-06-29</h2> <ul> <li>Update <code>vacuum@latest</code> to 0.29.7.</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/taiki-e/install-action/commit/bffeee26d4db9be238a4ea78d8826604ebcb594d"><code>bffeee2</code></a> Release 2.82.5</li> <li><a href="https://github.com/taiki-e/install-action/commit/5bafa69e5add729bd5cc9c0646ed12e1af612c99"><code>5bafa69</code></a> ci: Test GitHub-hosted ubuntu 26.04 runners</li> <li><a href="https://github.com/taiki-e/install-action/commit/e7b7497213f2263bcf738032dcc69fd4c14b1813"><code>e7b7497</code></a> Update wasmtime manifest</li> <li><a href="https://github.com/taiki-e/install-action/commit/e9838983e85b61bb8ceb03c66a676d5a37507462"><code>e983898</code></a> Update <code>wasmtime@latest</code> to 46.0.1</li> <li><a href="https://github.com/taiki-e/install-action/commit/0cd66872025ce95e85d8831011a3d60693cb5b06"><code>0cd6687</code></a> Update <code>wasm-bindgen@latest</code> to 0.2.126</li> <li><a href="https://github.com/taiki-e/install-action/commit/151275d7b873bc8f076b2e0addaaf908740405ca"><code>151275d</code></a> Update <code>vacuum@latest</code> to 0.29.6</li> <li><a href="https://github.com/taiki-e/install-action/commit/1a104bfb141e3d8faadbdc6c3306789dd461f1c9"><code>1a104bf</code></a> Update syft manifest</li> <li><a href="https://github.com/taiki-e/install-action/commit/edab3a363f10f8d5af8efc8633ad982b899bfbde"><code>edab3a3</code></a> Update <code>mise@latest</code> to 2026.6.14</li> <li><a href="https://github.com/taiki-e/install-action/commit/46c1ccf9e1e2e709f19c2846fb39337b5045c0f6"><code>46c1ccf</code></a> Update <code>cargo-rdme@latest</code> to 2.1.0</li> <li><a href="https://github.com/taiki-e/install-action/commit/682e7d9e49c5e653d371fc6adbda67653461378a"><code>682e7d9</code></a> Release 2.82.4</li> <li>Additional commits viewable in <a href="https://github.com/taiki-e/install-action/compare/9e1e5806d4a4822de933115878265be9aaa786d9...bffeee26d4db9be238a4ea78d8826604ebcb594d">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ngs/python (apache#2766) Bumps [huggingface-hub](https://github.com/huggingface/huggingface_hub) from 1.20.1 to 1.21.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/huggingface/huggingface_hub/releases">huggingface-hub's releases</a>.</em></p> <blockquote> <h2>[v1.21.0] Jobs filtering & pagination</h2> <h2>📊 Jobs listing revamped: filter, paginate, and <code>ls</code> instead of <code>ps</code></h2> <p>The Jobs listing API and CLI have been overhauled with server-side filtering, proper pagination, and a CLI rename that aligns with the rest of <code>hf</code>. <code>list_jobs()</code> now accepts <code>status</code> and <code>labels</code> parameters that push filtering to the server, and returns a lazy iterator (matching <code>list_models</code>, <code>list_datasets</code>, etc.) so large result sets are fetched page by page. On the CLI side, <code>hf jobs ps</code> has been renamed to <code>hf jobs ls</code> for consistency with <code>hf repos ls</code>, <code>hf models ls</code>, and friends — <code>ps</code> and <code>list</code> still work as aliases.</p> <p>⚠️ <strong>Breaking changes:</strong></p> <ul> <li><code>list_jobs()</code> now returns an <code>Iterable[JobInfo]</code> instead of <code>list[JobInfo]</code>. If you indexed the result (<code>jobs[0]</code>), wrap it with <code>list(...)</code>.</li> <li><code>-f</code>/<code>--filter</code> in <code>hf jobs ls</code> is deprecated. Use <code>--status</code> and <code>--label</code> instead. Glob patterns (<code>data-*</code>), negation (<code>key!=value</code>), and filtering by <code>id</code>/<code>image</code>/<code>command</code> are no longer supported.</li> </ul> <pre lang="python"><code>from huggingface_hub import list_jobs <h1>Filter by status and labels</h1> <p>list_jobs(status=["RUNNING", "SCHEDULING"], labels={"env": "prod"})</p> <h1>Iterate lazily</h1> <p>for job in list_jobs(): print(job.id)</p> <h1>Materialize all results</h1> <p>all_jobs = list(list_jobs()) </code></pre></p> <pre lang="bash"><code># Filter by status and labels hf jobs ls --status running,scheduling --label env=prod --label team=ml # Paginate with --limit hf jobs ls -a --limit 500 hf jobs ls -a --limit 0 # no limit </code></pre> <ul> <li>[Jobs] Add stage/label filtering to list_jobs API and CLI by <a href="https://github.com/Wauplin"><code>@Wauplin</code></a> in <a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4395">#4395</a></li> <li>[Jobs] Paginate list_jobs and add --limit to <code>hf jobs ps</code> by <a href="https://github.com/Wauplin"><code>@Wauplin</code></a> in <a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4403">#4403</a></li> <li>[CLI] [Jobs] Rename job listing to 'hf jobs ls' (and keep 'hf jobs ps' alias) by <a href="https://github.com/Wauplin"><code>@Wauplin</code></a> in <a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4409">#4409</a></li> </ul> <p>📚 <strong>Documentation:</strong> <a href="https://huggingface.co/docs/huggingface_hub/main/en/guides/cli">CLI guide</a>, <a href="https://huggingface.co/docs/huggingface_hub/main/en/guides/jobs">Jobs guide</a></p> <h2>🐛 Fix circular import on <code>from huggingface_hub import login</code></h2> <p>A regression introduced in v1.20.0 caused <code>from huggingface_hub import login</code> to raise an <code>ImportError</code> on a fresh interpreter, due to a circular dependency between <code>_oauth_device</code> and <code>utils._http</code>. The fix moves <code>_oauth_device.py</code> into the <code>utils</code> layer so all imports resolve downward, eliminating the cycle. No lazy imports or workarounds required.</p> <ul> <li>Fix circular import on <code>from huggingface_hub import login</code> by <a href="https://github.com/hanouticelina"><code>@hanouticelina</code></a> in <a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4385">#4385</a></li> </ul> <h2>🔧 Other QoL Improvements</h2> <ul> <li>Retry requests on httpx.RemoteProtocolError by <a href="https://github.com/hanouticelina"><code>@hanouticelina</code></a> in <a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4398">#4398</a></li> </ul> <h2>📖 Documentation</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/huggingface/huggingface_hub/commit/aea9b9de1284f54862df99820f963d6030803860"><code>aea9b9d</code></a> Release: v1.21.0</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/2e2ed568a0d8e3841d2fe9d192f17cdb30ccf129"><code>2e2ed56</code></a> Release: v1.21.0.rc0</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/4bd4c906cac546643ca5fe0ca08e301ce7a98b62"><code>4bd4c90</code></a> [CLI] [Jobs] Rename job listing to 'hf jobs ls' (and keep 'hf jobs ps' alias)...</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/fa3eb459eb3fdf15347930d7dc70f0a7e413d6e5"><code>fa3eb45</code></a> [Jobs] Paginate list_jobs and add --limit to <code>hf jobs ps</code> (<a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4403">#4403</a>)</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/ecfd4c5e88acfa885fc9146bf5df61d4de7b46a9"><code>ecfd4c5</code></a> [CLI] Fix ty/mypy errors with click 8.4.2 (<a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4408">#4408</a>)</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/341469f1f87fb9932410fde9bdb81ac269a64727"><code>341469f</code></a> Update hardware flavor enums (automated commit) (<a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4406">#4406</a>)</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/2fbcdfeec8a1bf689623ecb87afc7d5db4cd23bb"><code>2fbcdfe</code></a> [i18n-HI] Add Hindi translation for guides overview (<a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4405">#4405</a>)</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/5bc6bfa7df18752f1adfd649816be315937cfa6f"><code>5bc6bfa</code></a> [Jobs] Add stage/label filtering to list_jobs API and CLI (<a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4395">#4395</a>)</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/d991f099f53c7439edbf9432c5137d2069199e5a"><code>d991f09</code></a> Fix IndexError in filter_repo_objects on an empty pattern (<a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4402">#4402</a>)</li> <li><a href="https://github.com/huggingface/huggingface_hub/commit/0df783fb8f3d1a37b00255007e9af78e4e9dca8f"><code>0df783f</code></a> Retry requests on httpx.RemoteProtocolError (<a href="https://redirect.github.com/huggingface/huggingface_hub/issues/4398">#4398</a>)</li> <li>Additional commits viewable in <a href="https://github.com/huggingface/huggingface_hub/compare/v1.20.1...v1.21.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Which issue does this PR close? - None. Implements the `check` job split from discussion apache#2753. ## What changes are included in this PR? Splits the old `check` job (ubuntu + macos) into two ubuntu jobs: - `lint`: the non-compiling checks (license header, toml format, cargo fmt, Cargo.lock, cargo-machete, typos). The standalone Typos workflow is folded in, and taplo and cargo-machete come as prebuilt binaries. - `clippy`: `make check-clippy` only. As suggested in apache#2753, the heavier jobs now have `needs: lint`, so a fmt or typo mistake fails fast without spinning up builds. The tradeoff is that a lint failure blocks all downstream jobs. This also removes the macOS clippy run. Nothing in the repo is gated on `target_os`, so clippy output is identical across platforms, and macOS compile coverage stays in the build jobs. The macOS `check` job was CI's slowest at 10 to 12 minutes. ## Are these changes tested? CI-only change, exercised by this PR's own CI run. Co-authored-by: Abanoub Doss <abanoub.doss@gmail.com>
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Fix the asf yml pin error in apache#2729 ## Are these changes tested? ci. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
## Summary - Reject negative row positions while parsing positional delete files. - Return `ErrorKind::DataInvalid` instead of casting negative `i64` values to huge `u64` positions. - Add a focused regression test for `pos = -1`. ## Root cause The positional-delete parser reads positions from an `Int64Array`, but inserted each value with `pos as u64`. That allowed malformed negative positions to wrap into very large row offsets instead of failing validation. ## Tests - `cargo fmt --check` - `CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse cargo test -p iceberg test_parse_positional_deletes_rejects_negative_positions --locked` - `CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse cargo test -p iceberg arrow::caching_delete_file_loader::tests --locked` --------- Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com>
## What changes are included in this PR? Update changelog and dependency files for 0.10.0 release (RC3). 0.10.0 tracking: apache#2527 ## Are these changes tested? N/A
…nifest_file_list (apache#2596) ## Which issue does this PR close? - Closes apache#2529. ## What changes are included in this PR? Renames the trait method and adds rustdoc. The change is pretty small, although risks impacting open PRs. I would recommend to either merge now or park for a while. Originally suggested as the naming does not indicate what the purpose of the method is. ## Are these changes tested? N/A
## Which issue does this PR close? DataFusion to 54.0.0 was recently released, we should try and track latests releases. <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Closes #. ## What changes are included in this PR? [ExecutionPlan](apache/datafusion#21263) [TableProvider, SchemaProvider, CatalogProvider, CatalogProviderList](apache/datafusion#21346) and [PhysicalExpr](apache/datafusion#21573) have all had their `as_any` removed from the trait. That's really the only change needed here. [Upgrade guide documents this](https://datafusion.apache.org/library-user-guide/upgrading/54.0.0.html#remove-as-any-from-physicalexpr-scalarudfimpl-aggregateudfimpl-windowudfimpl-executionplan-tableprovider-schemaprovider-catalogprovider-catalogproviderlist-tablesource-filesource-fileformat-fileformatfactory-datasource-and-datasink) <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
…he#2781) ## Which issue does this PR close? - Closes apache#2780. ## What changes are included in this PR? Equality deletes are applied by building a predicate that keeps rows not matched by any delete row. For each equality column the keep term was `col != v`, produced by negating the equality match. Applied as a row filter, `null != v` evaluates to null, so a data row whose equality column is null was dropped. Per the spec ([Equality Delete Files](https://iceberg.apache.org/spec/#equality-delete-files)), a null value matches only a null delete value, so such rows should be kept. This builds the keep predicate directly in `parse_equality_deletes_record_batch_stream`: `col IS NULL OR col != v` for a non-null delete value, and `col IS NOT NULL` for a null delete value. Rows with a null value in an equality column are no longer deleted. ## Are these changes tested? Yes, unit tests in `caching_delete_file_loader` covering the predicate construction: - `test_equality_delete_predicate_preserves_null_rows`: a non-null delete value on a nullable column produces `(col IS NULL) OR (col != v)`. - `test_equality_delete_predicate_matches_null_delete_value`: a null delete value produces `col IS NOT NULL`. - `test_equality_delete_predicate_multiple_columns`: per-column keep terms are OR-ed. - `test_equality_delete_predicate_multiple_delete_rows`: per-row keep predicates are AND-ed. - `test_delete_file_loader_parse_equality_deletes` (existing, updated): the combined case with multiple columns, a required column, and null delete values.
…nfig (apache#2692) ## Which issue does this PR close? Related to apache#1690 (server-side scan planning), which relies on endpoint negotiation. This PR lands the negotiation piece on its own so it can be reviewed independently. ## What changes are included in this PR? The REST spec lets a server advertise the routes it supports in the `endpoints` field of its `GET /v1/config` response, so clients can negotiate optional capabilities instead of assuming every server implements every operation. The Rust client currently ignores this field. This mirrors the capability negotiation the Java client gained in apache/iceberg#10929 (the `endpoints` field in the config response, the `Endpoint` type, and the default-endpoint fallback when a server omits the list). This PR: - Adds an `Endpoint` type (HTTP method + path template) parsed from the `"<method> <path>"` wire form via `FromStr` — which validates the single-space shape and the HTTP method — with serde delegating to it. The method is stored as a typed `http::Method` (kept out of the public API; `method()` returns `&str`). - Parses the config response's `endpoints` into `Option<Vec<Endpoint>>` so an absent field and an explicit empty list are modelled distinctly, and stores the negotiated set on the catalog's runtime context. - Resolves the set following the spec's optional-field semantics: an absent field → a standard base set of namespace/table operations is assumed (so an older server still resolves its core operations as supported); a present list — even an empty one — is used verbatim. Optional endpoints outside the base set stay unsupported unless explicitly advertised. - Exposes `RestCatalog::supports_endpoint(&Endpoint)` to query the negotiated set. No existing behaviour changes for callers that don't consult `supports_endpoint`. ## Are these changes tested? Yes: - Unit tests for `Endpoint`: `FromStr`/serde round-trip, rejection of malformed input (no / extra / leading / trailing space, empty), HTTP-method normalization, and the default endpoint set. - `mockito` catalog tests: a server that advertises `endpoints` (asserting `supports_endpoint` is true/false for listed/unlisted routes), a server that omits the field (base operations resolve as supported), and a server that sends an explicit empty list (nothing is supported).
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? `Datum::to()` had no conversion between `float` and `double`, so binding a predicate whose literal type differed from the column type failed with `Can't convert datum from double type to float type`. This is common because engines often carry a floating point literal as a double even when the column is a float. This PR adds the two missing arms: - `Double -> Float`: narrows through a new `f64_to_f32` helper that returns `AboveMax` / `BelowMin` when the value is outside the finite float range, matching the existing integer narrowing helpers (`i64_to_i32`, `i128_to_i64`). A plain `as f32` cast would saturate to `+/-inf` and change comparison results, so the sentinels let predicate binding resolve the comparison correctly. - `Float -> Double`: widens losslessly. ## Are these changes tested? Yes. Four unit tests in `spec/values/tests.rs` cover the in range narrowing, the above max and below min cases, and the widening. The `AboveMax` / `BelowMin` handling is already exercised by the existing predicate binding tests in `expr::predicate`. --------- Co-authored-by: emkornfield <emkornfield@gmail.com> Co-authored-by: blackmwk <liurenjie1024@outlook.com>
…#2650) ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Working towards: apache#2034 ## What changes are included in this PR? Adds `KmsClientFactory` to the catalog builder to allow a catalog to build one KMS client per catalog, this mirrors [Java's implementation ](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/encryption/EncryptionUtil.java#L47). Note in rust we don't have reflection so using Java's `encryption.kms-impl` isn't an option for us. This is a very similar pattern to what we do for the `StorageFactory` so I believe the pattern is idiomatic for us. <!-- Provide a summary of the modifications in this PR. List the main changes such as new features, bug fixes, refactoring, or any other updates. --> ## Are these changes tested? <!-- Specify what test covers (unit test, integration test, etc.). If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? -->
## Which issue does this PR close? - None. CI maintenance for the Public API job. ## What changes are included in this PR? The job installs cargo-public-api twice per run. `taiki-e/install-action` has no prebuilt binary for it, so that step falls back to cargo-binstall and compiles the latest release from source (currently v0.52.0, ~76s). `make check-public-api` then compiles the pinned v0.51.0 from source again (~72s) because the installed version doesn't match the pin. And since the binary on disk never matches the pin at cache-save time, rust-cache never shortcuts either install. This deletes the install-action step. The Makefile's `cargo install --locked cargo-public-api@0.51.0` becomes the only install; on warm runs rust-cache restores the pinned binary and the install is a no-op. 0.51.0 is the version the checked-in `public-api.txt` files were generated with, so the check itself is unchanged. The Makefile also stops computing `PUBLIC_API_CRATES` with `$(shell cargo metadata ...)` at parse time, which made every `make` invocation pay a cargo metadata call. The list is now evaluated inside the two recipes that use it, with identical command and output. Warm Public API job time on a fork: 4:03 to 1:36. ## Are these changes tested? - Verified on a fork: warm runs skip the install and run the same checks. - `make check-public-api` and `make generate-public-api` expand to the same commands as before. Co-authored-by: Abanoub Doss <abanoub.doss@gmail.com>
## Which issue does this PR close? - None. CI maintenance. ## What changes are included in this PR? Deletes every `Install protoc` step: five in ci.yml, one in public-api.yml, one in website.yml. The installs were added with the datafusion 48 bump (apache#1501). That bump pulled in the `substrait` crate, whose build script runs prost-build, and prost-build shells out to protoc. The DataFusion 54 bump (apache#2648) dropped `substrait` and `prost-build` from the tree, and prost-build was the only thing that ever ran protoc. The remaining `prost`/`prost-derive` dependencies come from datafusion-proto, which ships pregenerated code and has no build script. So nothing in `make build`, the test suite, or the `cargo doc` run in website.yml needs protoc anymore. A side benefit of this simplification is that we remove a dependency on arduino/setup-protoc, which hasn't seen a commit since September 2024 and was printing a Node runtime deprecation warning on every run. ## Are these changes tested? - Verified on a fork based past apache#2648: all ci.yml and public-api.yml jobs run green without protoc, including the full docker-backed test suite. - `cargo metadata` confirms datafusion-proto and datafusion-proto-common have no build script. `prost-build` is absent from Cargo.lock and nothing in the workspace source references protoc. Co-authored-by: Abanoub Doss <abanoub.doss@gmail.com>
…pache#2800) Thus, dependabot can update these two jobs in a single PR: https://github.com/apache/iceberg-rust/blob/6077a42ea061f73a561c26880993f84fafb1ba0f/.github/workflows/codeql.yml#L48-L54 - apache/infrastructure-actions#1028
Bumps [rand](https://github.com/rust-random/rand) from 0.9.4 to 0.10.2. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/rust-random/rand/blob/master/CHANGELOG.md">rand's changelog</a>.</em></p> <blockquote> <h2>[0.10.2] — 2026-07-02</h2> <h3>Fixes</h3> <ul> <li>Fix possible memory safety violation due to deserialization of <code>UniformChar</code> from bad source (<a href="https://redirect.github.com/rust-random/rand/issues/1790">#1790</a>)</li> </ul> <h3>Changes</h3> <ul> <li>Document required output order of fn <code>partial_shuffle</code> and apply <code>#[must_use]</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1769">#1769</a>)</li> <li>Avoid usage of <code>unsafe</code> in contexts where non-local memory corruption could invalidate contract (<a href="https://redirect.github.com/rust-random/rand/issues/1791">#1791</a>)</li> </ul> <p><a href="https://redirect.github.com/rust-random/rand/issues/1769">#1769</a>: <a href="https://redirect.github.com/rust-random/rand/pull/1769">rust-random/rand#1769</a> <a href="https://redirect.github.com/rust-random/rand/issues/1790">#1790</a>: <a href="https://redirect.github.com/rust-random/rand/pull/1790">rust-random/rand#1790</a> <a href="https://redirect.github.com/rust-random/rand/issues/1791">#1791</a>: <a href="https://redirect.github.com/rust-random/rand/pull/1791">rust-random/rand#1791</a></p> <h2>[0.10.1] — 2026-02-11</h2> <p>This release includes a fix for a soundness bug; see <a href="https://redirect.github.com/rust-random/rand/issues/1763">#1763</a>.</p> <h3>Changes</h3> <ul> <li>Document panic behavior of <code>make_rng</code> and add <code>#[track_caller]</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1761">#1761</a>)</li> <li>Deprecate feature <code>log</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1763">#1763</a>)</li> </ul> <p><a href="https://redirect.github.com/rust-random/rand/issues/1761">#1761</a>: <a href="https://redirect.github.com/rust-random/rand/pull/1761">rust-random/rand#1761</a> <a href="https://redirect.github.com/rust-random/rand/issues/1763">#1763</a>: <a href="https://redirect.github.com/rust-random/rand/pull/1763">rust-random/rand#1763</a></p> <h2>[0.10.0] - 2026-02-08</h2> <h3>Changes</h3> <ul> <li>The dependency on <code>rand_chacha</code> has been replaced with a dependency on <code>chacha20</code>. This changes the implementation behind <code>StdRng</code>, but the output remains the same. There may be some API breakage when using the ChaCha-types directly as these are now the ones in <code>chacha20</code> instead of <code>rand_chacha</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1642">#1642</a>).</li> <li>Rename fns <code>IndexedRandom::choose_multiple</code> -> <code>sample</code>, <code>choose_multiple_array</code> -> <code>sample_array</code>, <code>choose_multiple_weighted</code> -> <code>sample_weighted</code>, struct <code>SliceChooseIter</code> -> <code>IndexedSamples</code> and fns <code>IteratorRandom::choose_multiple</code> -> <code>sample</code>, <code>choose_multiple_fill</code> -> <code>sample_fill</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1632">#1632</a>)</li> <li>Use Edition 2024 and MSRV 1.85 (<a href="https://redirect.github.com/rust-random/rand/issues/1653">#1653</a>)</li> <li>Let <code>Fill</code> be implemented for element types, not sliceable types (<a href="https://redirect.github.com/rust-random/rand/issues/1652">#1652</a>)</li> <li>Fix <code>OsError::raw_os_error</code> on UEFI targets by returning <code>Option<usize></code> (<a href="https://redirect.github.com/rust-random/rand/issues/1665">#1665</a>)</li> <li>Replace fn <code>TryRngCore::read_adapter(..) -> RngReadAdapter</code> with simpler struct <code>RngReader</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1669">#1669</a>)</li> <li>Remove fns <code>SeedableRng::from_os_rng</code>, <code>try_from_os_rng</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1674">#1674</a>)</li> <li>Remove <code>Clone</code> support for <code>StdRng</code>, <code>ReseedingRng</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1677">#1677</a>)</li> <li>Use <code>postcard</code> instead of <code>bincode</code> to test the serde feature (<a href="https://redirect.github.com/rust-random/rand/issues/1693">#1693</a>)</li> <li>Avoid excessive allocation in <code>IteratorRandom::sample</code> when <code>amount</code> is much larger than iterator size (<a href="https://redirect.github.com/rust-random/rand/issues/1695">#1695</a>)</li> <li>Rename <code>os_rng</code> -> <code>sys_rng</code>, <code>OsRng</code> -> <code>SysRng</code>, <code>OsError</code> -> <code>SysError</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1697">#1697</a>)</li> <li>Rename <code>Rng</code> -> <code>RngExt</code> as upstream <code>rand_core</code> has renamed <code>RngCore</code> -> <code>Rng</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1717">#1717</a>)</li> </ul> <h3>Additions</h3> <ul> <li>Add fns <code>IndexedRandom::choose_iter</code>, <code>choose_weighted_iter</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1632">#1632</a>)</li> <li>Pub export <code>Xoshiro128PlusPlus</code>, <code>Xoshiro256PlusPlus</code> prngs (<a href="https://redirect.github.com/rust-random/rand/issues/1649">#1649</a>)</li> <li>Pub export <code>ChaCha8Rng</code>, <code>ChaCha12Rng</code>, <code>ChaCha20Rng</code> behind <code>chacha</code> feature (<a href="https://redirect.github.com/rust-random/rand/issues/1659">#1659</a>)</li> <li>Fn <code>rand::make_rng() -> R where R: SeedableRng</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1734">#1734</a>)</li> </ul> <h3>Removals</h3> <ul> <li>Removed <code>ReseedingRng</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1722">#1722</a>)</li> <li>Removed unused feature "nightly" (<a href="https://redirect.github.com/rust-random/rand/issues/1732">#1732</a>)</li> <li>Removed feature <code>small_rng</code> (<a href="https://redirect.github.com/rust-random/rand/issues/1732">#1732</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/rust-random/rand/commit/1540ea327e8beaf0694ea64f6d9eb8eaadd47bd5"><code>1540ea3</code></a> Prepare rand 0.10.2 (<a href="https://redirect.github.com/rust-random/rand/issues/1800">#1800</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/a29964ad94b54c25b3865626de6964ce0f796a29"><code>a29964a</code></a> Bump chacha20 from 0.10.0 to 0.10.1 in the all-deps group (<a href="https://redirect.github.com/rust-random/rand/issues/1801">#1801</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/ced94914cb75c93a1f19140a966a466345185fff"><code>ced9491</code></a> Tweak docs for RngExt::random_range and SampleRange (<a href="https://redirect.github.com/rust-random/rand/issues/1798">#1798</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/db146647afaf002b866420d34e4501b0dd872163"><code>db14664</code></a> Check UniformChar validity on deser (<a href="https://redirect.github.com/rust-random/rand/issues/1790">#1790</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/bea8620204c7aeecdefc132b5cb0dec8134add4b"><code>bea8620</code></a> Bump the all-deps group with 2 updates (<a href="https://redirect.github.com/rust-random/rand/issues/1796">#1796</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/4f449322825498e4ec1f486119e5fd251ba97f8a"><code>4f44932</code></a> Bump actions/cache from 5 to 6 (<a href="https://redirect.github.com/rust-random/rand/issues/1795">#1795</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/b999a130a990b30af01743021e8ea97f3b09a17e"><code>b999a13</code></a> Bump actions/checkout from 6 to 7 (<a href="https://redirect.github.com/rust-random/rand/issues/1794">#1794</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/aeab810bd9704a3b7666ba0a78e1ad5d1d5ad1ae"><code>aeab810</code></a> Avoid unsafe where safety depends on non-local values (<a href="https://redirect.github.com/rust-random/rand/issues/1791">#1791</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/1896d7c660524a022b3dbc3a1e044e162d766b25"><code>1896d7c</code></a> Add typos CI job (<a href="https://redirect.github.com/rust-random/rand/issues/1789">#1789</a>)</li> <li><a href="https://github.com/rust-random/rand/commit/43eddee18c8cca2cebee929be3899cf183afe801"><code>43eddee</code></a> Bump the all-deps group with 2 updates (<a href="https://redirect.github.com/rust-random/rand/issues/1788">#1788</a>)</li> <li>Additional commits viewable in <a href="https://github.com/rust-random/rand/compare/0.9.4...0.10.2">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…thon (apache#3208) chore(deps-dev): Bump pyiceberg in /bindings/python Bumps [pyiceberg](https://github.com/apache/iceberg-python) from 0.11.1 to 0.12.0. - [Release notes](https://github.com/apache/iceberg-python/releases) - [Commits](apache/iceberg-python@pyiceberg-0.11.1...pyiceberg-0.12.0) --- updated-dependencies: - dependency-name: pyiceberg dependency-version: 0.12.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
) Resolves each data file's sort_order_id against the table's known sort orders during scan planning and carries the result on FileScanTask as sort_order (Some only when genuinely sorted; the reserved unsorted order, id 0, resolves to None) alongside the raw sort_order_id (kept unresolved so a non-zero id that fails to resolve is still recoverable as "physically sorted"). PlanContext precomputes a narrow sort-orders map once per scan, matching the existing unified_partition_type pattern, rather than carrying the full TableMetadataRef into every ManifestFileContext.
…pache#3204) The partition splitter built each group's selection mask as a throwaway `vec![false; num_rows]` and bit-packed it into a `BooleanArray` via `.into()`, allocating an n-byte `Vec<bool>` plus a full packing pass per group per batch on every partitioned write. Build the packed buffer directly with `BooleanBufferBuilder`, setting only the selected bits.
…loning (apache#3214) `ManifestFilterVisitor::bytes_to_datum` took the field `Type` by value, so every comparison-leaf call site cloned the field's boxed `Type` to decode a bound -- and the two in-predicate sites cloned the entire `NestedField`. `bytes_to_datum` only borrows the type to reach its `PrimitiveType`, so take `&Type` and pass `&reference.field().field_type` at the call sites, dropping the per-leaf clones on the manifest-pruning path.
* feat(writer): add PositionDeleteFileWriter Add PositionDeleteFileWriter and its builder under writer/base_writer. It writes a position delete file with the two required columns file_path (string, field id 2147483546) and pos (long, 2147483545) and sets DataContentType::PositionDeletes on close. position_delete_schema() returns the canonical Iceberg schema, built from the existing metadata_columns field definitions. write() checks that a batch has exactly those two required, non-nullable, correctly typed columns before writing, and rejects a closed writer. close() propagates the partition key and leaves sort_order_id null. Position delete files are a v2 construct: v3 replaces them with deletion vectors and forbids adding new ones, so callers must not route v3 writes here. This base writer has no format-version gate by design; that gating belongs at the transaction/commit layer. The writer also does not sort its input, so callers must supply rows sorted by (file_path, pos) until a higher-level writer enforces it. Setting referenced_data_file and a higher-level DeltaWriter are follow-ups. Tested: schema shape, a parquet round trip for single and multiple writes, partition propagation, that the reserved field ids survive into the written parquet schema, and the validation cases (wrong column count, wrong/missing/ unparsable field ids, wrong types including LargeUtf8, nullable columns, and writes after close). Refs apache#340. * docs(writer): note the optional row column is unsupported The position delete spec allows an optional `row` column alongside `file_path` and `pos` that inlines the deleted row's values. This writer handles only the two required columns, so say so in the module docs. The batch validator already rejects a third column with a matching message. --------- Co-authored-by: Shawn Chang <schang@apache.org>
…che#3194) The release scripts run each step in a `( trap - ERR; ... )` subshell so a failure is reported exactly once. bash 3.2 (macOS /bin/bash) does not run the parent's ERR trap for a failing subshell, so the script still exits on failure but does not report which step failed. bash 4.0 and later do; verified against 3.2 through 5.1. Print a warning at startup on bash older than 4 in the four scripts that install the trap. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…#3226) `AesKeySize::from_key_length` and `AesKeySize::from_str` returned `FeatureUnsupported` for anything outside 128/192/256-bit. AES defines no other key sizes, so these are invalid input rather than something a later version could support, and `FeatureUnsupported` here is indistinguishable from the genuinely unsupported cases such as an unknown key metadata version. `from_key_length` is reachable from `SecureKey::new`, `StandardKeyMetadata::try_new` and the `encryption.data-key-length` table property, all of which now surface `DataInvalid`. Messages change from "Unsupported ..." to "Invalid ..." to match.
* add standard ket metadata python bindings * fix * fix: clarify unsupported key metadata version error Report the received and supported key metadata versions, matching the wording expected by PyIceberg. Add Rust and Python regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * named struct * improve tests * tests --------- Co-authored-by: Kevin Liu <kevin.jq.liu@gmail.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* feat: bloom filter pushdown * test * avoid copy * new name * fmt * doc string * handle type widening * widen decimals * pub crate * full import * lifetime * log * add not supported comment * make it more safe * Add tests * fix * fmt * don't read not predicates * tests * remove boolean * doc * update 0 test * UUID * update comment * update test doc
* feat(spec): add unknown primitive type support Co-authored-by: Codex <codex@openai.com> * fix(arrow): validate row ID synthesis inputs * fix(spec): centralize unknown field validation * fix(spec): address unknown type review feedback * fix(spec): address remaining review feedback * fix(spec): centralize dropped partition source handling * fix(inspect): recover historical partition bounds * fix(spec): validate default partition sources on metadata load * Revert "fix(inspect): recover historical partition bounds" This reverts commit 6403d48. --------- Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* ci: make merge queue checks reliable Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * ci: keep website checks optional Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
# Conflicts: # crates/iceberg/public-api.txt # crates/iceberg/src/transaction/append.rs # crates/iceberg/src/transaction/mod.rs
A manifest rewritten by a second overwrite fell through to add_existing_entry for entries that were already Deleted, forcing their status back to Existing and resurrecting the files as live data. Guard on is_alive() first and re-add prior tombstones as Deleted with their original snapshot id, matching Java's filterManifestWithDeletedFiles, which only re-adds live entries.
The rewrite used the table's current schema and default partition spec, so after a schema or partition evolution the Avro header carried the wrong schema-id and partition-spec-id and readers interpreted the partition values against the wrong mapping. Take both from the manifest being rewritten, as Java does via newManifestWriter(reader.spec()).
Both manifest filters dropped a manifest whose entries were all Deleted, which lets the removed files reappear as live data. This is the same fix FastAppend took in apache#2545 for apache#2148, applied to the no-deletes fast path and to the rewrite loop.
The summary counted every file the caller passed to delete_data_files, so a path that was never committed or was passed twice inflated deleted-data-files and deleted-records. The rewrite pass now records the entries it marked Deleted, with the schema and partition spec of the manifest each was recorded in, and the summary counts those. Two changes follow from it. The summary is built after the manifests rather than before, since the counts are only known once the rewrite has run. And the full-table truncate path is now gated on an operation hook that a partial overwrite answers false, instead of on the operation being Overwrite; with truncate off, update_totals needs a saturating subtraction so an overwrite that deletes more than the previous total does not underflow.
The rewrite hardcoded {location}/metadata and a fresh Uuid::now_v7, so a table
with a custom write.metadata.path got its rewritten manifests in the wrong
prefix, and orphan cleanup could not find them after a failed commit because
the name was not tied to the commit. Reuse the path SnapshotProducer already
builds from metadata_location(), the commit uuid and the manifest counter; the
counter becomes an AtomicU64 because the rewrite runs behind a shared
reference.
rewrite_manifest built its own plain output file, so on an encrypted table the rewritten manifest went out in plaintext while its key metadata said otherwise. It now goes through SnapshotProducer::new_manifest_writer, which already branches on the table's encryption manager; the writer takes the schema and partition spec as arguments so the rewrite can keep the ones the manifest was written under. That leaves new_manifest_path with no caller outside this module, so it becomes private again.
add_deleted_entry set the status but left snapshot_id to the caller, while add_delete_entry right above it stamps both; the two differed only by tense and a tombstone written with the wrong snapshot_id misattributes the delete, which expire_snapshots then acts on. Taking the reviewer's second option: rename rather than stamp, because carrying an earlier tombstone forward has to keep its original snapshot_id, and document that the caller owns the field. Named add_tombstone_entry rather than something closer to main's add_delete_entry (apache#2367) precisely so the two cannot be confused.
A delete-only overwrite reported Operation::Overwrite, so tools keying off the summary read it as a replacement. Match Java's BaseOverwriteFiles: DELETE when there are only deletes, APPEND when there are only adds, OVERWRITE when there are both. test_overwrite_basic adds without deleting, so its expected operation becomes APPEND. Also names deleted data files in the precondition error, which has allowed a delete-only commit since the guard was relaxed.
is_err() alone passes on any error, so the test would stay green if the commit started failing earlier for an unrelated reason. Assert the kind and the message instead.
An overwrite silently ignored a delete path that was not live in any manifest, so a typo or an already-deleted file committed as a no-op delete. Java's BaseOverwriteFiles validates the same thing and fails with "Missing required files to delete"; the rewrite pass already records what it marked Deleted, so the check is a comparison against the requested set.
Regenerated with make generate-public-api (cargo-public-api 0.51.0); the snapshot was missing OverwriteAction and its builder methods.
Author
|
The first commit is a merge of apache/main; the branch no longer builds without it ( |
Author
|
The three producer commits (count removed files, unique manifest names, manifest writer with schema and spec) are proposed on their own against main as apache#3253; they drop out of here when that merges. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are included in this PR?
Merges main into the branch (
SnapshotProducer::newandManifestWriterBuilder::newlostkey_metadata,load_manifestbecamemanifest_reader().read()), then one commit per review comment from @laskoviymishka:SnapshotProducer::new_manifest_writer, so they are named after the commit uuid, honour the metadata location, and use the encryption manager.add_deleted_entryis nowadd_tombstone_entryand documents that the caller ownssnapshot_id; the other route, stamping this manifest's snapshot id, isadd_delete_entry.BaseOverwriteFiles.public-api.txtregenerated.Are these changes tested?
Every fix has a test in
overwrite.rsthat reads the committed manifests back and fails on the code before it.cargo test -p iceberg, fmt and clippy with-D warningsare clean.