From f8e0705f41798228c8642e2f296fc4caf7189fcb Mon Sep 17 00:00:00 2001 From: pvary Date: Mon, 5 Oct 2026 13:03:18 +0200 Subject: [PATCH 1/5] Docs: Track spec versions and changes since adoption in the specs Spec changes are adopted by community vote and are independent of the Java releases, so listing them in the Java release notes is misleading. Each spec now starts with a version table that shows the status, adoption date and adoption vote of every version. Specs with changes after adoption end with a chapter that lists each voted change with its vote, pull request and vote close date. The REST Catalog spec page gets the same treatment. The Releases menu links to the version table of each spec, and spec entries are removed from the Java release notes. Generated-by: GitHub Copilot CLI (Claude Opus 5.5) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- format/expressions-spec.md | 6 ++++ format/gcm-stream-spec.md | 6 ++++ format/mumbling-spec.md | 6 ++++ format/puffin-spec.md | 6 ++++ format/spec.md | 33 +++++++++++++++++++-- format/udf-spec.md | 12 ++++++++ format/view-spec.md | 6 ++++ site/docs/releases.md | 47 ------------------------------ site/docs/rest-catalog-spec.md | 53 ++++++++++++++++++++++++++++++++++ site/nav.yml | 10 ++++++- 10 files changed, 134 insertions(+), 51 deletions(-) diff --git a/format/expressions-spec.md b/format/expressions-spec.md index 2975648a258e..faf81f2e330f 100644 --- a/format/expressions-spec.md +++ b/format/expressions-spec.md @@ -24,6 +24,12 @@ This document defines the structure and behavior of expressions for use in Icebe Stored expressions are needed for use cases like data validations (`CHECK` constraints) and default values (for instance, `current_timestamp()`). Expressions are exchanged in use cases like server-side scan planning in the catalog protocol. +## Format Versioning + +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|---------|------------|--------------------------------------------------------------------------|------------------------| +| 1 | Adopted | 2026-06-30 | [vote](https://lists.apache.org/thread/6wfmjhgthlykwbk3f7df4zgcm40xtm2o) | - | + ## Overview diff --git a/format/gcm-stream-spec.md b/format/gcm-stream-spec.md index 8168780e118b..30e931fa9dd4 100644 --- a/format/gcm-stream-spec.md +++ b/format/gcm-stream-spec.md @@ -20,6 +20,12 @@ title: "AES GCM Stream Spec" # AES GCM Stream file format extension +## Format Versioning + +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|---------|------------|------|------------------------| +| 1 | Adopted | 2022-11-27 | - | - | + ## Background and Motivation Iceberg supports a number of data file formats. Two of these formats (Parquet and ORC) have built-in encryption capabilities, that allow to protect sensitive information in the data files. However, besides the data files, Iceberg tables also have metadata files, that keep sensitive information too (e.g., min/max values in manifest files, or bloom filter bitsets in puffin files). Metadata file formats (AVRO, JSON, Puffin) don't have encryption support. diff --git a/format/mumbling-spec.md b/format/mumbling-spec.md index 84012bc847db..109b6f361d2b 100644 --- a/format/mumbling-spec.md +++ b/format/mumbling-spec.md @@ -28,6 +28,12 @@ This spec is for version 1. [roaring]: https://roaringbitmap.org/ +## Format Versioning + +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|--------|---------|------|------------| +| 1 | Draft | | | | + ## Overview Mumbling bitmaps are based on the same idea as Roaring bitmaps: a bitmap is diff --git a/format/puffin-spec.md b/format/puffin-spec.md index 8617ae6d8e96..6d7ef04977cc 100644 --- a/format/puffin-spec.md +++ b/format/puffin-spec.md @@ -27,6 +27,12 @@ Puffin file contains arbitrary pieces of information (here called "blobs"), along with metadata necessary to interpret them. The blobs supported by Iceberg are documented at [Blob types](#blob-types). +## Format Versioning + +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|---------|------------|--------------------------------------------------------------------------|------------------------| +| 1 | Adopted | 2022-06-23 | [vote](https://lists.apache.org/thread/950rz31y3kr3kz0zzncwokvgzbrmmz4q) | - | + ## Format specification A file conforming to the Puffin file format specification should have the structure diff --git a/format/spec.md b/format/spec.md index 1510c0dd5ac4..17f7390e162d 100644 --- a/format/spec.md +++ b/format/spec.md @@ -24,9 +24,12 @@ This is a specification for the Iceberg table format that is designed to manage ## Format Versioning -Versions 1, 2 and 3 of the Iceberg spec are complete and adopted by the community. - -**Version 4 is under active development and has not been formally adopted.** +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|----------------|------------|--------------------------------------------------------------------------|---------------------------------------------------| +| 4 | In development | | | | +| 3 | Adopted | 2025-05-23 | [vote](https://lists.apache.org/thread/9oncq0j0222nrm6scvm89xdq3gozg24p) | [Changes since adoption](#changes-since-adoption) | +| 2 | Adopted | 2021-08-02 | [vote](https://lists.apache.org/thread/ws2gg52d124p7bx9jgrn3kctrtfgtltp) | [Changes since adoption](#changes-since-adoption) | +| 1 | Adopted | - | - | - | The format version number is incremented when new features are added that will break forward-compatibility---that is, when older readers would not read newer table features correctly. Tables may continue to be written with an older version of the spec to ensure compatibility by not using features that are not yet implemented by processing engines. @@ -2151,3 +2154,27 @@ The Geometry and Geography class hierarchy and its Well-known text (WKT) and Wel Points are always defined by the coordinates X, Y, Z (optional), and M (optional), in this order. X is the longitude/easting, Y is the latitude/northing, and Z is usually the height, or elevation. M is a fourth optional dimension, for example a linear reference value (e.g., highway milepost value), a timestamp, or some other value as defined by the CRS. The version of the OGC standard first used here is 1.2.1, but future versions may also be used if the WKB representation remains wire-compatible. + +## Changes since adoption + +Each change is listed under the most recent version it affects. + +### Version 3 + +| Date | Change | Pull request | Vote | +|------------|---------------------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| +| 2026-08-19 | Clarify content file uniqueness within a snapshot | [#17198](https://github.com/apache/iceberg/pull/17198) | [vote](https://lists.apache.org/thread/orwny91fqyfmzx7w7p6wj7cgmndvd8r3) | +| 2026-06-13 | Clarify the result type of the day partition transform in manifests | [#16446](https://github.com/apache/iceberg/pull/16446) | [vote](https://lists.apache.org/thread/gz432tvboxvno2v7g3l17c8tbtxckxrb) | +| 2026-05-18 | Clarify conventions for non-default CRS in geospatial types | [#15834](https://github.com/apache/iceberg/pull/15834) | [vote](https://lists.apache.org/thread/8z0kc8vz1379lghwlcwl0jmsoym6shd7) | +| 2025-10-14 | Clarify restrictions for geometry types | [#14250](https://github.com/apache/iceberg/pull/14250) | [vote](https://lists.apache.org/thread/v96xj6kxwclyl52d97dkcvzxytfypyp3) | +| 2025-09-18 | Bring back `added-rows` in snapshot fields | [#14048](https://github.com/apache/iceberg/pull/14048) | [vote](https://lists.apache.org/thread/5gff76hvqy7r1lksrt9ld9h3pdp03nb5) | +| 2025-07-31 | Correct the type of `snapshot-id` in table statistics | [#13513](https://github.com/apache/iceberg/pull/13513) | [vote](https://lists.apache.org/thread/z6loky5225j1vszorm199ywwncjwf6qz) | + +### Version 2 + +| Date | Change | Pull request | Vote | +|------------|-----------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| +| 2025-09-15 | Deprecate position delete files with row data | [#14045](https://github.com/apache/iceberg/pull/14045) | [vote](https://lists.apache.org/thread/tfy96bqmz1bmdxr73x17w3xxj3yzs606) | +| 2025-03-06 | Add an implementation note on `current-snapshot-id` | [#12334](https://github.com/apache/iceberg/pull/12334) | [vote](https://lists.apache.org/thread/54r4nm7qmr4vxhdpwmbx5rntynspskl7) | +| 2025-01-24 | Document optional snapshot summary fields | [#11660](https://github.com/apache/iceberg/pull/11660) | [vote](https://lists.apache.org/thread/mz01jwt69osqxhx9d3dd9xzncv9yncd0) | +| 2024-08-04 | Clarify file system tables | [#10833](https://github.com/apache/iceberg/pull/10833) | [vote](https://lists.apache.org/thread/1jm9zd6rtgjsvly3x099wvgdbohjn29v) | diff --git a/format/udf-spec.md b/format/udf-spec.md index 70ed53a5dbe6..58958a7fa079 100644 --- a/format/udf-spec.md +++ b/format/udf-spec.md @@ -20,6 +20,12 @@ title: "SQL UDF Spec" # Iceberg UDF Spec +## Format Versioning + +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|---------|------------|--------------------------------------------------------------------------|---------------------------------------------------| +| 1 | Adopted | 2026-02-05 | [vote](https://lists.apache.org/thread/whbgoc325o99vm4b599f0g1owhgww2kx) | [Changes since adoption](#changes-since-adoption) | + ## Background and Motivation A SQL user-defined function (UDF or UDTF) is a callable routine that accepts input parameters and executes a function body. @@ -375,3 +381,9 @@ RETURN SELECT name, color FROM fruits WHERE color = c; "secure": false } ``` + +## Changes since adoption + +| Date | Change | Pull request | Vote | +|------------|----------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| +| 2026-07-01 | Add optional `specific-name` to the UDF definition model | [#16727](https://github.com/apache/iceberg/pull/16727) | [vote](https://lists.apache.org/thread/75xs4trcgcpog8b2zoxpknf8bq6vfk4c) | diff --git a/format/view-spec.md b/format/view-spec.md index 34a671454778..46789ac8634c 100644 --- a/format/view-spec.md +++ b/format/view-spec.md @@ -20,6 +20,12 @@ title: "View Spec" # Iceberg View Spec +## Format Versioning + +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|---------|------------|------|------------------------| +| 1 | Adopted | 2022-04-03 | - | - | + ## Background and Motivation Most compute engines (e.g. Trino and Apache Spark) support views. A view is a logical table that can be referenced by future queries. Views do not contain any data. Instead, the query stored by the view is executed every time the view is referenced by another query. diff --git a/site/docs/releases.md b/site/docs/releases.md index e0a7b79f02aa..33ca3d408b13 100644 --- a/site/docs/releases.md +++ b/site/docs/releases.md @@ -287,15 +287,10 @@ Apache Iceberg 1.11.0 was released on May 19, 2026. The 1.11.0 release contains bug fixes and new features. For full release notes visit [Github](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.11.0) * Deprecation / End of Support - - Spec: Deprecate Position delete files with row data ([\#14045](https://github.com/apache/iceberg/pull/14045)) - Drop support for Java 11 ([\#14400](https://github.com/apache/iceberg/pull/14400)) - Spark 3.4: Deprecate support ([\#14099](https://github.com/apache/iceberg/pull/14099)) - Flink: Remove Flink 1.19 support ([\#13714](https://github.com/apache/iceberg/pull/13714)) - AWS, Core, Data, Spark: Remove deprecations for 1.11.0 ([\#14059](https://github.com/apache/iceberg/pull/14059)) -* Spec - - Introduce SQL UDF specification ([\#14117](https://github.com/apache/iceberg/pull/14117)) - - Bring back added-rows in snapshot fields ([\#14048](https://github.com/apache/iceberg/pull/14048)) - - Clarify restrictions for geometry types in V3 ([\#14250](https://github.com/apache/iceberg/pull/14250)) * API - Add geospatial bounding box types and implement intersects checking ([\#12667](https://github.com/apache/iceberg/pull/12667)) - Introduce foundational types for V4 manifest support ([\#15049](https://github.com/apache/iceberg/pull/15049)) @@ -546,15 +541,6 @@ The 1.10.0 release contains bug fixes and new features. For full release notes v - Hive: Throw NoSuchNamespaceException when listing a non-exist namespace ([\#13130](https://github.com/apache/iceberg/pull/13130)). Previously, an empty list was returned. - Core: fix spec non-confirming field ids for partition stats files ([\#13329](https://github.com/apache/iceberg/pull/13329)) -* Spec - - Table: Clarify write requirement to prevent orphaned DVs ([\#13042](https://github.com/apache/iceberg/pull/13042)) - - Table: Clarify behavior of special geo objects for lower and upper bounds ([\#12956](https://github.com/apache/iceberg/pull/12956)) - - Table: Add encryption keys ([\#12162](https://github.com/apache/iceberg/pull/12162)) - - Table: Avoid struct field conflicts in default values ([\#12841](https://github.com/apache/iceberg/pull/12841)) - - REST: Add row lineage fields ([\#13010](https://github.com/apache/iceberg/pull/13010)) - - REST: Add encryption keys ([\#12987](https://github.com/apache/iceberg/pull/12987)) - - REST: remove update to enable row lineage as it is always on for V3 table ([\#12986](https://github.com/apache/iceberg/pull/12986)) - - REST: mark 503 as non retryable ([\#13619](https://github.com/apache/iceberg/pull/13619)) * API - Add table metadata keys for encryption ([\#12927](https://github.com/apache/iceberg/pull/12927)) - Add deleteFile to RowDelta API ([\#12861](https://github.com/apache/iceberg/pull/12861)) @@ -696,14 +682,6 @@ the API. This is fixed in 1.9.1. * Deprecation / End of Support - Spark: Remove Spark 3.3 support ([\#12279](https://github.com/apache/iceberg/pull/12279)) - Build: Remove Hadoop 2 ([\#12348](https://github.com/apache/iceberg/pull/12348)) -* Spec - - Spec: Support geo type ([\#10981](https://github.com/apache/iceberg/pull/10981)) - - Spec: Allow Equality Deletes with Row Lineage and Define Behavior ([\#12230](https://github.com/apache/iceberg/pull/12230)) - - Spec: Add implementation note on `current-snapshot-id` ([\#12334](https://github.com/apache/iceberg/pull/12334)) - - Spec: update to reflect lineage is required ([\#12580](https://github.com/apache/iceberg/pull/12580)) - - Spec: Update row lineage requirements for upgrading tables ([\#12781](https://github.com/apache/iceberg/pull/12781)) - - Spec: Clarify variant lower/upper bounds ([\#12658](https://github.com/apache/iceberg/pull/12658)) - - Spec: Allow the use of source-id in V3 ([\#12644](https://github.com/apache/iceberg/pull/12644)) * API - Support default values in UpdateSchema ([\#12211](https://github.com/apache/iceberg/pull/12211)) - Move variant to API and add extract expression ([\#12304](https://github.com/apache/iceberg/pull/12304)) @@ -786,14 +764,6 @@ The 1.8.0 release contains bug fixes and new features. For full release notes vi * Deprecation / End of Support - Spark 3.3 - Removed Hive Runtime -* Spec - - Add Deletion vectors to the table specification ([\#11240](https://github.com/apache/iceberg/pull/11240)) - - Document optional snapshot summary fields ([\#11660](https://github.com/apache/iceberg/pull/11660)) - - Add Variant Type ([\#10831](https://github.com/apache/iceberg/pull/10831)) - - Add EnableRowLineage metadata update ([\#12050](https://github.com/apache/iceberg/pull/12050)) - - Add added-rows field to Snapshot ([\#11976](https://github.com/apache/iceberg/pull/11976)) - - Reassign row lineage field IDs ([\#12100](https://github.com/apache/iceberg/pull/12100)) - - Document S3 cross region enabled configuration in REST spec ([\#11260](https://github.com/apache/iceberg/pull/11260)) * API - Define Variant Data type ([\#11324](https://github.com/apache/iceberg/pull/11324)) - Add UnknownType ([\#12012](https://github.com/apache/iceberg/pull/12012)) @@ -944,10 +914,6 @@ The 1.7.0 release contains fixes, dependency updates, and new features. For full * Hive - Add View support for HIVE catalog ([\#9852](https://github.com/apache/iceberg/pull/9852)) * OpenAPI - - Add RemovePartitionSpecsUpdate REST update type ([\#10846](https://github.com/apache/iceberg/pull/10846)) - - Add endpoint to retrieve valid credentials for a given table ([\#11281](https://github.com/apache/iceberg/pull/11281)) - - Standardize credentials in loadTable/loadView responses ([\#10722](https://github.com/apache/iceberg/pull/10722)) - - Add Scan Planning Endpoints to open api spec ([\#9695](https://github.com/apache/iceberg/pull/9695)) - Add REST Compatibility Kit ([\#10908](https://github.com/apache/iceberg/pull/10908)) * Spark - Parallelize reading files in migrate procedures ([\#11043](https://github.com/apache/iceberg/pull/11043)) @@ -959,10 +925,6 @@ The 1.7.0 release contains fixes, dependency updates, and new features. For full - Spark Action to Analyze table ([\#10288](https://github.com/apache/iceberg/pull/10288)) - Support Column Stats ([\#10659](https://github.com/apache/iceberg/pull/10659)) - Add RewriteTablePath action interface ([\#10920](https://github.com/apache/iceberg/pull/10920)) -* Spec - - Add v3 types and type promotion ([\#10955](https://github.com/apache/iceberg/pull/10955)) - - Adds Row Lineage ([\#11130](https://github.com/apache/iceberg/pull/11130)) - - Deprecate the file system table scheme. ([\#10833](https://github.com/apache/iceberg/pull/10833)) ### 1.6.1 release @@ -1039,11 +1001,6 @@ The 1.6.0 release contains fixes, dependency updates, and new features (like Kaf - Fix metadata file not found ([\#10069](https://github.com/apache/iceberg/pull/10069)) - Use base table metadata to create HiveLock ([\#10016](https://github.com/apache/iceberg/pull/10016)) - Turn off the stats gathering when iceberg.hive.keep.stats is false ([\#10148](https://github.com/apache/iceberg/pull/10148)) -* Specs - - OpenAPI: TableRequirements should use union of subclasses ([\#10434](https://github.com/apache/iceberg/pull/10434)) - - OpenAPI: Deprecate oauth/tokens endpoint ([\#10603](https://github.com/apache/iceberg/pull/10603)) - - OpenAPI: Fix additionalProperties for SnapshotSummary ([\#9838](https://github.com/apache/iceberg/pull/9838)) - - OpenAPI: Fix property names for stats/partition stats ([\#10662](https://github.com/apache/iceberg/pull/10662)) * Vendor Integrations - AWS: Make sure Signer + User Agent config are both applied ([\#10198](https://github.com/apache/iceberg/pull/10198)) - AWS: Retain Glue Catalog column comment after updating Iceberg table ([\#10276](https://github.com/apache/iceberg/pull/10276)) @@ -1131,10 +1088,6 @@ The 1.5.0 release adds a variety of new features and bug fixes. * Kafka-Connect - Initial project setup and event data structures ([\#8701](https://github.com/apache/iceberg/pull/8701)) - Sink connector with data writers and converters ([\#9466](https://github.com/apache/iceberg/pull/9466)) -* Spec - - Add partition stats spec ([\#7105](https://github.com/apache/iceberg/pull/7105)) - - add nanosecond timestamp types ([\#8683](https://github.com/apache/iceberg/pull/8683)) - - Add multi-arg transform ([\#8579](https://github.com/apache/iceberg/pull/8579)) * Vendor Integrations - AWS: Support setting description for Glue table ([\#9530](https://github.com/apache/iceberg/pull/9530)) - AWS: Update S3FileIO test to run when CLIENT_FACTORY is not set ([\#9541](https://github.com/apache/iceberg/pull/9541)) diff --git a/site/docs/rest-catalog-spec.md b/site/docs/rest-catalog-spec.md index 3cf0cc01b971..57d4a2cba5f9 100644 --- a/site/docs/rest-catalog-spec.md +++ b/site/docs/rest-catalog-spec.md @@ -25,6 +25,12 @@ Iceberg defines a REST-based Catalog API for managing table metadata and perform You can also explore the API interactively using the [Swagger UI](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/iceberg/main/open-api/rest-catalog-open-api.yaml). +## Format Versioning + +| Version | Status | Adopted | Vote | Changes since adoption | +|---------|---------|---------|------|---------------------------------------------------| +| 1 | Adopted | - | - | [Changes since adoption](#changes-since-adoption) | + ## REST Catalog Protocol As the Iceberg project grew to support more languages and engines, pluggable catalogs started to cause some practical problems. Catalogs needed to be implemented in multiple languages and it proved difficult for commercial offerings to support many different catalogs and clients. @@ -40,3 +46,50 @@ The REST protocol is important for several reasons: - **Security**: The protocol supports secure table sharing using credential vending or remote signing. You can use the REST catalog protocol with any built-in catalog using translation in the `CatalogHandlers` class, or using the community maintained [`iceberg-rest-fixture`](https://hub.docker.com/r/apache/iceberg-rest-fixture) docker image. + +## Changes since adoption + +| Date | Change | Pull request | Vote | +|------------|--------------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| +| 2026-09-08 | Add catalog-provided `labels` to table and view load results | [#15750](https://github.com/apache/iceberg/pull/15750) | [vote](https://lists.apache.org/thread/do69l2nfm88m024ol34m2pdy0rqomqnz) | +| 2026-09-03 | Add finer grained read restrictions to `loadTable` | [#13879](https://github.com/apache/iceberg/pull/13879) | [vote](https://lists.apache.org/thread/0zloqhp8wkgyn04yg69j71cwcg5n7g74) | +| 2026-08-02 | Add the variant type | [#17256](https://github.com/apache/iceberg/pull/17256) | [vote](https://lists.apache.org/thread/gkj526shb0y96hdmms73sn3tvsw8mgw9) | +| 2026-07-30 | Formalize remote signing configuration | [#16822](https://github.com/apache/iceberg/pull/16822) | [vote](https://lists.apache.org/thread/n4hyqr3k2gnzbxvs3l1o4cz3ggdfwkx2) | +| 2026-07-27 | Align expressions with the expressions spec | [#17138](https://github.com/apache/iceberg/pull/17138) | [vote](https://lists.apache.org/thread/njo5bzgf4m126ly74ydz1pfznhz70h38) | +| 2026-06-06 | Add list and load function endpoints | [#15180](https://github.com/apache/iceberg/pull/15180) | [vote](https://lists.apache.org/thread/qd7jb3fdx65mcjvcjomrxg7rgxf3yn0m) | +| 2026-05-26 | Add the unregister table endpoint | [#16400](https://github.com/apache/iceberg/pull/16400) | [vote](https://lists.apache.org/thread/vy9fqyopp9c149oy40brdhb10grpq32g) | +| 2026-05-18 | Add the `CatalogObjectIdentifier` schema | [#16144](https://github.com/apache/iceberg/pull/16144) | [vote](https://lists.apache.org/thread/xyzlndcp4x7zq6xydpcbp192ov9p2skz) | +| 2026-04-22 | Clarify identifier uniqueness across tables and views | [#15691](https://github.com/apache/iceberg/pull/15691) | [vote](https://lists.apache.org/thread/2647zx7omfyocxl8og7s2tbomgprxvts) | +| 2026-04-15 | Add a 404 response to the config endpoint | [#15746](https://github.com/apache/iceberg/pull/15746) | [vote](https://lists.apache.org/thread/bcc62zxhc1rqjmwxxll81rpn8yfnr9fy) | +| 2026-03-31 | Promote the remote signing endpoint to the main spec | [#15450](https://github.com/apache/iceberg/pull/15450) | [vote](https://lists.apache.org/thread/rspzcx0o2mkm271fb8csghs4x98sfh3h) | +| 2026-03-13 | Add `scan-planning-mode` to the `loadTable` config | [#14867](https://github.com/apache/iceberg/pull/14867) | [vote](https://lists.apache.org/thread/mt4y8vw80glnqqwsr8czfmwvhqf09h46) | +| 2026-02-12 | Add `referenced-by` to `loadTable` | [#13810](https://github.com/apache/iceberg/pull/13810) | [vote](https://lists.apache.org/thread/jnv5f978gc530z0vx9h6l18dopoltm7j) | +| 2026-02-09 | Add access delegation to `registerTable` | [#15231](https://github.com/apache/iceberg/pull/15231) | [vote](https://lists.apache.org/thread/fgpwnsbslwkm4qxh10y6xn52pprbk274) | +| 2026-02-09 | Add access delegation to the scan planning endpoints | [#14781](https://github.com/apache/iceberg/pull/14781) | [vote](https://lists.apache.org/thread/80dm5b8rxcnqwfqwqoc43grjg8hktqgf) | +| 2026-01-17 | Add the register view endpoint | [#14869](https://github.com/apache/iceberg/pull/14869) | [vote](https://lists.apache.org/thread/520or6svg2yxcthbjc2wt5c9lbx0ntpz) | +| 2025-12-19 | Add `ETag` to `CommitTableResponse` | [#14760](https://github.com/apache/iceberg/pull/14760) | [vote](https://lists.apache.org/thread/fvjrkgm29ohh1f222opps0xnowgo8zz3) | +| 2025-12-08 | Add idempotency keys to the mutating scan planning endpoints | [#14730](https://github.com/apache/iceberg/pull/14730) | [vote](https://lists.apache.org/thread/6xo0v5bnvzq0ktd7tptszgm0csjp2p80) | +| 2025-12-05 | Make the namespace separator configurable by the server | [#14448](https://github.com/apache/iceberg/pull/14448) | [vote](https://lists.apache.org/thread/8k777v9l2lr7930g8fxrwjqnh5csl48k) | +| 2025-11-21 | Add `min-rows-requested` to scan planning requests | [#14565](https://github.com/apache/iceberg/pull/14565) | [vote](https://lists.apache.org/thread/t210ccrcz7fglv3vlwvnz827hb65217d) | +| 2025-11-20 | Add storage credentials to scan planning results | [#14563](https://github.com/apache/iceberg/pull/14563) | [vote](https://lists.apache.org/thread/vq6nzsc0vs81r8dhr94dp222s0gq1ykw) | +| 2025-11-19 | Add the `Idempotency-Key` header | [#14196](https://github.com/apache/iceberg/pull/14196) | [vote](https://lists.apache.org/thread/ybo8xst7h2p8vt6nctxf4nxfc0jbc6rq) | +| 2025-11-18 | Add `planId` to the credentials endpoint | [#14519](https://github.com/apache/iceberg/pull/14519) | [vote](https://lists.apache.org/thread/002y8ndg0s5l7vswkpr4nlr31ld3myg3) | +| 2025-08-21 | Mark 503 as non-retryable for table updates | [#13619](https://github.com/apache/iceberg/pull/13619) | [vote](https://lists.apache.org/thread/xno8c88sm4mlt0y5yw7p3tsob8wmhgkg) | +| 2025-05-27 | Add row lineage fields | [#13010](https://github.com/apache/iceberg/pull/13010) | [vote](https://lists.apache.org/thread/1q3xkvn2373n3g43qjjhnzj7p0v3d9p9) | +| 2025-05-12 | Add encryption key updates | [#12987](https://github.com/apache/iceberg/pull/12987) | [vote](https://lists.apache.org/thread/l055gx3spwy2n24kthy38fhoj2qy1tq7) | +| 2025-03-20 | Clarify dropping non-empty namespaces | [#12518](https://github.com/apache/iceberg/pull/12518) | [vote](https://lists.apache.org/thread/lxfbrv32wwn1tkvc37dmor2f8rbkrv2c) | +| 2025-02-17 | Add the overwrite option to table registration | [#12239](https://github.com/apache/iceberg/pull/12239) | [vote](https://lists.apache.org/thread/bm9x6f9lxh2901j2scwoqs2gf6mlm6yt) | +| 2025-02-14 | Add the `RemoveSchemas` update | [#12022](https://github.com/apache/iceberg/pull/12022) | [vote](https://lists.apache.org/thread/n97qo5nb09ggz5591g7cp99767fv5cf7) | +| 2025-01-28 | Add initial and write defaults to schema fields | [#12094](https://github.com/apache/iceberg/pull/12094) | [vote](https://lists.apache.org/thread/k606d81cv8h9f0oxtv5wzwg7qtb8mgxc) | +| 2025-01-27 | Add freshness-aware table loading | [#11946](https://github.com/apache/iceberg/pull/11946) | [vote](https://lists.apache.org/thread/cfxbdzs0rfopjg8qy101qc9vsokn75p5) | +| 2025-01-27 | Deprecate `snapshot-id` in `SetStatisticsUpdate` | [#12010](https://github.com/apache/iceberg/pull/12010) | [vote](https://lists.apache.org/thread/lcjvzp8o1n6bvvhbr1lo0zx7w9ncs1pm) | +| 2024-11-25 | Deprecate `last-column-id` in `AddSchemaUpdate` | [#11514](https://github.com/apache/iceberg/pull/11514) | [vote](https://lists.apache.org/thread/of8x50751vy3hv3xk3gcgncp4jry8z6x) | +| 2024-11-02 | Add deletion vector fields to delete files | [#11240](https://github.com/apache/iceberg/pull/11240) | [vote](https://lists.apache.org/thread/gxkkpvv2n69q8xq4n3k83ykp11cb1kzn) | +| 2024-10-24 | Add the credentials refresh endpoint | [#11281](https://github.com/apache/iceberg/pull/11281) | [vote](https://lists.apache.org/thread/jkpy0zhf5ppjbrxfz5q8gzdsxbg6xrdx) | +| 2024-10-18 | Standardize vended credentials in load results | [#10722](https://github.com/apache/iceberg/pull/10722) | [vote](https://lists.apache.org/thread/trvnplkbx9kkdxn3m6owwrm8dsc31dhf) | +| 2024-09-10 | Add scan planning endpoints | [#9695](https://github.com/apache/iceberg/pull/9695) | [vote](https://lists.apache.org/thread/7xtpt6m95qwrk8tc8h40p7c743218ylc) | +| 2024-08-28 | Add the `RemovePartitionSpecs` update | [#10846](https://github.com/apache/iceberg/pull/10846) | [vote](https://lists.apache.org/thread/5pt5g4q9zt856cpscob37y30mczlf9rx) | +| 2024-08-23 | Add endpoint discovery to the config response | [#10928](https://github.com/apache/iceberg/pull/10928) | [vote](https://lists.apache.org/thread/ptg7yoz0tv5c5mb0p1dbc75p9blfos7h) | +| 2024-08-16 | Require commits to fail on unknown updates and requirements | [#10848](https://github.com/apache/iceberg/pull/10848) | [vote](https://lists.apache.org/thread/99lo7stnprchjzosjcq9k3mns1mq8fwc) | +| 2024-07-15 | Fix property names for statistics and partition statistics | [#10662](https://github.com/apache/iceberg/pull/10662) | [vote](https://lists.apache.org/thread/tp4dpw6qj5fyhrh17197bf6fg4gj3rk6) | +| 2024-07-11 | Deprecate the `oauth/tokens` endpoint | [#10603](https://github.com/apache/iceberg/pull/10603) | [vote](https://lists.apache.org/thread/o4qmrm5jx50mk1mqws0t9f1z2op4gvvm) | diff --git a/site/nav.yml b/site/nav.yml index 2e52c16faf19..cb068286366b 100644 --- a/site/nav.yml +++ b/site/nav.yml @@ -98,7 +98,15 @@ nav: - Starrocks: https://docs.starrocks.io/en-us/latest/data_source/catalog/iceberg_catalog - Tinybird: https://www.tinybird.co/docs/forward/get-data-in/table-functions/iceberg - Trino: https://trino.io/docs/current/connector/iceberg.html - - Releases: releases.md + - Releases: + - Java: releases.md + - REST catalog spec: /rest-catalog-spec/#format-versioning + - Table spec: /spec/#format-versioning + - View spec: /view-spec/#format-versioning + - Puffin spec: /puffin-spec/#format-versioning + - AES GCM stream spec: /gcm-stream-spec/#format-versioning + - SQL UDF spec: /udf-spec/#format-versioning + - Mumbling bitmap spec: /mumbling-spec/#format-versioning - Project: - Contributing: contribute.md - Multi-engine support: multi-engine-support.md From 61ec7ad8d752b437b16ca6de6d30e98ba487deb5 Mon Sep 17 00:00:00 2001 From: pvary Date: Wed, 7 Oct 2026 18:34:36 +0200 Subject: [PATCH 2/5] Docs: Remove remaining spec changes from the Java release notes Remove the spec-only entries that were still listed in older Java releases: the AES GCM stream spec and the Puffin spec snapshot ID update in 1.2.0, the Puffin spec pull request in 0.14.0, and the format version 2 adoption vote in 0.12.0. Entries that also contain Java changes are kept intact. Generated-by: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- site/docs/releases.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/site/docs/releases.md b/site/docs/releases.md index 33ca3d408b13..49adacfb52c0 100644 --- a/site/docs/releases.md +++ b/site/docs/releases.md @@ -1332,14 +1332,12 @@ The 1.2.0 release adds a variety of new features and bug fixes. Here is an overview: * Core - - Added AES GCM encryption stream spec ([\#5432](https://github.com/apache/iceberg/pull/5432)) - Added support for Delta Lake to Iceberg table conversion ([\#6449](https://github.com/apache/iceberg/pull/6449), [\#6880](https://github.com/apache/iceberg/pull/6880)) - Added support for `position_deletes` metadata table ([\#6365](https://github.com/apache/iceberg/pull/6365), [\#6716](https://github.com/apache/iceberg/pull/6716)) - Added support for scan and commit metrics reporter that is pluggable through catalog ([\#6404](https://github.com/apache/iceberg/pull/6404), [\#6246](https://github.com/apache/iceberg/pull/6246), [\#6410](https://github.com/apache/iceberg/pull/6410)) - Added support for branch commit for all operations ([\#4926](https://github.com/apache/iceberg/pull/4926), [\#5010](https://github.com/apache/iceberg/pull/5010)) - Added `FileIO` support for ORC readers and writers ([\#6293](https://github.com/apache/iceberg/pull/6293)) - Updated all actions to leverage bulk delete whenever possible ([\#6682](https://github.com/apache/iceberg/pull/6682)) - - Updated snapshot ID definition in Puffin spec to support statistics file reuse ([\#6272](https://github.com/apache/iceberg/pull/6267)) - Added human-readable metrics information in `files` metadata table ([\#5376](https://github.com/apache/iceberg/pull/5376)) - Fixed incorrect Parquet row group skipping when min and max values are `NaN` ([\#6517](https://github.com/apache/iceberg/pull/6517)) - Fixed a bug that location provider could generate paths with double slash (`//`) which is not compatible in a Hadoop file system ([\#6777](https://github.com/apache/iceberg/pull/6777)) @@ -1481,7 +1479,7 @@ Apache Iceberg 0.14.0 was released on 16 July 2022. - Added optional interfaces to FileIO, for batch deletes ([#4052](https://github.com/apache/iceberg/pull/4052)), prefix operations ([#5096](https://github.com/apache/iceberg/pull/5096)), and ranged reads ([#4608](https://github.com/apache/iceberg/pull/4608)) * Core - Added a common client for REST-based catalog services that uses a change-based protocol ([#4320](https://github.com/apache/iceberg/pull/4320), [#4319](https://github.com/apache/iceberg/pull/4319)) - - Added Puffin, a file format for statistics and index payloads or sketches ([#4944](https://github.com/apache/iceberg/pull/4944), [#4537](https://github.com/apache/iceberg/pull/4537)) + - Added Puffin, a file format for statistics and index payloads or sketches ([#4537](https://github.com/apache/iceberg/pull/4537)) - Added snapshot references to track tags and branches ([#4019](https://github.com/apache/iceberg/pull/4019)) - ManageSnapshots now supports multiple operations using transactions, and added branch and tag operations ([#4128](https://github.com/apache/iceberg/pull/4128), [#4071](https://github.com/apache/iceberg/pull/4071)) - ReplacePartitions and OverwriteFiles now support serializable isolation ([#2925](https://github.com/apache/iceberg/pull/2925), [#4052](https://github.com/apache/iceberg/pull/4052)) @@ -1804,7 +1802,6 @@ Apache Iceberg 0.12.0 was released on August 15, 2021. It consists of 395 commit **Other notable changes:** -* The Iceberg Community [voted to approve](https://mail-archives.apache.org/mod_mbox/iceberg-dev/202107.mbox/%3cCAMwmD1-k1gnShK=wQ0PD88it6cg9mY7Y1hKHjDZ7L-jcDzpyZA@mail.gmail.com%3e) version 2 of the Apache Iceberg Format Specification. The differences between version 1 and 2 of the specification are documented [here](spec.md#version-2). * Bugfixes and stability improvements for NessieCatalog. * Improvements and fixes for Iceberg's Python library. * Added a vectorized reader for Apache Arrow [[\#2286](https://github.com/apache/iceberg/pull/2286)]. From c1affa4023b2060b4872b379d56e342722e0b86d Mon Sep 17 00:00:00 2001 From: pvary Date: Wed, 7 Oct 2026 22:33:56 +0200 Subject: [PATCH 3/5] Docs: Revert spec and navigation changes, keep release notes cleanup Following the review decision, this PR only removes the spec changes from the Java release notes. The format versioning tables and changes since adoption sections in the specs, the REST Catalog spec page and the Releases menu changes are reverted. Generated-by: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- format/expressions-spec.md | 6 ---- format/gcm-stream-spec.md | 6 ---- format/mumbling-spec.md | 6 ---- format/puffin-spec.md | 6 ---- format/spec.md | 33 ++------------------- format/udf-spec.md | 12 -------- format/view-spec.md | 6 ---- site/docs/rest-catalog-spec.md | 53 ---------------------------------- site/nav.yml | 10 +------ 9 files changed, 4 insertions(+), 134 deletions(-) diff --git a/format/expressions-spec.md b/format/expressions-spec.md index faf81f2e330f..2975648a258e 100644 --- a/format/expressions-spec.md +++ b/format/expressions-spec.md @@ -24,12 +24,6 @@ This document defines the structure and behavior of expressions for use in Icebe Stored expressions are needed for use cases like data validations (`CHECK` constraints) and default values (for instance, `current_timestamp()`). Expressions are exchanged in use cases like server-side scan planning in the catalog protocol. -## Format Versioning - -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|---------|------------|--------------------------------------------------------------------------|------------------------| -| 1 | Adopted | 2026-06-30 | [vote](https://lists.apache.org/thread/6wfmjhgthlykwbk3f7df4zgcm40xtm2o) | - | - ## Overview diff --git a/format/gcm-stream-spec.md b/format/gcm-stream-spec.md index 30e931fa9dd4..8168780e118b 100644 --- a/format/gcm-stream-spec.md +++ b/format/gcm-stream-spec.md @@ -20,12 +20,6 @@ title: "AES GCM Stream Spec" # AES GCM Stream file format extension -## Format Versioning - -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|---------|------------|------|------------------------| -| 1 | Adopted | 2022-11-27 | - | - | - ## Background and Motivation Iceberg supports a number of data file formats. Two of these formats (Parquet and ORC) have built-in encryption capabilities, that allow to protect sensitive information in the data files. However, besides the data files, Iceberg tables also have metadata files, that keep sensitive information too (e.g., min/max values in manifest files, or bloom filter bitsets in puffin files). Metadata file formats (AVRO, JSON, Puffin) don't have encryption support. diff --git a/format/mumbling-spec.md b/format/mumbling-spec.md index 109b6f361d2b..84012bc847db 100644 --- a/format/mumbling-spec.md +++ b/format/mumbling-spec.md @@ -28,12 +28,6 @@ This spec is for version 1. [roaring]: https://roaringbitmap.org/ -## Format Versioning - -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|--------|---------|------|------------| -| 1 | Draft | | | | - ## Overview Mumbling bitmaps are based on the same idea as Roaring bitmaps: a bitmap is diff --git a/format/puffin-spec.md b/format/puffin-spec.md index 6d7ef04977cc..8617ae6d8e96 100644 --- a/format/puffin-spec.md +++ b/format/puffin-spec.md @@ -27,12 +27,6 @@ Puffin file contains arbitrary pieces of information (here called "blobs"), along with metadata necessary to interpret them. The blobs supported by Iceberg are documented at [Blob types](#blob-types). -## Format Versioning - -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|---------|------------|--------------------------------------------------------------------------|------------------------| -| 1 | Adopted | 2022-06-23 | [vote](https://lists.apache.org/thread/950rz31y3kr3kz0zzncwokvgzbrmmz4q) | - | - ## Format specification A file conforming to the Puffin file format specification should have the structure diff --git a/format/spec.md b/format/spec.md index 17f7390e162d..1510c0dd5ac4 100644 --- a/format/spec.md +++ b/format/spec.md @@ -24,12 +24,9 @@ This is a specification for the Iceberg table format that is designed to manage ## Format Versioning -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|----------------|------------|--------------------------------------------------------------------------|---------------------------------------------------| -| 4 | In development | | | | -| 3 | Adopted | 2025-05-23 | [vote](https://lists.apache.org/thread/9oncq0j0222nrm6scvm89xdq3gozg24p) | [Changes since adoption](#changes-since-adoption) | -| 2 | Adopted | 2021-08-02 | [vote](https://lists.apache.org/thread/ws2gg52d124p7bx9jgrn3kctrtfgtltp) | [Changes since adoption](#changes-since-adoption) | -| 1 | Adopted | - | - | - | +Versions 1, 2 and 3 of the Iceberg spec are complete and adopted by the community. + +**Version 4 is under active development and has not been formally adopted.** The format version number is incremented when new features are added that will break forward-compatibility---that is, when older readers would not read newer table features correctly. Tables may continue to be written with an older version of the spec to ensure compatibility by not using features that are not yet implemented by processing engines. @@ -2154,27 +2151,3 @@ The Geometry and Geography class hierarchy and its Well-known text (WKT) and Wel Points are always defined by the coordinates X, Y, Z (optional), and M (optional), in this order. X is the longitude/easting, Y is the latitude/northing, and Z is usually the height, or elevation. M is a fourth optional dimension, for example a linear reference value (e.g., highway milepost value), a timestamp, or some other value as defined by the CRS. The version of the OGC standard first used here is 1.2.1, but future versions may also be used if the WKB representation remains wire-compatible. - -## Changes since adoption - -Each change is listed under the most recent version it affects. - -### Version 3 - -| Date | Change | Pull request | Vote | -|------------|---------------------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| -| 2026-08-19 | Clarify content file uniqueness within a snapshot | [#17198](https://github.com/apache/iceberg/pull/17198) | [vote](https://lists.apache.org/thread/orwny91fqyfmzx7w7p6wj7cgmndvd8r3) | -| 2026-06-13 | Clarify the result type of the day partition transform in manifests | [#16446](https://github.com/apache/iceberg/pull/16446) | [vote](https://lists.apache.org/thread/gz432tvboxvno2v7g3l17c8tbtxckxrb) | -| 2026-05-18 | Clarify conventions for non-default CRS in geospatial types | [#15834](https://github.com/apache/iceberg/pull/15834) | [vote](https://lists.apache.org/thread/8z0kc8vz1379lghwlcwl0jmsoym6shd7) | -| 2025-10-14 | Clarify restrictions for geometry types | [#14250](https://github.com/apache/iceberg/pull/14250) | [vote](https://lists.apache.org/thread/v96xj6kxwclyl52d97dkcvzxytfypyp3) | -| 2025-09-18 | Bring back `added-rows` in snapshot fields | [#14048](https://github.com/apache/iceberg/pull/14048) | [vote](https://lists.apache.org/thread/5gff76hvqy7r1lksrt9ld9h3pdp03nb5) | -| 2025-07-31 | Correct the type of `snapshot-id` in table statistics | [#13513](https://github.com/apache/iceberg/pull/13513) | [vote](https://lists.apache.org/thread/z6loky5225j1vszorm199ywwncjwf6qz) | - -### Version 2 - -| Date | Change | Pull request | Vote | -|------------|-----------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| -| 2025-09-15 | Deprecate position delete files with row data | [#14045](https://github.com/apache/iceberg/pull/14045) | [vote](https://lists.apache.org/thread/tfy96bqmz1bmdxr73x17w3xxj3yzs606) | -| 2025-03-06 | Add an implementation note on `current-snapshot-id` | [#12334](https://github.com/apache/iceberg/pull/12334) | [vote](https://lists.apache.org/thread/54r4nm7qmr4vxhdpwmbx5rntynspskl7) | -| 2025-01-24 | Document optional snapshot summary fields | [#11660](https://github.com/apache/iceberg/pull/11660) | [vote](https://lists.apache.org/thread/mz01jwt69osqxhx9d3dd9xzncv9yncd0) | -| 2024-08-04 | Clarify file system tables | [#10833](https://github.com/apache/iceberg/pull/10833) | [vote](https://lists.apache.org/thread/1jm9zd6rtgjsvly3x099wvgdbohjn29v) | diff --git a/format/udf-spec.md b/format/udf-spec.md index 58958a7fa079..70ed53a5dbe6 100644 --- a/format/udf-spec.md +++ b/format/udf-spec.md @@ -20,12 +20,6 @@ title: "SQL UDF Spec" # Iceberg UDF Spec -## Format Versioning - -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|---------|------------|--------------------------------------------------------------------------|---------------------------------------------------| -| 1 | Adopted | 2026-02-05 | [vote](https://lists.apache.org/thread/whbgoc325o99vm4b599f0g1owhgww2kx) | [Changes since adoption](#changes-since-adoption) | - ## Background and Motivation A SQL user-defined function (UDF or UDTF) is a callable routine that accepts input parameters and executes a function body. @@ -381,9 +375,3 @@ RETURN SELECT name, color FROM fruits WHERE color = c; "secure": false } ``` - -## Changes since adoption - -| Date | Change | Pull request | Vote | -|------------|----------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| -| 2026-07-01 | Add optional `specific-name` to the UDF definition model | [#16727](https://github.com/apache/iceberg/pull/16727) | [vote](https://lists.apache.org/thread/75xs4trcgcpog8b2zoxpknf8bq6vfk4c) | diff --git a/format/view-spec.md b/format/view-spec.md index 46789ac8634c..34a671454778 100644 --- a/format/view-spec.md +++ b/format/view-spec.md @@ -20,12 +20,6 @@ title: "View Spec" # Iceberg View Spec -## Format Versioning - -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|---------|------------|------|------------------------| -| 1 | Adopted | 2022-04-03 | - | - | - ## Background and Motivation Most compute engines (e.g. Trino and Apache Spark) support views. A view is a logical table that can be referenced by future queries. Views do not contain any data. Instead, the query stored by the view is executed every time the view is referenced by another query. diff --git a/site/docs/rest-catalog-spec.md b/site/docs/rest-catalog-spec.md index 57d4a2cba5f9..3cf0cc01b971 100644 --- a/site/docs/rest-catalog-spec.md +++ b/site/docs/rest-catalog-spec.md @@ -25,12 +25,6 @@ Iceberg defines a REST-based Catalog API for managing table metadata and perform You can also explore the API interactively using the [Swagger UI](https://editor-next.swagger.io/?url=https://raw.githubusercontent.com/apache/iceberg/main/open-api/rest-catalog-open-api.yaml). -## Format Versioning - -| Version | Status | Adopted | Vote | Changes since adoption | -|---------|---------|---------|------|---------------------------------------------------| -| 1 | Adopted | - | - | [Changes since adoption](#changes-since-adoption) | - ## REST Catalog Protocol As the Iceberg project grew to support more languages and engines, pluggable catalogs started to cause some practical problems. Catalogs needed to be implemented in multiple languages and it proved difficult for commercial offerings to support many different catalogs and clients. @@ -46,50 +40,3 @@ The REST protocol is important for several reasons: - **Security**: The protocol supports secure table sharing using credential vending or remote signing. You can use the REST catalog protocol with any built-in catalog using translation in the `CatalogHandlers` class, or using the community maintained [`iceberg-rest-fixture`](https://hub.docker.com/r/apache/iceberg-rest-fixture) docker image. - -## Changes since adoption - -| Date | Change | Pull request | Vote | -|------------|--------------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------------------------| -| 2026-09-08 | Add catalog-provided `labels` to table and view load results | [#15750](https://github.com/apache/iceberg/pull/15750) | [vote](https://lists.apache.org/thread/do69l2nfm88m024ol34m2pdy0rqomqnz) | -| 2026-09-03 | Add finer grained read restrictions to `loadTable` | [#13879](https://github.com/apache/iceberg/pull/13879) | [vote](https://lists.apache.org/thread/0zloqhp8wkgyn04yg69j71cwcg5n7g74) | -| 2026-08-02 | Add the variant type | [#17256](https://github.com/apache/iceberg/pull/17256) | [vote](https://lists.apache.org/thread/gkj526shb0y96hdmms73sn3tvsw8mgw9) | -| 2026-07-30 | Formalize remote signing configuration | [#16822](https://github.com/apache/iceberg/pull/16822) | [vote](https://lists.apache.org/thread/n4hyqr3k2gnzbxvs3l1o4cz3ggdfwkx2) | -| 2026-07-27 | Align expressions with the expressions spec | [#17138](https://github.com/apache/iceberg/pull/17138) | [vote](https://lists.apache.org/thread/njo5bzgf4m126ly74ydz1pfznhz70h38) | -| 2026-06-06 | Add list and load function endpoints | [#15180](https://github.com/apache/iceberg/pull/15180) | [vote](https://lists.apache.org/thread/qd7jb3fdx65mcjvcjomrxg7rgxf3yn0m) | -| 2026-05-26 | Add the unregister table endpoint | [#16400](https://github.com/apache/iceberg/pull/16400) | [vote](https://lists.apache.org/thread/vy9fqyopp9c149oy40brdhb10grpq32g) | -| 2026-05-18 | Add the `CatalogObjectIdentifier` schema | [#16144](https://github.com/apache/iceberg/pull/16144) | [vote](https://lists.apache.org/thread/xyzlndcp4x7zq6xydpcbp192ov9p2skz) | -| 2026-04-22 | Clarify identifier uniqueness across tables and views | [#15691](https://github.com/apache/iceberg/pull/15691) | [vote](https://lists.apache.org/thread/2647zx7omfyocxl8og7s2tbomgprxvts) | -| 2026-04-15 | Add a 404 response to the config endpoint | [#15746](https://github.com/apache/iceberg/pull/15746) | [vote](https://lists.apache.org/thread/bcc62zxhc1rqjmwxxll81rpn8yfnr9fy) | -| 2026-03-31 | Promote the remote signing endpoint to the main spec | [#15450](https://github.com/apache/iceberg/pull/15450) | [vote](https://lists.apache.org/thread/rspzcx0o2mkm271fb8csghs4x98sfh3h) | -| 2026-03-13 | Add `scan-planning-mode` to the `loadTable` config | [#14867](https://github.com/apache/iceberg/pull/14867) | [vote](https://lists.apache.org/thread/mt4y8vw80glnqqwsr8czfmwvhqf09h46) | -| 2026-02-12 | Add `referenced-by` to `loadTable` | [#13810](https://github.com/apache/iceberg/pull/13810) | [vote](https://lists.apache.org/thread/jnv5f978gc530z0vx9h6l18dopoltm7j) | -| 2026-02-09 | Add access delegation to `registerTable` | [#15231](https://github.com/apache/iceberg/pull/15231) | [vote](https://lists.apache.org/thread/fgpwnsbslwkm4qxh10y6xn52pprbk274) | -| 2026-02-09 | Add access delegation to the scan planning endpoints | [#14781](https://github.com/apache/iceberg/pull/14781) | [vote](https://lists.apache.org/thread/80dm5b8rxcnqwfqwqoc43grjg8hktqgf) | -| 2026-01-17 | Add the register view endpoint | [#14869](https://github.com/apache/iceberg/pull/14869) | [vote](https://lists.apache.org/thread/520or6svg2yxcthbjc2wt5c9lbx0ntpz) | -| 2025-12-19 | Add `ETag` to `CommitTableResponse` | [#14760](https://github.com/apache/iceberg/pull/14760) | [vote](https://lists.apache.org/thread/fvjrkgm29ohh1f222opps0xnowgo8zz3) | -| 2025-12-08 | Add idempotency keys to the mutating scan planning endpoints | [#14730](https://github.com/apache/iceberg/pull/14730) | [vote](https://lists.apache.org/thread/6xo0v5bnvzq0ktd7tptszgm0csjp2p80) | -| 2025-12-05 | Make the namespace separator configurable by the server | [#14448](https://github.com/apache/iceberg/pull/14448) | [vote](https://lists.apache.org/thread/8k777v9l2lr7930g8fxrwjqnh5csl48k) | -| 2025-11-21 | Add `min-rows-requested` to scan planning requests | [#14565](https://github.com/apache/iceberg/pull/14565) | [vote](https://lists.apache.org/thread/t210ccrcz7fglv3vlwvnz827hb65217d) | -| 2025-11-20 | Add storage credentials to scan planning results | [#14563](https://github.com/apache/iceberg/pull/14563) | [vote](https://lists.apache.org/thread/vq6nzsc0vs81r8dhr94dp222s0gq1ykw) | -| 2025-11-19 | Add the `Idempotency-Key` header | [#14196](https://github.com/apache/iceberg/pull/14196) | [vote](https://lists.apache.org/thread/ybo8xst7h2p8vt6nctxf4nxfc0jbc6rq) | -| 2025-11-18 | Add `planId` to the credentials endpoint | [#14519](https://github.com/apache/iceberg/pull/14519) | [vote](https://lists.apache.org/thread/002y8ndg0s5l7vswkpr4nlr31ld3myg3) | -| 2025-08-21 | Mark 503 as non-retryable for table updates | [#13619](https://github.com/apache/iceberg/pull/13619) | [vote](https://lists.apache.org/thread/xno8c88sm4mlt0y5yw7p3tsob8wmhgkg) | -| 2025-05-27 | Add row lineage fields | [#13010](https://github.com/apache/iceberg/pull/13010) | [vote](https://lists.apache.org/thread/1q3xkvn2373n3g43qjjhnzj7p0v3d9p9) | -| 2025-05-12 | Add encryption key updates | [#12987](https://github.com/apache/iceberg/pull/12987) | [vote](https://lists.apache.org/thread/l055gx3spwy2n24kthy38fhoj2qy1tq7) | -| 2025-03-20 | Clarify dropping non-empty namespaces | [#12518](https://github.com/apache/iceberg/pull/12518) | [vote](https://lists.apache.org/thread/lxfbrv32wwn1tkvc37dmor2f8rbkrv2c) | -| 2025-02-17 | Add the overwrite option to table registration | [#12239](https://github.com/apache/iceberg/pull/12239) | [vote](https://lists.apache.org/thread/bm9x6f9lxh2901j2scwoqs2gf6mlm6yt) | -| 2025-02-14 | Add the `RemoveSchemas` update | [#12022](https://github.com/apache/iceberg/pull/12022) | [vote](https://lists.apache.org/thread/n97qo5nb09ggz5591g7cp99767fv5cf7) | -| 2025-01-28 | Add initial and write defaults to schema fields | [#12094](https://github.com/apache/iceberg/pull/12094) | [vote](https://lists.apache.org/thread/k606d81cv8h9f0oxtv5wzwg7qtb8mgxc) | -| 2025-01-27 | Add freshness-aware table loading | [#11946](https://github.com/apache/iceberg/pull/11946) | [vote](https://lists.apache.org/thread/cfxbdzs0rfopjg8qy101qc9vsokn75p5) | -| 2025-01-27 | Deprecate `snapshot-id` in `SetStatisticsUpdate` | [#12010](https://github.com/apache/iceberg/pull/12010) | [vote](https://lists.apache.org/thread/lcjvzp8o1n6bvvhbr1lo0zx7w9ncs1pm) | -| 2024-11-25 | Deprecate `last-column-id` in `AddSchemaUpdate` | [#11514](https://github.com/apache/iceberg/pull/11514) | [vote](https://lists.apache.org/thread/of8x50751vy3hv3xk3gcgncp4jry8z6x) | -| 2024-11-02 | Add deletion vector fields to delete files | [#11240](https://github.com/apache/iceberg/pull/11240) | [vote](https://lists.apache.org/thread/gxkkpvv2n69q8xq4n3k83ykp11cb1kzn) | -| 2024-10-24 | Add the credentials refresh endpoint | [#11281](https://github.com/apache/iceberg/pull/11281) | [vote](https://lists.apache.org/thread/jkpy0zhf5ppjbrxfz5q8gzdsxbg6xrdx) | -| 2024-10-18 | Standardize vended credentials in load results | [#10722](https://github.com/apache/iceberg/pull/10722) | [vote](https://lists.apache.org/thread/trvnplkbx9kkdxn3m6owwrm8dsc31dhf) | -| 2024-09-10 | Add scan planning endpoints | [#9695](https://github.com/apache/iceberg/pull/9695) | [vote](https://lists.apache.org/thread/7xtpt6m95qwrk8tc8h40p7c743218ylc) | -| 2024-08-28 | Add the `RemovePartitionSpecs` update | [#10846](https://github.com/apache/iceberg/pull/10846) | [vote](https://lists.apache.org/thread/5pt5g4q9zt856cpscob37y30mczlf9rx) | -| 2024-08-23 | Add endpoint discovery to the config response | [#10928](https://github.com/apache/iceberg/pull/10928) | [vote](https://lists.apache.org/thread/ptg7yoz0tv5c5mb0p1dbc75p9blfos7h) | -| 2024-08-16 | Require commits to fail on unknown updates and requirements | [#10848](https://github.com/apache/iceberg/pull/10848) | [vote](https://lists.apache.org/thread/99lo7stnprchjzosjcq9k3mns1mq8fwc) | -| 2024-07-15 | Fix property names for statistics and partition statistics | [#10662](https://github.com/apache/iceberg/pull/10662) | [vote](https://lists.apache.org/thread/tp4dpw6qj5fyhrh17197bf6fg4gj3rk6) | -| 2024-07-11 | Deprecate the `oauth/tokens` endpoint | [#10603](https://github.com/apache/iceberg/pull/10603) | [vote](https://lists.apache.org/thread/o4qmrm5jx50mk1mqws0t9f1z2op4gvvm) | diff --git a/site/nav.yml b/site/nav.yml index cb068286366b..2e52c16faf19 100644 --- a/site/nav.yml +++ b/site/nav.yml @@ -98,15 +98,7 @@ nav: - Starrocks: https://docs.starrocks.io/en-us/latest/data_source/catalog/iceberg_catalog - Tinybird: https://www.tinybird.co/docs/forward/get-data-in/table-functions/iceberg - Trino: https://trino.io/docs/current/connector/iceberg.html - - Releases: - - Java: releases.md - - REST catalog spec: /rest-catalog-spec/#format-versioning - - Table spec: /spec/#format-versioning - - View spec: /view-spec/#format-versioning - - Puffin spec: /puffin-spec/#format-versioning - - AES GCM stream spec: /gcm-stream-spec/#format-versioning - - SQL UDF spec: /udf-spec/#format-versioning - - Mumbling bitmap spec: /mumbling-spec/#format-versioning + - Releases: releases.md - Project: - Contributing: contribute.md - Multi-engine support: multi-engine-support.md From 2a5e40b9cbb897f11e3175c53ba5ebcf4b3a15d7 Mon Sep 17 00:00:00 2001 From: pvary Date: Wed, 7 Oct 2026 22:37:31 +0200 Subject: [PATCH 4/5] Docs: Remove spec changes from the 1.12.0 release notes Remove the Spec section and the spec-only Open API / REST entries from the 1.12.0 Java release notes, which were added on main after this change was started. The only remaining entry, the REST client path encoding fix, contains Java changes only, so it is moved to the Core section and the Open API / REST section is removed. Generated-by: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- site/docs/releases.md | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/site/docs/releases.md b/site/docs/releases.md index 49adacfb52c0..7578139423f0 100644 --- a/site/docs/releases.md +++ b/site/docs/releases.md @@ -95,15 +95,6 @@ The 1.12.0 release contains bug fixes and new features. For full release notes v Previously, default instances printed the bare type name (`geometry` / `geography`). - The default AWS SDK HTTP client migrated to Apache HttpClient 5. Users who provide AWS dependencies separately must switch from `software.amazon.awssdk:apache-client` to `software.amazon.awssdk:apache5-client` ([\#18195](https://github.com/apache/iceberg/pull/18195)). - The REST client now retries POST requests carrying an `Idempotency-Key` on retriable errors (408, 500, 502, 503, 504) ([\#17947](https://github.com/apache/iceberg/pull/17947)). -* Spec - - Add spec for expressions ([\#16652](https://github.com/apache/iceberg/pull/16652)) - - Add finer grained read restrictions as part of loadTable ([\#13879](https://github.com/apache/iceberg/pull/13879)) - - Add relative paths to the v4 spec ([\#15630](https://github.com/apache/iceberg/pull/15630)) - - Add content stats to the spec ([\#14234](https://github.com/apache/iceberg/pull/14234)) - - Add optional specific-name to UDF definition model ([\#16727](https://github.com/apache/iceberg/pull/16727)) - - Clarify variant type classification and primitive type scoping ([\#16836](https://github.com/apache/iceberg/pull/16836)) - - Clarify decimal type serialization ([\#16798](https://github.com/apache/iceberg/pull/16798)) - - Clarify content file uniqueness within a snapshot ([\#17198](https://github.com/apache/iceberg/pull/17198)) * API - Single-value binary serialization for geometry and geography ([\#16607](https://github.com/apache/iceberg/pull/16607)) - Define `RepairTable` action interface ([\#17399](https://github.com/apache/iceberg/pull/17399)) @@ -146,6 +137,7 @@ The 1.12.0 release contains bug fixes and new features. For full release notes v - Preserve manifest content pruning when ignoring residuals ([\#17443](https://github.com/apache/iceberg/pull/17443)) - Fix `SerializableTable.sortOrders()` throwing on historical sort orders with dropped fields ([\#16521](https://github.com/apache/iceberg/pull/16521)) - Fix `RESTMetricsReporter.report()` blocking the calling thread ([\#16695](https://github.com/apache/iceberg/pull/16695)) + - Fix REST path segment encoding to use RFC 3986 percent-encoding ([\#15989](https://github.com/apache/iceberg/pull/15989)) - Read catalog labels on the load table and view responses ([\#18045](https://github.com/apache/iceberg/pull/18045)) - Expose catalog labels on the loaded table via `SupportsLabels` ([\#18046](https://github.com/apache/iceberg/pull/18046)) - Add `max-file-group-input-files` to valid rewrite options ([\#17544](https://github.com/apache/iceberg/pull/17544)) @@ -237,16 +229,6 @@ The 1.12.0 release contains bug fixes and new features. For full release notes v - Track control topic offsets as a high-water mark ([\#17933](https://github.com/apache/iceberg/pull/17933)) - Add metric for partial commit failures ([\#16433](https://github.com/apache/iceberg/pull/16433)) - Fix coordinator committing files from a prior commit in certain rebalance scenarios ([\#17713](https://github.com/apache/iceberg/pull/17713)) -* Open API / REST - - Add VariantType to the REST catalog spec ([\#17256](https://github.com/apache/iceberg/pull/17256)) - - Add unregister table endpoint ([\#16400](https://github.com/apache/iceberg/pull/16400)) - - Add list and load function endpoints to the OpenAPI spec ([\#15180](https://github.com/apache/iceberg/pull/15180)) - - Formalize remote signing configuration ([\#16822](https://github.com/apache/iceberg/pull/16822)) - - Update REST spec expressions to match the new expressions spec ([\#17138](https://github.com/apache/iceberg/pull/17138)) - - Fix path segment encoding to use RFC 3986 percent-encoding ([\#15989](https://github.com/apache/iceberg/pull/15989)) - - Fix schema of data-access object in REST spec ([\#16594](https://github.com/apache/iceberg/pull/16594)) - - Add specific-name to UDF definition ([\#17364](https://github.com/apache/iceberg/pull/17364)) - - Add a `labels` field for catalog metadata enrichment ([\#15750](https://github.com/apache/iceberg/pull/15750)) * Vendor integrations - AWS: Use assumed-role credentials for REST SigV4 signing ([\#16794](https://github.com/apache/iceberg/pull/16794)) - AWS: Handle duplicate column names in IcebergToGlueConverter comment map ([\#16853](https://github.com/apache/iceberg/pull/16853)) From 05d1c59716d14f14ac18c1dca1695dfdecd70946 Mon Sep 17 00:00:00 2001 From: Kevin Liu Date: Tue, 13 Oct 2026 19:59:34 -0700 Subject: [PATCH 5/5] Docs: Keep release notes for spec PRs with Java changes #14045, #13619, and #10603 also change Java behavior (PositionDelete deprecation, REST 503 handling, OAuth2 endpoint warning), so restore them with wording that describes the Java change. Link the 1.1.0 Puffin statistics entry to the Java implementation (#4741) instead of the spec-only #4945. Generated-by: GitHub Copilot (Claude Opus 5.5) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- site/docs/releases.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/site/docs/releases.md b/site/docs/releases.md index 7578139423f0..3b9b2adea45a 100644 --- a/site/docs/releases.md +++ b/site/docs/releases.md @@ -269,6 +269,7 @@ Apache Iceberg 1.11.0 was released on May 19, 2026. The 1.11.0 release contains bug fixes and new features. For full release notes visit [Github](https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.11.0) * Deprecation / End of Support + - Core: Deprecate `PositionDelete` methods that set row data ([\#14045](https://github.com/apache/iceberg/pull/14045)) - Drop support for Java 11 ([\#14400](https://github.com/apache/iceberg/pull/14400)) - Spark 3.4: Deprecate support ([\#14099](https://github.com/apache/iceberg/pull/14099)) - Flink: Remove Flink 1.19 support ([\#13714](https://github.com/apache/iceberg/pull/13714)) @@ -554,6 +555,7 @@ The 1.10.0 release contains bug fixes and new features. For full release notes v - REST: Add property to configure user agent in http client ([\#13234](https://github.com/apache/iceberg/pull/13234)) - REST: add option to configure TLS settings in REST client ([\#13190](https://github.com/apache/iceberg/pull/13190)) - REST: Avoid table corruption by stop retrying on 502 and 504 ([\#13352](https://github.com/apache/iceberg/pull/13352)) + - REST: Retry 503 only when `Retry-After` is set, and treat 503 on commit as an unknown commit state ([\#13619](https://github.com/apache/iceberg/pull/13619)) - REST: Add HTTP proxy support for REST client ([\#12406](https://github.com/apache/iceberg/pull/12406)) - REST: make metrics reporting async ([\#13507](https://github.com/apache/iceberg/pull/13507)) - REST: allow retries for idempotent requests with some status codes ([\#13449](https://github.com/apache/iceberg/pull/13449)) @@ -944,6 +946,7 @@ The 1.6.0 release contains fixes, dependency updates, and new features (like Kaf - JDBC Catalog: Retry connections in JDBC catalog with user configured error code list ([\#10140](https://github.com/apache/iceberg/pull/10140)) - JDBC Catalog: fix namespace SQL statement using ESCAPE character working with MySQL, PostgreSQL, ... ([\#10167](https://github.com/apache/iceberg/pull/10167)) - REST Catalog: Assume issued_token_type is access_token to fully comply with RFC 6749 ([\#10314](https://github.com/apache/iceberg/pull/10314)) + - REST Catalog: Warn when `oauth2-server-uri` is not set and the client falls back to the deprecated `oauth/tokens` endpoint ([\#10603](https://github.com/apache/iceberg/pull/10603)) - REST Catalog: Fix create v1 table on REST Catalog ([\#10369](https://github.com/apache/iceberg/pull/10369)) - REST Catalog: Handles possible heap data corruption of OAuth2Util.AuthSession#headers ([\#10615](https://github.com/apache/iceberg/pull/10615)) - REST Catalog: Handles potential NPE in RESTSessionCatalog#newSessionCache ([\#10607](https://github.com/apache/iceberg/pull/10607)) @@ -1374,7 +1377,7 @@ and adds a variety of new features. Here is an overview: * Core - - Puffin statistics have been [added to the Table API](https://github.com/apache/iceberg/pull/4945) + - Puffin statistics have been [added to the Table API](https://github.com/apache/iceberg/pull/4741) - Support for [Table scan reporting](https://github.com/apache/iceberg/pull/5268), which enables collection of statistics of the table scans. - [Add file sequence number to ManifestEntry](https://github.com/apache/iceberg/pull/6002) - [Support register table](https://github.com/apache/iceberg/pull/5037) for all the catalogs (previously it was only for Hive)