Polaris Catalog Spec: Tag Management APIs by flyingImer · Pull Request #5366 · apache/polaris

flyingImer · 2026-08-25T00:01:08Z

Tip

First of four Tag slices. Definitions: #5391. Assignments: #5469. Reads and discovery follow. Management grants for Tag privileges are a later slice.

GH proposal tracking: #5442

Why

Polaris has no catalog-native way to classify catalog objects, read direct or inherited classifications, or discover objects by classification. This PR defines the Tag API contract so clients and backends agree on those semantics before an implementation fixes them.

The contract follows the Policy API shape where it fits: catalog-scoped definition CRUD, structured targets for catalogs, namespaces, table-like objects and columns, assignment and unassignment, direct and effective reads for one target, and reverse lookup of direct assignments by Tag.

Scope

This PR ships the spec, its error vocabulary, and the generated models. target-types is create-only. A present, non-null allowed-values list replaces the whole list on update. Request-side cardinality and uniqueness stay Tag-specific server validation, so the generated DTOs keep the submitted order and any duplicates for the server to reject. apply-method is an open string, so later inheritance modes do not break generated clients.

Codegen registers models only. The API resource is registered together with its implementation in #5391.

The September 8 dev-list replies changed four things in this revision. Definition version tokens are opaque non-empty strings: a client returns the token unchanged, a stale token returns 409 even when the request would change nothing, and a backend keeps its own representation. getObjectTags joins namespace levels with U+001F and URI-encodes the joined value once, the Iceberg namespace query convention; empty, duplicate, and malformed values return 400. A successful detach-all=true makes the definition and its assignments invisible together, physical cleanup may finish later, and an implementation that cannot guarantee that result returns 501 before changing anything. Column identity is a logical column id, supplied in v1 by the Iceberg top-level field id, with no wire change.

Two encoding details are still open for list agreement: how strictly servers reject invalid UTF-8, and whether U+001F stays excluded from namespace level names.

V1 covers catalogs, namespaces, Iceberg and generic tables as whole objects, and top-level Iceberg columns. Views, generic-table columns, nested fields, and multi-value assignments are deferred. Tags classify resources; they do not grant or deny access in v1.

Validation

OpenAPI generation was inspected for the generated request model shapes and nullable update behavior, including the version fields now generating as String. The source and bundled specifications were compared after rebundling.

The local Gradle check run on this head is green. CI on this PR is the authoritative matrix.

Context

AI-assisted contribution

AI assistance was used for consistency review. I reviewed the final API contract and diff and take full responsibility for the contribution.

Checklist

🛡️ Don't disclose security issues! (contact security@apache.org)
🔗 Clearly explained why the changes are needed, with design and dev-list links
🧪 Added/updated tests with good coverage, or manually tested
💡 Added comments for complex logic (not applicable to this spec-only PR)
🧾 Updated CHANGELOG.md (not needed until runtime support is added)
📚 Updated documentation in site/content/in-dev/unreleased (not needed until runtime support is added)

Add the OpenAPI specification for tag definitions and tag assignments to the Polaris Catalog API: tag definition CRUD, assign/unassign against a structured target, a reverse lookup from a definition to its directly assigned targets, and a per-target read with direct and effective views. Targets are catalog, namespace, table-like, and one top-level column. v1 supports Iceberg tables; Iceberg views are excluded because Iceberg view metadata does not maintain stable column field ids across replaces. Generic tables can be tagged as whole objects, but not their columns, since v1 defines no stable column identifier for them. The mappings route shape follows the existing Policy APIs so both features expose the same relationship model. This change registers the generated models only; the API resource is registered together with its service implementation. Related to apache#5442

dimas-b

Thanks for your work on this @flyingImer !

This my my first API review round. I did not want to post too may comments upfront, but I'll probably have more later... depending on how the initial comment threads develop :)

dimas-b · 2026-09-10T14:34:46Z

+          $ref: '#/components/responses/ServiceUnavailableResponse'
+        5XX:
+          $ref: '#/components/responses/ServerErrorResponse'
+  /polaris/v1/{prefix}/tags:


So we have these base URIs ATM:
https://polaris.example.com/api/catalog/v1/{prefix}/ - IRC
https://polaris.example.com/api/management - Management API
https://polaris.example.com/api/polaris/v1/{prefix}/namespaces/{namespace}/generic-tables: - Generic Tables
https://polaris.example.com/api/polaris/v1/{prefix}/tags - Tags (this PR)

Some irksome points from my POV:

Tags and Generic Tables share the v1 path segment (they are path siblings), but they are unrelated APIs

tags is the identifying name for this API, but it is buried deep in the path. This will (potentially) make general-purpose proxies / gateways harder to configure.

Proposal:

https://polaris.example.com/api/tags/v1/{prefix}/

The tags API has a distinct prefix (can be remapped / forwarded)

v1 is specific to the tags API and can evolve to the next version independently of the Generic Tables API

The fact that the tags API is a "Polaris" API is implicit from the base URI. Even if the base URI does not have the word polaris in it, the user/admin must be aware that it refers to a Polaris server and the URI prefix distinguishes it from other APIs supported by Polaris.

Thanks, this one is worth pinning down before I touch the paths. One correction first: today the base is /api/catalog/polaris/v1/{prefix}/…, not /api/polaris/v1/…. Policies and generic tables use the same base, and the Iceberg REST API is /api/catalog/v1/{prefix}/…, so every catalog-scoped API currently sits under /api/catalog.

I can read your proposal as one of two shapes, and they cost very different things:

/api/catalog/tags/v1/{prefix}/…: tags gets its own segment and its own version inside the catalog service. Cheap (this PR plus the route constants in the follow-ups), but it adds a third pattern next to /api/catalog/v1 and /api/catalog/polaris/v1.

/api/tags/v1/{prefix}/…: a new top-level API family next to /api/catalog and /api/management. That needs its own spec document and generator wiring, and it moves a catalog-scoped API out of the catalog service.

We went over this on the design doc in August and I kept tags under the catalog API for consistency with policies and generic tables. If the project wants per-extension roots, I'd rather policies and generic tables move with it than have tags be the one exception. Which of the two did you have in mind, and should this PR lead that change or follow a project-wide decision?

Thanks for the base URI correction, @flyingImer !

My preference is /api/tags/v1/{prefix}/.

I do not think we should automatically move policies and generic tables APIs, but I'd be +1 to separating them into distinct REST prefixes eventually.

dimas-b · 2026-09-10T14:40:36Z

+      description: Return the tag definition names in the catalog identified by {prefix}, with pagination.
+      parameters:
+        - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-token'
+        - $ref: '../iceberg-rest-catalog-open-api.yaml#/components/parameters/page-size'


I believe it is preferable to avoid referencing IRC for pagination parameters. IRC pagination has "interesting" spec features, which had better be avoided in Polaris, IMHO.

See related discussion in #5282

Proposal:

Clients should always expect pagination to be in effect (no "get everything in one request")

Client-provided page size is a hint. Servers can reduce it at their discretion.

There's no need to submit empty page-token or any page-size - the server will choose a default page size.

dimas-b · 2026-09-10T14:47:52Z

+      value: {
+        "error": {
+          "message": "The given tag definition does not exist",
+          "type": "NoSuchTagException",


I believe the word "exception" is misleading here. REST errors do not have to be related to "exceptions". Some languages do not have exceptions at all.

dimas-b · 2026-09-10T14:49:09Z

+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/IcebergErrorResponse'


Let's use a Polaris type for error responses (even if it matches IRC responses field-by-field). This API != IRC and has a different evolution path.

dimas-b · 2026-09-10T14:51:22Z

+      value: {
+        "error": {
+          "message": "A tag definition with the given name already exists in the catalog",
+          "type": "AlreadyExistsException",


We should have a well-defined list of possible error types. I believe we will eventually have more than one error case mapped to the same HTTP response code (e.g. 409). Clients need to be able to distinguish them.

dimas-b · 2026-09-10T15:33:24Z

+                  $ref: '#/components/examples/NoSuchTagError'
+        501:
+          description: |
+            Not Implemented - this implementation cannot guarantee the detach-all=true result, so no


I suggest 422 with a specific error type in the payload. 501 feels line the server simply does not support something in principle ("for any resource" in RFC 9110 wording), but in this case, clearly, the server understood the request, but is unable to fulfill it according to requested parameters.

https://www.rfc-editor.org/info/rfc9110/#name-422-unprocessable-content

https://www.rfc-editor.org/info/rfc9110/#status.501

dimas-b · 2026-09-10T15:43:07Z

+        5XX:
+          $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse'
+
+  /polaris/v1/{prefix}/tags/{tag-name}/mappings:


.../assignments maybe?

A "mapping" feels like something static and unchanging.

dimas-b · 2026-09-10T15:44:35Z

+    post:
+      tags:
+        - Tag API
+      summary: Remove the assignment of a tag from a target


Why not DELETE /polaris/v1/{prefix}/tags/{tag-name}/assignments/namespaceA/tableB/columnC?

dimas-b · 2026-09-10T15:48:18Z

+    put:
+      tags:
+        - Tag API
+      summary: Assign a tag value to a target


Why not identify the target in the URI path and only use payload for the tag value? IMHO, it would be more natural from the REST design POV.

dimas-b · 2026-09-10T15:56:43Z

+        5XX:
+          $ref: '../iceberg-rest-catalog-open-api.yaml#/components/responses/ServerErrorResponse'
+
+  /polaris/v1/{prefix}/tags/{tag-name}/objects:


Why not reuse tags/{tag-name}/assignments for the GET method?

github-project-automation Bot added this to Basic Kanban Board Aug 25, 2026

github-project-automation Bot moved this to PRs In Progress in Basic Kanban Board Aug 25, 2026

flyingImer requested review from dimas-b, flyrain and jbonofre August 25, 2026 00:01

flyingImer force-pushed the tag/pr1-spec branch 2 times, most recently from 27b878e to 82134e0 Compare August 26, 2026 20:38

flyingImer mentioned this pull request Aug 26, 2026

Polaris Tag Management: tag definition CRUD end-to-end #5391

Draft

6 tasks

This was referenced Sep 3, 2026

Support key-value tag on namespace/table/column #5442

Open

Polaris Tag Management: tag assignment write path #5469

Draft

flyingImer force-pushed the tag/pr1-spec branch from 82134e0 to 1ee9642 Compare September 10, 2026 04:28

dimas-b reviewed Sep 10, 2026

View reviewed changes

Navigation Menu

Uh oh!

Polaris Catalog Spec: Tag Management APIs - #5366

Polaris Catalog Spec: Tag Management APIs#5366
flyingImer wants to merge 1 commit into
apache:mainfrom
flyingImer:tag/pr1-spec

flyingImer commented Aug 25, 2026 •

edited

Loading

Uh oh!

dimas-b left a comment

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

flyingImer Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026 •

edited

Loading

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

dimas-b Sep 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

flyingImer commented Aug 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

Scope

Validation

Context

AI-assisted contribution

Checklist

Uh oh!

dimas-b left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

flyingImer commented Aug 25, 2026 •

edited

Loading

dimas-b Sep 10, 2026 •

edited

Loading