Polaris Tag Management: tag assignment write path - #5469
Draft
flyingImer wants to merge 3 commits into
Draft
Conversation
flyingImer
force-pushed
the
tag/pr3-assignment
branch
from
September 8, 2026 21:50
e32be42 to
c223bd1
Compare
Add the OpenAPI specification for tag definitions and tag assignments to the Polaris Catalog API: tag definition CRUD, assign/unassign against a structured target, a reverse lookup from a definition to its directly assigned targets, and a per-target read with direct and effective views. Targets are catalog, namespace, table-like, and one top-level column. v1 supports Iceberg tables; Iceberg views are excluded because Iceberg view metadata does not maintain stable column field ids across replaces. Generic tables can be tagged as whole objects, but not their columns, since v1 defines no stable column identifier for them. The mappings route shape follows the existing Policy APIs so both features expose the same relationship model. This change registers the generated models only; the API resource is registered together with its service implementation. Related to apache#5442
Implement the definition side of the Tag Management API: create, list, load, update and drop a tag definition, with catalog-scoped authorization. - TagEntity: a TAG entity type parented by the catalog, storing comment, allowed-values and target-types in the entity properties map, following PolicyEntity's storage pattern; no persistence schema change. A live tag blocks dropping its catalog the same way live namespaces do. - Privileges TAG_CREATE/TAG_READ/TAG_DROP/TAG_WRITE/TAG_LIST/ TAG_FULL_METADATA and operations CREATE/LOAD/DROP/UPDATE/LIST_TAG, registered with the same RBAC semantics shape the policy operations use; CATALOG_MANAGE_CONTENT covers the tag privileges through the existing super-privilege mapping. - TagCatalogHandler/TagCatalog: authorize-then-delegate following PolicyCatalogHandler, resolving the tag as a catalog-child leaf the same way catalog roles resolve; updateTag enforces current-tag-version and treats target-types as create-only; a lost compare-and-swap surfaces as a retryable 409 conflict, matching the policy store; dropTag authorizes first and answers 501 for detach-all until tag assignments land; the assignment PR implements the two-privilege detach-all path. - Tag endpoints are gated by ENABLE_TAG_STORE, disabled by default, the same way the policy store is gated: the adapter enforces the flag and the catalog config contributor advertises tag endpoints only when it is enabled. Enable it on deployments backed by the JDBC or in-memory metastores; the NoSQL metastore does not support tags yet. - TagCatalogAdapter and the TagApi codegen registration land together, so the generated resource's service injection resolves; endpoints are advertised via the catalog config contributor; before/after events are emitted like other catalog APIs. - updateTag uses the generated UpdateTagRequest model. An omitted mutable field stays unchanged, and for the two nullable lists, allowed-values and target-types, an explicit JSON null is treated as omitted; a present allowed-values list replaces the whole list, an empty-string comment clears the comment, and a non-null target-types value, including an empty list, is rejected because target-types is create-only. A no-op update writes nothing and returns the tag with its current version token. - Named top-level string enums now generate with their declared wire values: TargetType previously serialized Java constant names, so table-like could never round-trip. Only enums whose names differ from their values change shape on the wire; all others are byte-identical. - Integration tests mirror the policy service tests across the in-memory and JDBC backends, including an order round-trip for allowed-values and negative tests for the documented definition-side errors reachable before assignments exist. Related to apache#5442
Adds the tag assignment write path end to end: a TagAssignmentRecord persistence interface with default-throw capability probes (mirroring the policy mapping persistence split), JDBC and in-memory implementations, a new JDBC schema v7 carrying the tag_assignment_record table with a fail-closed version gate for assignment writes, assignTag and unassignTag with two-securable authorization and column-to-field-id resolution, and an atomic combined delete backing dropTag with detach-all=true: no Tag API read observes the definition without its assignments or a partially removed state. Related to apache#5442
6 tasks
flyingImer
force-pushed
the
tag/pr3-assignment
branch
from
September 10, 2026 05:07
c223bd1 to
fff2ce1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tip
Until #5391 merges, the files view includes the spec commit from #5366 and the definition commit from #5391. For this slice's own diff, review the head commit: flyingImer@fff2ce1
GH proposal tracking: #5442
Why
#5391 makes Tag definitions usable, but nothing can be attached to them yet, and an authorized
detach-all=truedrop answers 501. This PR lets a tag be assigned to a catalog, a namespace, a table-like object, or a top-level Iceberg table column, one value per definition and target, validated against the allowed values, and unassigned again. Dropping a definition withdetach-all=truenow removes the definition and every assignment together, or nothing at all.Reads and reverse lookup are the next slice. Management grants for the new privileges are a later slice. NoSQL stays unsupported.
Scope
PUTandPOSTon/tags/{tag-name}/mappingsassign and unassign; assigning again replaces the stored value.TAG_ATTACHorTAG_DETACHon the definition, plus a target-side{CATALOG,NAMESPACE,TABLE}_{ATTACH,DETACH}_TAG.CATALOG_MANAGE_CONTENTcovers all of them. Tags classify resources only; they do not grant or deny access.tag_assignment_recordtable; existing databases need a one-time manual SQL, documented in the CHANGELOG and the metastore docs); writes on an older schema fail with an error naming the v7 requirement. In-memory is supported. NoSQL rejects assignment operations and tag drops explicitly. A backend that cannot guarantee the detach-all result answers 501 todetach-all=trueand changes nothing.ENABLE_TAG_STORE. Target deletion cleans up assignments best-effort on the shared entity drop path; leftover rows are hidden by the read slice.Extension points
Three mixin interfaces carry the assignment contract, threaded the same way
PolicyMappingPersistencewas:BasePersistencegainsTagAssignmentPersistence,TransactionalPersistencegainsTransactionalTagAssignmentPersistence, andPolarisMetaStoreManagergainsPolarisTagAssignmentManager. Every persistence method, including thedetach-allcapability, is adefaultthat reports "not supported", so an out-of-tree backend keeps compiling and stays fail-closed until it opts in. The manager'sassignTagToEntityandunassignTagFromEntityare abstract, the same shape asPolarisPolicyMappingManager's attach and detach. A note describing this delta is posted on the design thread with this PR.Validation
The integration test covers assign, unassign and detach-all end to end on JDBC and in-memory, replacement and grandfathering, malformed targets answering 400 and the value bound. The authorization test covers the privilege matrix and the detach-all two-privilege rule. The schema gate test pins the write rejection below v7. The JDBC unit tests inject a failure between the two deletes of the atomic drop and assert the rollback, and drive the two-writer conflict cases for the definition check and the first-write race. The shared metastore contract runs against PostgreSQL at every schema version.
This PR will remain Draft until #5391 merges and the rebased SHA passes CI.
Context
AI-assisted contribution
AI assistance was used for implementation and consistency review. I reviewed the final behavior and diff and take full responsibility for the contribution.
Checklist
CHANGELOG.mdsite/content/in-dev/unreleased