Remove unused IdempotencyStore and idempotency_records table by huaxingao · Pull Request #5086 · apache/polaris

huaxingao · 2026-07-17T18:25:34Z

Removes the unused durable-replay idempotency implementation:

IdempotencyStore / IdempotencyRecord / IdempotencyPersistenceException (polaris-core)
RelationalJdbcIdempotencyStore / ModelIdempotencyRecord and its Postgres IT
The idempotency_records table + index from the postgres/cockroachdb/h2 schema-v4.sql
The two QueryGenerator extended overloads (only used by the store) and their tests

This code is superseded by the shipped entity-property idempotency (createTable/updateTable).

Checklist

🛡️ Don't disclose security issues! (contact security@apache.org)
🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
🧪 Added/updated tests with good coverage, or manually tested (and explained how)
💡 Added comments for complex logic
🧾 Updated CHANGELOG.md (if needed)
📚 Updated documentation in site/content/in-dev/unreleased (if needed)
[ ]

Copilot

Pull request overview

This PR removes the durable-replay, key-based idempotency implementation (store interface + JDBC backend + schema table) that is described as unused and superseded by entity-property idempotency in the wider Polaris codebase.

Changes:

Removed IdempotencyStore/IdempotencyRecord/IdempotencyPersistenceException from polaris-core.
Removed the relational-JDBC idempotency store implementation and its Postgres integration test.
Removed the idempotency_records table/index from the v4 SQL schemas and deleted the now-unused QueryGenerator extended overloads and related tests.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
polaris-core/src/main/java/org/apache/polaris/core/persistence/IdempotencyStore.java	Removed the core idempotency persistence abstraction.
polaris-core/src/main/java/org/apache/polaris/core/persistence/IdempotencyPersistenceException.java	Removed store-specific persistence exception type.
polaris-core/src/main/java/org/apache/polaris/core/entity/IdempotencyRecord.java	Removed the core domain record for durable-replay idempotency.
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/idempotency/RelationalJdbcIdempotencyStore.java	Removed the JDBC-backed `IdempotencyStore` implementation.
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/models/ModelIdempotencyRecord.java	Removed the JDBC model mapping for `idempotency_records`.
persistence/relational-jdbc/src/test/java/org/apache/polaris/persistence/relational/jdbc/idempotency/RelationalJdbcIdempotencyStorePostgresIT.java	Removed Postgres IT coverage for the deleted store.
persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/QueryGenerator.java	Removed extended DELETE/UPDATE overloads that were only used by the deleted store.
persistence/relational-jdbc/src/test/java/org/apache/polaris/persistence/relational/jdbc/QueryGeneratorTest.java	Removed tests that exercised the deleted extended overloads.
persistence/relational-jdbc/src/main/resources/postgres/schema-v4.sql	Removed `idempotency_records` DDL from Postgres v4 schema script.
persistence/relational-jdbc/src/main/resources/h2/schema-v4.sql	Removed `idempotency_records` DDL from H2 v4 schema script.
persistence/relational-jdbc/src/main/resources/cockroachdb/schema-v4.sql	Removed `idempotency_records` DDL from CockroachDB v4 schema script.

  /**
   * Builds an UPDATE query.
   *
   * @param allColumns Columns to update.
   * @param tableName Target table.
   * @param values New values (must match columns in order).
   * @param whereClause Conditions for filtering rows to update.
   * @return UPDATE query with parameter values.
   */
  public static PreparedQuery generateUpdateQuery(
      @NonNull List<String> allColumns,
      @NonNull String tableName,
      @NonNull List<Object> values,
      @NonNull Map<String, Object> whereClause) {
    List<Object> bindingParams = new ArrayList<>(values);
    QueryFragment where = generateWhereClause(new HashSet<>(allColumns), whereClause, Map.of());
    String setClause = allColumns.stream().map(c -> c + " = ?").collect(Collectors.joining(", "));
    String sql =
        "UPDATE " + getFullyQualifiedTableName(tableName) + " SET " + setClause + where.sql();
    bindingParams.addAll(where.parameters());
    return new PreparedQuery(sql, bindingParams);
  }


dimas-b

+1 to remove unused tables with one schema management comment below.

dimas-b · 2026-07-17T18:45:09Z

    PRIMARY KEY (event_id)
 );

-CREATE TABLE IF NOT EXISTS idempotency_records (


Schema v4 is part of the 1.6.0 release. I think its DDL should be immutable now.

Let's switch to v5 and not have idempotency tables there.

huaxingao · 2026-07-18T04:18:08Z

Thanks @flyrain @dimas-b

Upstream introduced schema v5 as the new latest version (apache#5086), which makes v4 a legacy version. Per the review agreement to leave legacy schema versions untouched, the v4 scripts are restored to their original form and the new v5 scripts drop the CREATE SCHEMA / SET search_path header instead, so the configurable schema name applies to the current schema version.

…hDB schemas The CockroachDB schema resources were forked from the PostgreSQL v4 schema in apache#3352. One week later, apache#3939 added three performance indexes to fix the realm-wide `grant_records` scan reported in apache#3685, but only to the H2 and PostgreSQL v4 schemas. apache#5086 then derived the v5 schemas from their v4 ancestors, carrying the omission forward. CockroachDB has therefore never declared them: idx_grants_realm_grantee ON grant_records (realm_id, grantee_id) idx_grants_realm_securable ON grant_records (realm_id, securable_id) idx_entities_catalog_id_id ON entities (catalog_id, id) This matters most for `idx_grants_realm_grantee`. The `grant_records` primary key leads with the securable columns, so `loadAllGrantRecordsOnGrantee` - issued on every resolver cache miss for a principal, principal role, or catalog role - has `realm_id` as its only usable key prefix and scans every grant record in the realm. On CockroachDB the primary key is also the physical storage and range split key, so that scan fans out across nodes. Diffing the two dialect files shows the change restores parity with the PostgreSQL schema the CockroachDB one was forked from: apart from the header and footer comments and the deliberate `INTEGER`/`INT` to `INT4` substitution, these three indexes were the only difference. `idx_locations` is byte-identical in both. No schema version bump: the indexes affect query planning only, every statement is `CREATE INDEX IF NOT EXISTS`, and no column, constraint, or query changes. This mirrors how apache#3939 itself landed - its commit message records that a proposed new schema version was moved into the existing file during review. `SchemaIndexParityTest` guards against this class of drift recurring. It compares only `(index name, table name)` pairs per schema version, across the dialects that ship that version. Column lists are deliberately not compared, because dialects legitimately differ - `idx_locations` is a partial index on PostgreSQL but a plain index on H2. Without the schema change the test fails on v4 parity, v5 parity, and the CockroachDB grant-record assertion, naming the three missing indexes. Because Polaris has no automated schema migrations, an existing CockroachDB database only picks these up when a new realm is bootstrapped, since bootstrapping is the only production path that runs the schema script. The CHANGELOG and the Relational JDBC metastore documentation carry the one-time `CREATE INDEX` statements for databases bootstrapped by an earlier release. Related to apache#3685

Remove unused IdempotencyStore and idempotency_records table

c06939f

Copilot AI review requested due to automatic review settings July 17, 2026 18:25

github-project-automation Bot added this to Basic Kanban Board Jul 17, 2026

github-project-automation Bot moved this to PRs In Progress in Basic Kanban Board Jul 17, 2026

Copilot started reviewing on behalf of huaxingao July 17, 2026 18:26 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

dimas-b reviewed Jul 17, 2026

View reviewed changes

add JDBC schema v5 without idempotency_records

2434e53

dimas-b approved these changes Jul 17, 2026

View reviewed changes

github-project-automation Bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Jul 17, 2026

dimas-b requested review from flyrain and singhpk234 July 17, 2026 20:22

flyrain approved these changes Jul 17, 2026

View reviewed changes

flyrain merged commit 7f05287 into apache:main Jul 18, 2026
25 checks passed

github-project-automation Bot moved this from Ready to merge to Done in Basic Kanban Board Jul 18, 2026

huaxingao deleted the remove-idempotency-records-store branch July 18, 2026 04:17

dimas-b mentioned this pull request Jul 18, 2026

feat(persistence): replace EventEntity.REALM_SCOPED sentinel with nullable catalog_id #4981

Merged

6 tasks done

Navigation Menu

Remove unused IdempotencyStore and idempotency_records table - #5086

Remove unused IdempotencyStore and idempotency_records table#5086
flyrain merged 2 commits into
apache:mainfrom
huaxingao:remove-idempotency-records-store

huaxingao commented Jul 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

dimas-b left a comment •

edited

Loading

Uh oh!

dimas-b Jul 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

huaxingao commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

huaxingao commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

dimas-b left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

huaxingao commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

huaxingao commented Jul 17, 2026 •

edited

Loading

dimas-b left a comment •

edited

Loading

dimas-b Jul 17, 2026 •

edited

Loading