RELATIONAL-JDBC: Add support for cockroach DB - #3352
Merged
Merged
Conversation
sfc-gh-prsingh
force-pushed
the
feature/cockroach
branch
from
January 5, 2026 05:04
5a57873 to
b25ebae
Compare
singhpk234
commented
Jan 5, 2026
jbonofre
self-requested a review
January 5, 2026 14:06
Contributor
Please use the standard |
dimas-b
reviewed
Jan 5, 2026
dimas-b
left a comment
dimas-b
left a comment
Contributor
There was a problem hiding this comment.
+1 to supporting CockroachDB and thanks for picking this work up, @singhpk234 !
Some comments below.
This was referenced Jan 6, 2026
|
This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
sfc-gh-prsingh
force-pushed
the
feature/cockroach
branch
from
February 26, 2026 07:48
0998281 to
56dd5c3
Compare
Contributor
|
@singhpk234, Thanks for adding more tests. There is a conflict and CI failures. Would you mind rebasing it? |
sfc-gh-prsingh
force-pushed
the
feature/cockroach
branch
from
March 4, 2026 09:21
56dd5c3 to
4e6112c
Compare
dimas-b
reviewed
Mar 4, 2026
dimas-b
reviewed
Mar 4, 2026
Contributor
Author
|
This is based on #3360 would be really to great this in first, if you all have time i added more tests there as well |
sfc-gh-prsingh
force-pushed
the
feature/cockroach
branch
from
March 4, 2026 19:53
f3a9d2d to
875a923
Compare
CockroachDB uses PostgreSQL wire protocol and is indistinguishable from PostgreSQL at the JDBC driver level. Compatibility is ensured through INT4 schema declarations and comprehensive test coverage. All PostgreSQL tests run against both PostgreSQL and CockroachDB.
- Created 5 new integration test classes that mirror PostgreSQL tests - CockroachApplicationIT, CockroachManagementServiceIT, CockroachPolicyServiceIT, CockroachRestCatalogIT, CockroachViewFileIT - All tests extend the same base test classes as PostgreSQL tests - Use CockroachRelationalJdbcProfile for CockroachDB container lifecycle - Tests run with same logic as PostgreSQL to ensure compatibility - Enabled parallel test execution (maxParallelForks = 2) - PostgreSQL and CockroachDB tests can now run concurrently This ensures comprehensive test coverage for CockroachDB backend.
CockroachDB Support: - Added COCKROACHDB to DatabaseType enum (maps to "cockroachdb" directory) - Created separate schema directory: cockroachdb/schema-v1.sql - CockroachDB schema v1 based on PostgreSQL schema v3 (includes all tables) - Uses INT4 explicitly for integer columns (required for CockroachDB JDBC driver) Database Type Detection and Validation: - Implemented DatabaseType.inferFromConnection() to detect database from JDBC metadata - If database-type is configured: uses configured type and validates against connection - If configured type mismatches connection: throws IllegalStateException with clear error - If no configured type: auto-detects from connection (CockroachDB, PostgreSQL, H2) - Validation ensures configuration matches actual database Configuration: - New property: polaris.persistence.relational.jdbc.database-type - Supported values: "postgresql", "cockroachdb", "h2" - If set, configuration is authoritative and validated against connection - If not set, auto-detects from JDBC connection metadata Bootstrap Support: - Added DatabaseType.getLatestSchemaVersion() method - PostgreSQL: latest version is 3 - CockroachDB: latest version is 1 - Updated JdbcBootstrapUtils to use database-specific latest version - Bootstrap automatically selects correct schema for each database type Test Configuration: - CockroachDB test profiles explicitly set database-type=cockroachdb - Ensures proper identification even with PostgreSQL JDBC driver - Added 5 integration test classes for server module - Enabled parallel test execution (PostgreSQL and CockroachDB tests run concurrently) PostgreSQL Schema: - Kept original INTEGER/INT types (no changes to existing schemas) - Removed trailing newlines from schema files
CockroachDB schema was versioned as v1 but its contents matched PostgreSQL v3 (including location_without_scheme column). This caused ModelEntity.getAllColumnNames(1) to return the wrong column list, silently skipping the location_without_scheme column. Rename schema-v1.sql to schema-v3.sql and update getLatestSchemaVersion() to return 3 for CockroachDB, keeping versions in sync with PostgreSQL. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add missing databaseType() to RelationalJdbcConfiguration implementations in MetricsReportPersistenceTest and RelationalJdbcIdempotencyStorePostgresIT - Override testBootstrapFailsWhenAddingRealmWithDifferentSchemaVersion in CockroachJdbcBootstrapCommandTest to use v3 (CockroachDB only has schema-v3)
- Create cockroachdb/schema-v4.sql matching PostgreSQL v4 with idempotency_records, scan_metrics_report, and commit_metrics_report tables (using INT4 for CockroachDB JDBC compatibility) - Remove cockroachdb/schema-v3.sql since CockroachDB is new and doesn't need older schema versions - Update DatabaseType.getLatestSchemaVersion() to return 4 for all types - Add missing database-type property to config docs - Update CockroachJdbcBootstrapCommandTest to bootstrap with v4
- Use proper imports for java.sql.Connection and java.sql.SQLException instead of inline fully-qualified names (review nit from dimas-b) - Update JdbcBootstrapUtilsTest expected versions from 3 to 4 - Add v4 test cases for parameterized schema version tests
Replace hardcoded Docker image in CockroachRelationalJdbcLifeCycleManagement with ContainerSpecHelper pattern, matching PostgreSQL's approach. This enables Renovate to manage the CockroachDB image version automatically.
sfc-gh-prsingh
force-pushed
the
feature/cockroach
branch
from
March 8, 2026 21:13
875a923 to
6a6277b
Compare
dimas-b
approved these changes
Mar 9, 2026
dimas-b
left a comment
dimas-b
left a comment
Contributor
There was a problem hiding this comment.
We should probably mention this in the CHANGELOG, WYDT?
github-project-automation
Bot
moved this from PRs In Progress
to Ready to merge
in Basic Kanban Board
flyrain
approved these changes
Mar 9, 2026
flyrain
left a comment
flyrain
left a comment
Contributor
There was a problem hiding this comment.
LGTM. Thanks @singhpk234 !
github-project-automation
Bot
moved this from Ready to merge
to Done
in Basic Kanban Board
Contributor
Author
Totally agree, creating a pr for it right away ! Thank you so much for review @dimas-b @flyrain marked @sathesuraj as the primary author : 8f9ff62 |
3 of 6 tasks
6 tasks
shyundev
pushed a commit
to shyundev/polaris
that referenced
this pull request
Aug 5, 2026
…hDB schemas The CockroachDB schema resources were forked from the PostgreSQL v4 schema in apache#3352. One week later, apache#3939 added three performance indexes to fix the realm-wide `grant_records` scan reported in apache#3685, but only to the H2 and PostgreSQL v4 schemas. apache#5086 then derived the v5 schemas from their v4 ancestors, carrying the omission forward. CockroachDB has therefore never declared them: idx_grants_realm_grantee ON grant_records (realm_id, grantee_id) idx_grants_realm_securable ON grant_records (realm_id, securable_id) idx_entities_catalog_id_id ON entities (catalog_id, id) This matters most for `idx_grants_realm_grantee`. The `grant_records` primary key leads with the securable columns, so `loadAllGrantRecordsOnGrantee` - issued on every resolver cache miss for a principal, principal role, or catalog role - has `realm_id` as its only usable key prefix and scans every grant record in the realm. On CockroachDB the primary key is also the physical storage and range split key, so that scan fans out across nodes. Diffing the two dialect files shows the change restores parity with the PostgreSQL schema the CockroachDB one was forked from: apart from the header and footer comments and the deliberate `INTEGER`/`INT` to `INT4` substitution, these three indexes were the only difference. `idx_locations` is byte-identical in both. No schema version bump: the indexes affect query planning only, every statement is `CREATE INDEX IF NOT EXISTS`, and no column, constraint, or query changes. This mirrors how apache#3939 itself landed - its commit message records that a proposed new schema version was moved into the existing file during review. `SchemaIndexParityTest` guards against this class of drift recurring. It compares only `(index name, table name)` pairs per schema version, across the dialects that ship that version. Column lists are deliberately not compared, because dialects legitimately differ - `idx_locations` is a partial index on PostgreSQL but a plain index on H2. Without the schema change the test fails on v4 parity, v5 parity, and the CockroachDB grant-record assertion, naming the three missing indexes. Because Polaris has no automated schema migrations, an existing CockroachDB database only picks these up when a new realm is bootstrapped, since bootstrapping is the only production path that runs the schema script. The CHANGELOG and the Relational JDBC metastore documentation carry the one-time `CREATE INDEX` statements for databases bootstrapped by an earlier release. Related to apache#3685
shyundev
added a commit
to shyundev/polaris
that referenced
this pull request
Aug 5, 2026
The CockroachDB schema resources were forked from the PostgreSQL v4 schema in apache#3352. apache#3939 then added three performance indexes to fix the realm-wide `grant_records` scan reported in apache#3685, but it was opened before the CockroachDB schema landed on main, so it only covered the H2 and PostgreSQL v4 schemas. The v5 schemas were later derived from their v4 ancestors, carrying the omission forward. CockroachDB has therefore never declared them: idx_grants_realm_grantee ON grant_records (realm_id, grantee_id) idx_grants_realm_securable ON grant_records (realm_id, securable_id) idx_entities_catalog_id_id ON entities (catalog_id, id) `idx_grants_realm_grantee` is the one that fixes an access path. After `realm_id`, the `grant_records` primary key continues with the securable columns, so `loadAllGrantRecordsOnGrantee` - issued on every resolver cache miss for a principal, principal role, or catalog role - has `realm_id` as its only usable key prefix and scans every grant record in the realm. On CockroachDB the primary key is also the physical storage and range split key, so that scan fans out across nodes. The other two are included so the dialects agree, not because a query needs them: `loadAllGrantRecordsOnSecurable` constrains a full primary key prefix, and `idx_entities` already leads with `realm_id` for the only query shaped like `idx_entities_catalog_id_id`. Diffing the two dialect files shows the change restores parity with the PostgreSQL schema the CockroachDB one was forked from: apart from comments, whitespace and the deliberate `INTEGER`/`INT` to `INT4` substitution, these three indexes were the only difference. No schema version bump. Schema v4 and v5 already declare these indexes for PostgreSQL and H2, so the CockroachDB files are brought in line with the version they already claim rather than a new version being introduced; a v6 would instead say the indexes are new, and leave CockroachDB v5 permanently short of its own contract. The trade-off is that v4 and v5 have both shipped, so a CockroachDB database bootstrapped by an earlier release reports the same version without these indexes. Because Polaris has no automated schema migrations, such a database only picks them up when a new realm is bootstrapped, since bootstrapping is the only production path that runs the schema script. The CHANGELOG and the Relational JDBC metastore documentation carry the one-time `CREATE INDEX` statements for that case. `SchemaIndexParityTest` guards against this class of drift recurring. It compares only `(index name, table name)` pairs per schema version, across the dialects that ship that version and declare standalone indexes. Column lists are deliberately not compared, so it proves the dialects agree on which indexes exist by name, not that they cover the same columns - today they do not, since `idx_locations` is declared on different columns on H2 than on PostgreSQL and CockroachDB. Related to apache#3685
4 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
About the change
Add support of cockroach DB in relational-jdbc persistence
There was pr long ago :
seems like the author got busy, all pending was to testing it E2E with cockroach with all the integ tests along with making it work with schema evolution logic. also marking them as co-author for their contribution.
Happy to close it if they wanna resume their original pr
co-author : @sathesuraj
Checklist
CHANGELOG.md(if needed) -- will add latersite/content/in-dev/unreleased(if needed) - will add later