Skip to content

JDBC: Use single DDL init script #5349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
adutra wants to merge 3 commits into
base: main
Choose a base branch
from
Open

JDBC: Use single DDL init script #5349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e484bce
JDBC: Replace per-version schema scripts with a single `schema.sql`
adutra Aug 22, 2026
6f26a55
review
adutra Aug 31, 2026
0668f5d
review
adutra Sep 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
WHERE location_without_scheme IS NOT NULL;
```
H2 is unaffected.
- Relational JDBC: The per-schema-version runtime fallback has been removed. The migration to schema
v6 is now **required** before starting this version of Polaris. The first request to any realm
whose recorded schema version does not match what the binary expects will fail fast with a clear
error message. See the [Relational JDBC metastore documentation] for the full upgrade path.

[Relational JDBC metastore documentation]:https://polaris.apache.org/releases/latest/metastores/relational-jdbc/#schema-upgrades

- Relational JDBC: schema version 6 also declares `idx_grants_realm_grantee`,
`idx_grants_realm_securable` and `idx_entities_catalog_id_id` on CockroachDB (see Fixes), which
Expand Down Expand Up @@ -72,6 +78,11 @@ request adding CHANGELOG notes for breaking (!) changes and possibly other secti
Either remove the setting from the URL, or point it at the schema that already holds your
Polaris tables.
- Internal JWTs minted before credentials-generation binding (tokens without the `polaris-cv` claim) can no longer be used as subject tokens in token exchange; they remain valid as bearer tokens until expiry. During a rolling upgrade, an old node may still mint claim-less tokens: exchanging such a token on any already-upgraded node fails with `invalid_grant`, so clients can see intermittent exchange failures until the last old node is gone; after that, rejection is consistent.
- Relational JDBC: Per-version schema scripts (`schema-v1.sql` through `schema-v5.sql`) have been
replaced by a single `schema.sql` that is safe to run on every startup. Per-version runtime
compatibility fallbacks and the `SCHEMA_VERSION_FALL_BACK_ON_DNE` configuration key have been
removed. Operators must ensure their database is at the right schema version before upgrading to
this version.

### New Features

Expand Down
4 changes: 2 additions & 2 deletions ...c/src/test/java/org/apache/polaris/extension/metrics/jdbc/JdbcMetricsPersistenceTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,9 @@ void setUp() throws SQLException {
DatasourceOperations datasourceOperations =
new DatasourceOperations(dataSource, new TestJdbcConfiguration());

// Execute main schema v4 (includes metrics tables)
// Execute schema script (includes metrics tables)
ClassLoader classLoader = DatasourceOperations.class.getClassLoader();
InputStream schemaStream = classLoader.getResourceAsStream("h2/schema-v5.sql");
InputStream schemaStream = classLoader.getResourceAsStream("h2/schema.sql");
datasourceOperations.executeScript(schemaStream);

RealmContext realmContext = () -> "TEST_REALM";
Expand Down
34 changes: 5 additions & 29 deletions ...ional-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/DatabaseType.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,18 +47,6 @@ public String getDisplayName() {
return displayName;
}

/**
* Returns the latest schema version available for this database type. This is used as the default
* schema version for new installations.
*/
public int getLatestSchemaVersion() {
return switch (this) {
case POSTGRES -> 6; // PostgreSQL has schemas v1, v2, v3, v4, v5, v6
case COCKROACHDB -> 6; // CockroachDB schema version kept in sync with PostgreSQL
case H2 -> 6; // H2 uses same schemas as PostgreSQL
};
}

public static DatabaseType fromDisplayName(String displayName) {
return switch (displayName.toLowerCase(Locale.ROOT)) {
case "h2" -> DatabaseType.H2;
Expand Down Expand Up @@ -145,29 +133,17 @@ public static DatabaseType inferFromConnection(
}

/**
* Open an InputStream that contains data from an init script. This stream should be closed by the
* caller.
* Open an InputStream that contains data from the init script. This stream should be closed by
* the caller.
*/
public InputStream openInitScriptResource(int schemaVersion) {
// Validate schema version is within acceptable range for this database type
int latestVersion = getLatestSchemaVersion();
if (schemaVersion <= 0 || schemaVersion > latestVersion) {
throw new IllegalArgumentException(
String.format(
"Invalid schema version %d for database type %s. Valid range: 1-%d",
schemaVersion, this, latestVersion));
}

final String resourceName =
String.format("%s/schema-v%d.sql", this.getDisplayName(), schemaVersion);
public InputStream openInitScriptResource() {
final String resourceName = String.format("%s/schema.sql", this.getDisplayName());
Comment thread
adutra marked this conversation as resolved.

ClassLoader classLoader = DatasourceOperations.class.getClassLoader();
InputStream stream = classLoader.getResourceAsStream(resourceName);
if (stream == null) {
throw new IllegalStateException(
String.format(
"Schema resource not found: %s (database type: %s, version: %d)",
resourceName, this, schemaVersion));
String.format("Schema resource not found: %s (database type: %s)", resourceName, this));
}
return stream;
}
Expand Down
60 changes: 60 additions & 0 deletions ...bc/src/main/java/org/apache/polaris/persistence/relational/jdbc/DatasourceOperations.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import java.util.Arrays;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.Random;
import java.util.concurrent.TimeUnit;
Expand All @@ -54,6 +55,15 @@ public class DatasourceOperations {

private static final Logger LOGGER = LoggerFactory.getLogger(DatasourceOperations.class);

/** The schema version this binary expects to find in the version table. */
// Whenever a schema change requires a migration:
// 1) Increment the version number in all schema.sql scripts
// (the version number MUST be the same on all scripts)
// 2) Increment this constant to match the version number in schema.sql
// 3) Add the corresponding migration SQL to the Relational JDBC metastore documentation
// 4) Add a note to the changelog
public static final int CURRENT_SCHEMA_VERSION = 6;

// PG STATUS CODES
// 23505 = unique key violation, consistent across PG/Cockroach/H2; other checks (FK, NOT NULL,
// CHECK) all propagate
Expand Down Expand Up @@ -460,6 +470,56 @@ public boolean isUniquenessConstraintViolation(SQLException e) {
return UNIQUENESS_CONSTRAINT_VIOLATION_SQL_CODE.equals(e.getSQLState());
}

/**
* Checks whether the version table reports the schema version this binary expects. Throws {@link
* IllegalStateException} on a mismatch, an absent version table, or an empty version table.
*/
public void validateSchemaCompatibility() {
PreparedQuery query = QueryGenerator.generateVersionQuery();
try {
List<Integer> versions =
executeSelect(
query,
new Converter<>() {
@Override
public Integer fromResultSet(ResultSet rs) throws SQLException {
return rs.getInt("version_value");
}

@Override
public Map<String, Object> toMap(DatabaseType databaseType) {
return Map.of();
}
});
if (versions.isEmpty()) {
throw new IllegalStateException(
"Version table exists but contains no rows. "
+ "The database may be in a corrupted state. "
+ "See the Relational JDBC metastore documentation for upgrade instructions.");
}
int version = versions.getFirst();
if (version != CURRENT_SCHEMA_VERSION) {
throw new IllegalStateException(
String.format(
"Incompatible JDBC schema version %d (expected %d). "
+ "Please apply the required DDL migration before starting Polaris. "
+ "See the Relational JDBC metastore documentation for upgrade instructions.",
version, CURRENT_SCHEMA_VERSION));
}
} catch (SQLException e) {
if (isRelationDoesNotExist(e)) {
throw new IllegalStateException(
"Version table not found. "
+ "Please run the bootstrap command before starting Polaris, "
+ "or apply the required DDL migration. "
+ "See the Relational JDBC metastore documentation for upgrade instructions.",
e);
}
throw new IllegalStateException(
"Could not validate JDBC schema compatibility: " + e.getMessage(), e);
}
}

public boolean isRelationDoesNotExist(SQLException e) {
return (RELATION_DOES_NOT_EXIST.equals(e.getSQLState())
&& (databaseType == DatabaseType.POSTGRES || databaseType == DatabaseType.COCKROACHDB))
Expand Down
Loading