Repository navigation
Core, Open-API: Add server-side warehouse validation for /v1/config endpoint - #16247
kevinjqliu wants to merge 7 commits into
Conversation
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
|
@gaborkaszab you've been in this part of the codebase, could you take a look? |
gaborkaszab
left a comment
There was a problem hiding this comment.
Thanks for the PR, @kevinjqliu !
I took a quick look and suggested some alternative ways to do this. Let me know what you think!
Now that we have both the client and server side of this, would be nice to see some tests in TestRESTCatalog for mismatching warehouse names and/or locations.
| @@ -31,6 +31,7 @@ private RESTCatalogProperties() {} | |||
| public static final String SNAPSHOT_LOADING_MODE = "snapshot-loading-mode"; | |||
| public static final SnapshotMode SNAPSHOT_LOADING_MODE_DEFAULT = SnapshotMode.ALL; | |||
| public static final String SNAPSHOTS_QUERY_PARAMETER = "snapshots"; | |||
There was a problem hiding this comment.
RESTSessionCatalog uses CatalogProperties.WAREHOUSE_LOCATION for the query param name. I'd recommend using the same param both at the client and the server side. My preference would be to replace the one in RESTSessionCatalog.fetchConfig() to use this one.
Additionally, when we populate this param on the client side for the request, the code comments talks about warehouse location. I'm a bit confused. Isn't this param a warehouse name, or is it a warehouse location?
| return castResponse(responseType, handleOAuthRequest(body)); | ||
|
|
||
| case CONFIG: | ||
| if (vars.containsKey(RESTCatalogProperties.WAREHOUSE_QUERY_PARAMETER)) { |
There was a problem hiding this comment.
nit: to keep this as simple as possible, can we simply do this and return early if the warehouse is null?
validateWarehouse(vars.get(RESTCatalogProperties.WAREHOUSE_QUERY_PARAMETER));
|
|
||
| /** Validates that the given warehouse exists. Subclasses may override. */ | ||
| protected void validateWarehouse(String warehouse) { | ||
| // no-op: accept any warehouse by default |
There was a problem hiding this comment.
The reason we leave the implementation to RESTServerCatalogAdapter is because it has the catalog configs that we need to check for equality with the warehouse query param, right?
This seems a bit odd to me. Can't we pass the catalog configs too to RESTCatalogAdapter? Maybe pass CatalogContext via a new constructor?
Other option could be what the applyCredentials does. It doesn't go through inheritance but we do the checks for response types and enhance with credentials if needed. If we have to keep the warehouse checks on this level of abstraction and not pass the catalog configs to RESTCatalogAdapter, we might want to do the same: check for ConfigResponse and validate the param in vars.
I'd personally go for the former option and pass RESTCatalogAdapter whatever is needed to perform the check itself.
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
This PR adds server-side warehouse validation to the REST catalog test fixture server, complementing the client-side changes in #16059.
When a
warehousequery parameter is provided to/v1/configthat doesn't match the configured warehouse, the server returns a404 NoSuchWarehouseException.Testing
Build and run the REST catalog server locally:
Default (without setting
catalog.name)Preserves old behavior, everything returns 200. Internally the default catalog name is
rest_backend. We are explicitly not gating on that to be backwards compatibleWith setting
catalog.nameCATALOG_CATALOG_NAME(catalog.name)