Skip to content

Preserve external catalog connection fields in Python clients - #5410

Open
mattfaltyn wants to merge 2 commits into
apache:mainfrom
mattfaltyn:fix-5409-external-catalog-deserialization
Open

Preserve external catalog connection fields in Python clients#5410
mattfaltyn wants to merge 2 commits into
apache:mainfrom
mattfaltyn:fix-5409-external-catalog-deserialization

Conversation

@mattfaltyn

@mattfaltyn mattfaltyn commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Summary

The generated Python management client currently loses subtype-specific connection fields for Iceberg REST, Hadoop, and Hive external catalogs. OpenAPI Generator does not merge schema properties declared as siblings of allOf by default, so ConnectionConfigInfo.from_dict() discards remoteCatalogName and warehouse; polaris setup export then writes those fields as null.

This change:

  • enables the OpenAPI Generator REFACTOR_ALLOF_WITH_PROPERTIES_ONLY normalizer for Python management-client generation, leaving the source OpenAPI specification unchanged;
  • adds a regression test that deserializes all three discriminator variants and verifies setup export retains their location fields; and
  • documents the user-visible fix in the changelog.

The wire contract, source OpenAPI specification, and requiredness are unchanged; this corrects generated-client model composition.

Fixes #5409

Validation

  • make client-regenerate
  • make client-unit-test — 198 passed
  • make client-lint
  • make client-license-check
  • make client-build
  • ./gradlew format compileAll with Java 21

AI assistance

AI assistance was used for investigation, reproduction, implementation, and validation. I reviewed the issue, generated diff, tests, and contribution text and understand the change end to end.

Checklist

  • 🛡️ Do not disclose security issues! (contact security@apache.org)
  • 🔗 Clearly explained why the changes are needed, or linked related issues: Fixes Python client drops external catalog subtype fields during deserialization #5409
  • 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
  • 💡 Added comments for complex logic (none needed for this generator configuration correction)
  • 🧾 Updated CHANGELOG.md (if needed)
  • 📚 Updated documentation in site/content/in-dev/unreleased (not needed; the API contract and usage are unchanged)

@MonkeyCanCode

Copy link
Copy Markdown
Contributor

Hello @mattfaltyn , for spec file change, please start a dev ML for this change.

@flyrain

flyrain commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

is there a way to resolve it without spec change?

@mattfaltyn

mattfaltyn commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Yes. OpenAPI Generator 7.12.0 supports REFACTOR_ALLOF_WITH_PROPERTIES_ONLY=true, which applies the same structural normalization during Python client generation. I verified it against the PR’s base spec: discriminator deserialization retains remoteCatalogName and both warehouse fields. I replaced the three YAML changes with this generator option, keeping the source OpenAPI and wire contract unchanged.

@flyrain flyrain left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @mattfaltyn !

@github-project-automation github-project-automation Bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Sep 3, 2026
@MonkeyCanCode

Copy link
Copy Markdown
Contributor

LGTM @mattfaltyn . Mind fix the conflict when you get a chance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python client drops external catalog subtype fields during deserialization

3 participants