Update dependency org.apache.hive:hive-metastore to v4

[renovate-bot]

This PR contains the following updates:

Package	Change	Age	Confidence
org.apache.hive:hive-metastore (source)	3.1.3 → 4.2.1

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[flyingImer]

This is a major version bump-up, the checks seem to be just fine.

[flyrain]

Looks like there is no tests for the Hive federation. I'd suggest add integration tests before we merge this.

Minor finding:
Interesting that the comment says it is using hive 4.1.0,

polaris/extensions/federation/hive/build.gradle.kts

Line 35 in 8cbfb26

// Add our own Hive 4.1.0 dependencies

.

[flyrain]

Need integration tests to verify if there are regression for the affect component (HMS federation).

[jbonofre]

@flyrain I'm also not convinced we should update HMS version, because, afair, HMS 4.x plans to add IRC support.

[nandorKollar]

@flyrain I'm also not convinced we should update HMS version, because, afair, HMS 4.x plans to add IRC support.

That's what I heard too. With IRC support I don't think one should use HMS federation with using the Thrift interface. Integration tests would still make sense though.

[flyrain]

@flyrain I'm also not convinced we should update HMS version, because, afair, HMS 4.x plans to add IRC support.

That's what I heard too. With IRC support I don't think one should use HMS federation with using the Thrift interface. Integration tests would still make sense though.

I think we only care about the Hive client side lib here, which isn't necessarily related to the IRC support. In case of HMS federation support in Polaris, it should always use the thrift clients, no matter what the client lib version is.

For the use case of federation via HMS IRC interface, that would belong to the REST federation, which doesn't depend on the hive version at all.

[nandorKollar]

@flyrain I'm also not convinced we should update HMS version, because, afair, HMS 4.x plans to add IRC support.

That's what I heard too. With IRC support I don't think one should use HMS federation with using the Thrift interface. Integration tests would still make sense though.

I think we only care about the Hive client side lib here, which isn't necessarily related to the IRC support. In case of HMS federation support in Polaris, it should always use the thrift clients, no matter what the client lib version is.

For the use case of federation via HMS IRC interface, that would belong to the REST federation, which doesn't depend on the hive version at all.

Ah okay, makes sense. I recall that I tried to upgrade the HMS client lib version, and there were indeed problems with connecting to HMS 3.x server. I'll try to reproduce that.

[flyrain]

#4315 has added HMS integration tests, we can reconsider this upgrade.

[flyrain]

It did fail on new added hive integration tests. cc @adutra

HiveCatalogFederationIT > testFederatedWriteVisibleInHms() FAILED
    java.lang.NoSuchFieldError at HiveCatalog.java:113

HiveCatalogFederationIT > testFederatedReadSeesHmsTables() FAILED
    org.opentest4j.AssertionFailedError at ManagementApi.java:71

HiveCatalogFederationIT > testFederatedCatalogEnforcesNamespaceRbac() FAILED
    org.opentest4j.AssertionFailedError at ManagementApi.java:71