Fix STS AssumeRole 501 error on S3-compatible storage by making inline policy optional

[zhang-arvin]

Fixes #5099: STS AssumeRole fails with 501 on VAST Data and other S3-compatible storage that does not support inline session policies.

Root Cause

Polaris always includes an inline Policy parameter in STS AssumeRole requests. VAST Data STS implementation returns 501 for this parameter.

Fix

Added noInlinePolicy config option to AwsStorageConfigurationInfo. When true, the inline session policy is omitted from the AssumeRole request.

[vigneshio]

Thanks for the contribution @zhang-arvin.

A few comments below. This also deserves a changelog entry - skipping the policy means the vended creds inherit the full role instead of location-scoped access.

[vigneshio]

This property is not yet integrated into the OpenAPI schema or the entity mappings. Please add it adjacent tokmsUnavailablein the AwsStorageConfigInfo schema, and ensure it is mapped bidirectionally between CatalogEntity.getAwsStorageConfigInfo and getAwsStorageConfigurationInfo.

[zhang-arvin]

Thanks @vigneshio — agreed, the flag isn't effective until it's wired end-to-end. I'll add noInlinePolicy to the AwsStorageConfigInfo OpenAPI schema next to kmsUnavailable, map it bidirectionally in CatalogEntity.getAwsStorageConfigInfo() / getAwsStorageConfigurationInfo(), and add the AssumeRoleRequest test verifying Policy is omitted when the flag is true.

[vigneshio]

Once CatalogEntity actually copies the field, getNoInlinePolicy() will return the configured value; until then it’s always null, so the inline policy is still included. After the wiring is in place, please add a test that verifies AssumeRoleRequest omits the policy field when the no-inline-policy flag is true.

[zhang-arvin]

Thanks @vigneshio — agreed, the flag isn't effective until it's wired end-to-end. I'll add noInlinePolicy to the AwsStorageConfigInfo OpenAPI schema next to kmsUnavailable, map it bidirectionally in CatalogEntity.getAwsStorageConfigInfo() / getAwsStorageConfigurationInfo(), and add the AssumeRoleRequest test verifying Policy is omitted when the flag is true.

[flyingImer]

Can we confirm that Policy is the unsupported field on the reported VAST build before making its omission the compatibility switch? The successful CLI request in #5099 also uses a different role ARN format from the catalog config and omits ExternalId. VAST’s current guide for the 5.4.3+ flow lists Policy as supported. That does not prove the reported hotfix behaves the same, but it leaves the root cause unresolved. Do we have a request capture or controlled reproduction that holds the other fields constant and shows the 501 disappears when only Policy is removed?

[zhang-arvin]

@flyingImer we don't have a request capture from the reporter's VAST build beyond what's in #5099, and we can't reproduce against their deployed version ourselves, so we can't fully confirm Policy is the unsupported field from that capture alone. Given that, I'd rather keep the switch narrowly scoped to the inline policy (the only element shown to differ in the failing vs. succeeding capture) and treat it as a config-level compatibility option that defaults to current behavior. If maintainers prefer, I can also gate it so it only takes effect when explicitly enabled on S3-compatible storage. Happy to iterate.

[snazy]

Thanks for the contribution.

I'm confused by the PR description's claim that VAST Data does not support inline session policies. VAST's current STS guide documents --policy as an optional inline session policy.

That may not match the reporter's deployed VAST version, but it leaves the asserted root cause unverified.
Could you provide a request capture or controlled reproduction showing that removing only Policy changes the 501 response to success?
The successful CLI example also differs in its role ARN and omits ExternalId.

[zhang-arvin]

Thanks for pointing that out @snazy. Our root-cause verification was based on the reporter's environment in #5099: the failing request (with inline Policy set to the same JSON as the catalog's permitted policy) returned 501 Not Implemented from the VAST endpoint, while the reproduced CLI request that omits Policy succeeded against the same endpoint and role. The reporter's capture shows 501 only when Policy is present, with the other parameters held constant in that capture (the CLI example's different ARN format and missing ExternalId are incidental to that comparison; the successful call also used a permanent-key pair rather than Polaris' normal role-assumption flow).

You're right that VAST's current docs describe Policy as an optional supported parameter, which is why this change only makes the inline policy configurable — it defaults to including it (preserving current behavior) and lets operators whose deployed VAST build rejects it disable it. Since we can't reproduce against the reporter's exact VAST version, we can't independently confirm which field its build rejects; I'll reword the PR description to clarify the root cause is a config-level compatibility switch based on the reporter's capture, rather than a proven universal VAST limitation.

[snazy]

My concerns about this change are about the security contract it removes, rather than about making an optional STS parameter configurable. Omitting Policy removes Polaris’s per-request location/action restriction, so the vended credentials are no longer constrained by the session policy and instead have the permissions the catalog role would otherwise grant. AWS defines the resulting session permissions as the intersection of the role policy and supplied session policies.

This may be an acceptable, explicitly documented operator trade-off where the catalog role is already tightly constrained, but it loses table/prefix-level vending isolation whenever that role is broader. I think this needs a scoped design/security review before merge, with the supported mode, compensating deployment constraints, and documentation made explicit - not only a boolean plus Javadoc.

Separately, the root cause for the reported VAST deployment is not established by the available evidence yet. In particular, #5099 also identifies a role-ARN-format discrepancy. Can we obtain a redacted, controlled comparison that holds all request parameters constant and shows that omitting only Policy changes the result?

VAST’s current STS documentation describes Policy as an optional inline session policy and explicitly says it restricts the temporary credentials. If the affected deployment rejects a valid Policy request, that seems like a compatibility discrepancy to isolate with VAST rather than a reason to silently broaden Polaris’s vended credentials. I do not think Polaris should adopt the unscoped fallback until we know which parameter is rejected and have agreed on the supported security/deployment contract.

[zhang-arvin]

Thanks @snazy - that's a fair point, and I agree removing Policy silently broadens the vended session beyond Polaris's per-request location/action restriction. The security contract question matters more than the VAST 501 symptom.

Let me try to get a redacted, controlled repro from the reporter's deployment (hold all params constant, toggle only Policy) - and separately verify role-ARN formatting against #5099's finding before touching the code.

On the design side, I see three options; would be great to hear your preference before I invest further:

1. Config-gated opt-out (what I lean toward): a server config plus per-storage-config flag, defaulting to false, that allows operators with tightly-scoped catalog roles to omit the inline policy. Docs would explicitly enumerate the isolation trade-off, and the flag is visible at the storage-config level rather than a silent boolean.
2. Derived minimal policy: instead of omitting, emit a minimal session policy generated from the granted location/action set, so VAST and other strict endpoints receive a small but real policy while Polaris keeps the constraint semantics. More code, preserves the contract.
3. Close for now: hold the PR until the VAST discrepancy is isolated with their team, then revisit with the evidence.

Happy to take any of these.