Fix: Group transaction changes by table

[singhpk234]

Problem

. Groups ALL changes by table- even if they appear randomly in the input like [A1, B1, A2, C1, A3] → groups to {A:
[A1, A2, A3], B: [B1], C: [C1]}
2. For each table, processes changes sequentially :
- Validate R1 against base metadata → Apply U1 → update currentMetadata
- Validate R2 against updated metadata → Apply U2 → update currentMetadata
- Validate R3 against updated metadata → Apply U3 → update currentMetadata
3. Single commit per table prevents duplicate entity IDs in pendingUpdates

This ensures:

• Each change's requirements validate against the evolved state
• Updates are applied in order within each table
• Only one commit per table (solves the original problem)
• Conflicts detected early (e.g., second change expecting schema ID 0 fails when it's already 1)

related discussion: #3352 (comment)

Checklist

• 🛡️ Don't disclose security issues! (contact security@apache.org)
• 🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
• 🧪 Added/updated tests with good coverage, or manually tested (and explained how)
• 💡 Added comments for complex logic
• 🧾 Updated CHANGELOG.md (if needed)
• 📚 Updated documentation in site/content/in-dev/unreleased (if needed)

[dimas-b]

nit: Only one commit per table (solves the original problem) - I'd simply say Only one commit per table. This will end up in the git commit log, where the "original problem" might be hard to understand later on... It's also fine to give a summary of the problem instead of the "original" reference if you prefer.

[dimas-b]

Nice fix. Thanks, @singhpk234 !

Just a couple of minor comments :)

[adnanhemani]

Awesome change!

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[singhpk234]

Apologies it took me a while to get back to it, was super swamped with internal stuff.
Addressed @dennishuo feedback to add detailed comment.
Addressed @flyrain's offline feedback to add more tests to iron this.

Please have another pass when you all get some time, really apprecaite your feedbacks here !

[dimas-b]

@singhpk234 : thanks for pushing this forward 👍 Please fix fresh IcebergCatalogHandler conflicts 😅

[flyrain]

LGTM. Thanks @singhpk234 !

[flyrain]

This check allows the operation setLocation when ALLOW_NAMESPACE_LOCATION_OVERLAP is true. I think the behavior isn't correct. cc @dennishuo @collado-mike
However, it isn't a blocker for me as this PR doesn't the logic.

[flyrain]

Is behavior deterministic when we have two property updates like this?

1. transaction.updateProperties().set("prop-key", "prop-val1").commit();
2. transaction.updateProperties().set("prop-key", "prop-val2").commit();

If not, we may check with the Iceberg community to clarify the behavior. Not a blocker.

[singhpk234]

within a transaction (taking iceberg's client side transcation as an example) it will be consistent as if prop-key value would be prop-val2 as we create a new metadata locally and keep on apply update on top of last updated metadata, is my understanding.

[flyrain]

Are updateRequests ordered? If not, the final result could be either val2 or val1.

[singhpk234]

yes they are (atleast in java impl) I believe spec has an implicit assumption on this, we can iron the spec more bcz otherwise it will not be a true transaction if they are not ordered, is my take.

https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L1320

The serializer and deserializer both respect the order :
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/requests/CommitTransactionRequestParser.java#L42

https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/requests/CommitTransactionRequestParser.java#L62

an E2E tests for this
https://github.com/apache/iceberg/blob/main/core/src/test/java/org/apache/iceberg/rest/requests/TestCommitTransactionRequestParser.java#L162

plz let me know how feel about it

[singhpk234]

Thanks everyone for the review !