fix: preserve input ordering required by limits - #23744
Conversation
6ce79b4 to
eb5a24a
Compare
|
@alamb PTAL🙏 |
| /// Unlike [`Self::required_input_ordering`], this property does not require | ||
| /// a particular ordering that can be expressed using the child's schema. | ||
| /// It only requires that an existing input ordering is not discarded or | ||
| /// replaced with an incompatible ordering by physical optimizations. For |
There was a problem hiding this comment.
This property seems like it is some internal invariant of EnforceDistribution rather than something that describes how the ExecutionPlan behaves
I don't think we should add this to the ExecutionPlan trait
|
Isn't LIMIT already handled for subqueries? Perhaps we just need to extend the same logic to handle I don't understand the need to add a new method to ExecutionPlan |
5a7680b to
11ddca6
Compare
9f4b5c8 to
322826b
Compare
322826b to
2f97395
Compare
I tried extending PushDownLimit for OFFSET, but correctness still depended on optimizer pass order. With one pass, Limit → Projection → Sort remains and EnsureRequirements incorrectly removes the sort. So, I simplified the patch: no ExecutionPlan API changes—just use the existing is_limit check to preserve linked sorts below a limit, with a regression test proving the fix. Please review again @alamb. |
Which issue does this PR close?
GROUP BYconsumes a subquery containingORDER BY ... OFFSET. #23534.Rationale for this change
EnsureRequirements may remove a descendant SortExec when the child has no visible output ordering.
This is incorrect at a LIMIT boundary because LIMIT and OFFSET depend on the input sequence to determine which rows are returned. When an intervening ProjectionExec removes the sort key, its output_ordering() becomes None, even though it still preserves the row sequence. The optimizer then incorrectly removes the sort below the projection, producing incorrect results.
What changes are included in this PR?
Prevent EnsureRequirements from removing a linked SortExec at a limit boundary.
Add a SQL regression test that verifies the correct row is selected by an ordered subquery with OFFSET.
Configure the regression test with one logical optimizer pass to ensure correctness does not depend on a later logical rewrite.
No public API changes are included.
Are these changes tested?
Yes. The regression test fails without the fix, returning sorted_first instead of physical_second.
The following checks were run:
Are there any user-facing changes?
Yes. Ordered subqueries followed by LIMIT or OFFSET now preserve the sort needed to select the correct rows when an intervening projection hides the sort key.
There are no public API or documentation changes.