You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
PushDownLimit keeps a redundant Limit above a Sort it gave a fetch, so every ORDER BY ... LIMIT n query runs one extra optimizer pass #25671
When PushDownLimit visits Limit(skip=0, fetch=n) over a Sort without a fetch, it sets Sort.fetch = n but keeps the Limit node. The rule removes the Limit only the next time it visits it, which is in the next optimizer pass. That pass reports a change, so the optimizer then runs one more pass to confirm the fixed point.
The result: every query of the shape SELECT ... ORDER BY x LIMIT n runs 3 logical optimizer passes (the datafusion.optimizer.max_passes default), where 2 are enough. The final plan is correct. Only planning time is affected.
A query engine that wraps every query in a row-cap LIMIT over an ORDER BY hits this on nearly every query.
With skip = 0, a Sort with fetch = n returns at most n rows, so the Limit is redundant as soon as the rule sets the fetch. The rule already drops it on the next visit (the new_fetch == sort.fetch branch with skip == 0). It should drop it in the same visit, so the query settles in pass 0 and the optimizer stops after pass 1.
Additional context
The same plan shape with a filter and a subquery (SELECT a, s FROM (SELECT * FROM t WHERE s != 'pending') AS t WHERE s = 'x' ORDER BY ts DESC LIMIT 10) shows the same extra pass. There, push_down_limit is again the only rule that changes the plan in pass 1.
Activity
fix: drop the redundant `Limit` above a `Sort` in the same `PushDownL…
fix: drop the redundant `Limit` above a `Sort` in the same `PushDownL…