Repository navigation
Supporting filters in the left base table for join datasources - #10697
Merged
clintropolis merged 9 commits intoMar 4, 2021
Merged
Conversation
abhishekagarwal87
force-pushed
the
correlated-subquery
branch
2 times, most recently
from
January 27, 2021 11:15
9c58008 to
8fe0229
Compare
abhishekagarwal87
force-pushed
the
correlated-subquery
branch
from
March 3, 2021 14:00
8fe0229 to
538244a
Compare
abhishekagarwal87
changed the title
clintropolis
reviewed
Mar 4, 2021
| public static DataSourceAnalysis forDataSource(final DataSource dataSource) | ||
| { | ||
| // Strip outer queries, retaining querySegmentSpecs as we go down (lowest will become the 'baseQuerySegmentSpec'). | ||
| // Strip outer queries, retaining querySegmentSpecs as we go down (lowest will become the 'baseQuerySegmentSpec'o). |
Member
There was a problem hiding this comment.
this was probably an accident?
| return Pair.of(dataSource, toFiltration(filter, virtualColumnRegistry)); | ||
| } | ||
| //TODO: We should avoid promoting the time filter as interval for right outer and full outer joins. This is not | ||
| // donw now as we apply the intervals to left base table today irrespective of the join type. |
clintropolis
approved these changes
Mar 4, 2021
gianm
reviewed
Mar 5, 2021
|
|
||
| if (left.getPartialDruidQuery().stage() == PartialDruidQuery.Stage.SELECT_PROJECT | ||
| && left.getPartialDruidQuery().getWhereFilter() == null) { | ||
| if (left.getPartialDruidQuery().stage() == PartialDruidQuery.Stage.SELECT_PROJECT) { |
Contributor
There was a problem hiding this comment.
This may not work right if left is a DruidJoinQueryRel or DruidOuterQueryRel instead of a DruidQueryRel (since leftFilter is only supported for regular tables).
Contributor
Author
There was a problem hiding this comment.
oops. good catch. will fix this.
gianm
reviewed
Mar 5, 2021
gianm
reviewed
Mar 5, 2021
riovic918data
pushed a commit
to riovic918data/druid
that referenced
this pull request
Jun 12, 2026
…e#10697) * where filter left first draft * Revert changes in calcite test * Refactor a bit * Fixing the Tests * Changes * Adding tests * Add tests for correlated queries * Add comment * Fix typos
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As of now, if the left query in the JOIN has a filter present, we plan it as the join between two query data sources even if the left source is a base table. This is of course not efficient. We had even introduced some custom calcite rules so that filters on join query are not pushed into the children tables. This issue is more significant for correlated queries since the generated physical plan is a left join with a filter on the left base table. Hence, the execution of correlation queries is not efficient.
This PR adds the support for pushing filters into the left child. We would add the support for the right child in a different PR and also get rid of the custom calcite rules once that happens.
What does not work
This PR has:
Key changed/added classes in this PR