Repository navigation
push_down_leaf_projections fails for self joins in some scenarios #24241
Description
Describe the bug
Some special shapes of queries, namely self joins on structs/json data trip up the push down leaf expressions optimiser.
To Reproduce
From datafusion-cli:
CREATE TABLE t AS SELECT named_struct('a', 'x', 'b', 'y') AS s, 1 AS id;
SELECT p.s['a'], count(1)
FROM t p
LEFT JOIN t c ON p.id = c.id
WHERE c.s['b'] IS NOT NULL
GROUP BY 1;Will produce:
Optimizer rule 'push_down_leaf_projections' failed
caused by
Schema error: Schema contains qualified field name p.__datafusion_extracted_1
and unqualified field name __datafusion_extracted_1 which would be ambiguous
Expected behavior
Should output something like:
+--------+-----------------+
| p.s[a] | count(Int64(1)) |
+--------+-----------------+
| x | 1 |
+--------+-----------------+
Additional context
No response
Activity
Fix self join leaf expressions (apache#24242)
test: differential fuzz for leaf expression pushdown and parquet filt…
test: differential fuzz for leaf expression pushdown and parquet filt…