You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Substrait filter validation (lambda/window/subquery rejection) is skipped when no schema fields are pruned #7805
remap_expr_references doubles as both an index remapper and a validation pass — it rejects unsupported filter operators (WindowFunction, Subquery, and now Lambda/LambdaInvocation):
So a Substrait filter whose base schema needs no field pruning skips validation entirely, and an unsupported expression (window function, subquery, lambda) reaches the DataFusion consumer instead of being rejected up front with our descriptive error.
Expected: run the rejection checks unconditionally, independent of whether remapping is needed. Suggested approach: split validation from index remapping — a validate_expr traversal that always runs, plus the existing remap that runs only when the mapping is non-trivial.
Add a regression test with an unpruned base schema containing an unsupported operator, asserting it errors.
Note: pre-existing behavior (the WindowFunction/Subquery arms have always had this gap); the Lambda arm was added in #7793 for the substrait 0.63 bump, which is what surfaced it via CodeRabbit review.
Activity
fix: validate substrait filters that need no field pruning (#8484)