You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support row group skipping for shredded variant columns #15510
Iceberg supports writing shredded variant columns to Parquet (PR #14297).
For a query like WHERE variant_get(v, '$.price', 'int') > 100, Iceberg can already skip Parquet files using manifest-level bounds, where the lower/upper bounds for $.price are stored as a Variant-encoded object on the v column's manifest entry (PR #15385).
For better performance, Iceberg should also skip row groups within a file: when $.price is shredded, the physical Parquet column v.typed_value.price.typed_value has standard Parquet min/max statistics. Iceberg should use those stats to skip row groups where max < 100, the same way ParquetMetricsRowGroupFilter handles regular column predicates.
The change is in parquet module, and should work with all engines.
Scope
Add BoundExtract handling to ParquetMetricsRowGroupFilter
Activity
steveloughran commentedon Apr 23, 2026
i will have a go at this
apache#15510 Support row group skipping for shredded variant columns
apache#15510 Support row group skipping for shredded variant columns
apache#15510: checkstyles
apache#15510 Support row group skipping for shredded variant columns
apache#15510: checkstyles
apache#15510 Support row group skipping for shredded variant columns
soumilshah199500 commentedon May 19, 2026
Hi there, just wanted to inquire when we plan to merge this. I wanted to rerun the tests again. as filters on shredding variant table are very slow
apache#15510 Support row group skipping for shredded variant columns
steveloughran commentedon May 26, 2026
@soumilshah199500 yes they are. As for schedule, lots to work on. See : https://steveloughran.github.io/benchmarking-variants/
Some things we would love to have here are
apache#15510 Support row group skipping for shredded variant columns