You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Iceberg data tables whose names end with a metadata-table name fall back to Spark #6719
CometScanRule.isIcebergMetadataTable matches the table name with endsWith against the metadata table names (files, snapshots, history, entries, manifests, partitions, ...) with no separator. Any ordinary Iceberg table whose name ends with one of those words is treated as a metadata table and falls back to Spark with Iceberg Metadata tables are not supported, for example user_files, order_history, daily_partitions, or a table named snapshots.
CREATETABLEcat.db.user_files (id INT, name STRING) USING iceberg;
INSERT INTOcat.db.user_files VALUES (1, 'a'), (2, 'b');
SELECT*FROMcat.db.user_files;
With spark.comet.scan.icebergNative.enabled=true and spark.comet.explain.fallback.enabled=true the scan is reported as:
BatchScan cat.db.user_files [COMET: Iceberg Metadata tables are not supported]
The same table named users gets CometIcebergNativeScan.
Expected behavior
Only real metadata tables (cat.db.user_files.files, ...metadata.json#files) should fall back.
Additional context
Proposed fix: recognize a metadata table by its Iceberg type. Every metadata table extends org.apache.iceberg.BaseMetadataTable, and the table is reachable through the existing IcebergReflection.getTable(scan). Fall back to a whole-segment name match (.files, #files) only when the table cannot be reached. Happy to open a PR with a test covering user_files, snapshots and order_history plus a real .files read.
Activity
aryan9948 commentedon Oct 6, 2026
Hi! I’d like to work on this issue and submit a PR with the regression tests and fix described above.
Would it be possible to assign this issue to me?