Repository navigation
Cast FixedSizeList to List will lost datatype metadata in list #10004
Description
Describe the bug
impl<OffsetSize: OffsetSizeTrait> GenericListArray<OffsetSize> {
/// The data type constructor of list array.
/// The input is the schema of the child array and
/// the output is the [`DataType`], List or LargeList.
pub const DATA_TYPE_CONSTRUCTOR: fn(FieldRef) -> DataType = if OffsetSize::IS_LARGE {
DataType::LargeList
} else {
DataType::List
};
DATA_TYPE_CONSTRUCTOR would be used here, however, here it will lost the metadata within field.
To Reproduce
cast the way below:
src FixedSizeList(Field { name: "element", data_type: Float32, nullable: true }, 1024)
target List(Field { name: "element", data_type:
Float32, nullable: true, metadata: {"PARQUET:field_id": "89"} })
Target would be:
target List(Field { name: "element", data_type:
Float32, nullable: true} )
Expected behavior
target List(Field { name: "element", data_type:
Float32, nullable: true, metadata: {"PARQUET:field_id": "89"} })
Additional context
No
Activity
fix(cast): Trying to fix cast losting schema problem (#10005)
fix(cast): Trying to fix cast losting schema problem (apache#10005)