Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In order to register a "table" with DataFusion information must be known about the underlying data. When a user is using DataFusion as both a logical and physical engine this is already handled. However, if the user would like to use DataFusion for just the logical side it is the burden of the user to create the SqlTable instance and register it with DataFusion.
In order to prevent duplicate code and further expand adoption we should offer utilities that allow for popular data sources to be examined and then a SqlTable instance generated so the consuming library doesn't have to do that.
Describe the solution you'd like
Python module that accepts a filepath. Reads the file. Parquet and CSV for now. And generates the SqlTable instance as part of the CreateTable() method call.
Describe alternatives you've considered
None
Additional context
None
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
In order to register a "table" with DataFusion information must be known about the underlying data. When a user is using DataFusion as both a logical and physical engine this is already handled. However, if the user would like to use DataFusion for just the logical side it is the burden of the user to create the
SqlTableinstance and register it with DataFusion.In order to prevent duplicate code and further expand adoption we should offer utilities that allow for popular data sources to be examined and then a
SqlTableinstance generated so the consuming library doesn't have to do that.Describe the solution you'd like
Python module that accepts a filepath. Reads the file. Parquet and CSV for now. And generates the
SqlTableinstance as part of theCreateTable()method call.Describe alternatives you've considered
None
Additional context
None