Skip to content

Make from_substrait_plan return DataFrame instead of LogicalPlan#164

Merged
andygrove merged 2 commits into
apache:mainfrom
andygrove:from_substrait_plan_df
Feb 5, 2023
Merged

Make from_substrait_plan return DataFrame instead of LogicalPlan#164
andygrove merged 2 commits into
apache:mainfrom
andygrove:from_substrait_plan_df

Conversation

@andygrove
Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A

Rationale for this change

I want to be able to execute a substrait plan and there is no way to do that after deserializing a logical plan from substrait.

What changes are included in this PR?

Change from_substrait_plan to return DataFrame instead of LogicalPlan

Are there any user-facing changes?

@andygrove
Copy link
Copy Markdown
Member Author

@jdye64 fyi

Copy link
Copy Markdown
Contributor

@jdye64 jdye64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine with the changes but thinking we could add another method, in the PySubstraitSerializer maybe?, to have that functionality to get the LogicalPlan itself. I know Dask-SQL is planning on using this to get just the logical plan for example and they don't use the DataFrame since they use a Dask Dataframe. We can just make another issue and add that later though unless you want to copy/paste it now

@andygrove andygrove force-pushed the from_substrait_plan_df branch from 71d5fb1 to d6a2362 Compare February 5, 2023 15:15
@andygrove
Copy link
Copy Markdown
Member Author

I changed the approach here and reverted my previous changes. There is now a new create_dataframe_from_logical_plan method on the context instead.

@andygrove andygrove merged commit 8c618f9 into apache:main Feb 5, 2023
@andygrove andygrove deleted the from_substrait_plan_df branch February 5, 2023 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants