Skip to content

[OPPRO-77] adding hdfs support - #152

Closed
zhouyuan wants to merge 1 commit into
apache:mainfrom
zhouyuan:wip_hdfs
Closed

zhouyuan wants to merge 1 commit into
apache:mainfrom
zhouyuan:wip_hdfs

Conversation

@zhouyuan

@zhouyuan zhouyuan commented May 11, 2022 •

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This patch adds HDFS support

Signed-off-by: Yuan Zhou yuan.zhou@intel.com

How was this patch tested?

locally verified

@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[Gluten-${ISSUES_ID}] ${detailed message}

See also:

@zhouyuan

Copy link
Copy Markdown
Member Author

depends on: oap-project/velox#11

@zhztheplayer
zhztheplayer force-pushed the main branch 2 times, most recently from c6de69a to df3ea7a Compare May 24, 2022 02:35
@zhouyuan
zhouyuan force-pushed the wip_hdfs branch 2 times, most recently from a3f8cb8 to f6fc466 Compare July 6, 2022 04:10
@jinchengchenghh
jinchengchenghh force-pushed the main branch 2 times, most recently from 8c08f38 to b850a55 Compare July 8, 2022 02:57
@zhouyuan
zhouyuan force-pushed the wip_hdfs branch 7 times, most recently from 1d5fe07 to 8ae8625 Compare July 13, 2022 15:00
@rui-mo rui-mo mentioned this pull request Jul 20, 2022
@zhouyuan
zhouyuan force-pushed the wip_hdfs branch 2 times, most recently from f52e4aa to 28002f9 Compare September 6, 2022 03:58
@zhouyuan
zhouyuan marked this pull request as ready for review September 6, 2022 03:59
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
@KevinyhZou

Copy link
Copy Markdown
Contributor

@zhouyuan hello, I use this pr to support read dwrf file from hdfs, but encounter the error in DwrfDatasource#InspectSchema, which do not support hdfs. and I have make another pr: #389, please help to review.

@FelixYBW

FelixYBW commented Oct 8, 2022

Copy link
Copy Markdown
Contributor

@KevinyhZou FYI, Velox redesigned their parquet reader and we have add it to Gluten. You can test parquet file format now.

@zhouyuan

zhouyuan commented Oct 8, 2022

Copy link
Copy Markdown
Member Author

@KevinyhZou Thanks for working on this! let's switch to your new PR

@zhouyuan

zhouyuan commented Oct 8, 2022

Copy link
Copy Markdown
Member Author

replaced by #389

@zhouyuan zhouyuan closed this Oct 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants