Skip to content

Add minimal forwarding RPC server for host driven python execution on Hexagon#9526

Merged
jwfromm merged 10 commits into
apache:mainfrom
csullivan:hexagon/minimal_rpc
Nov 30, 2021
Merged

Add minimal forwarding RPC server for host driven python execution on Hexagon#9526
jwfromm merged 10 commits into
apache:mainfrom
csullivan:hexagon/minimal_rpc

Conversation

@csullivan

@csullivan csullivan commented Nov 18, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor

This PR adds a proxy RPC server for Hexagon. It is a wrapper which takes standard TVM RPC calls from a python host to a remote Android device and forwards them across FastRPC to Hexagon. This RPC flow will be replaced by running a minimal RPC server directly on Hexagon. For now we provide a prototype forwarding RPC server for host driven execution on Hexagon.

This PR also implements a few supporting fixes:

  • Adds HexagonHostModuleNode which is now the default runtime hexagon runtime module. It separates out the current functionality of HexagonModule from the FastRPC offload device api in src/runtime/hexagon/android. This allows libtvm.so to be built with codegen support for hexagon, without target or simulator runtime support.
  • Updates CodeGenHexagon to sync. with recent changes to the LLVM code generator
    • A TODO from the PR is to consider if we can move Hexagon over to using the LLVM runtime module. In that case module.export_library could provide DSO linking rather than codegen.
  • Add CodeGenHexagon::AddMainFunction to expose __tvm_main__.

kparzysz-quic
kparzysz-quic reviewed Nov 19, 2021
View reviewed changes
Comment thread src/runtime/hexagon/rpc/device_api.cc
};

static HexagonRPCDeviceAPI* Global() {
static auto* inst = new HexagonRPCDeviceAPI();

@kparzysz-quic kparzysz-quic Nov 19, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be a static local (i.e. not allocated via new)?

static HexagonRPCDeviceAPI inst;

This way it will eventually be destroyed.

@csullivan csullivan Nov 23, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I just followed the convention of the other device APIs on their global singleton access.

Comment thread apps/hexagon_proxy_rpc/rpc_env.cc
@@ -0,0 +1,321 @@
/*

@kparzysz-quic kparzysz-quic Nov 19, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this loaded? Would it make sense to have this under src/runtime/hexagon/rpc?

@csullivan csullivan Nov 23, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rpc_env.cc is a hexagon version of rpc_env.cc from apps/cpp_rpc. In this case, all other sources from the C++ RPC server are reused. As noted in the description above we are working on a solution to run a minrpc server directly on hexagon which will provide more general functionality (e.g. support for tuning). This minrpc implementation will land in src/runtime/hexagon/rpc as you suggest. Until then, we are submitting this prototype forwarding RPC server application for use in development.

@csullivan csullivan Nov 23, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @mehrdadh who is working on the minrpc server implementation.

@mehrdadh mehrdadh Nov 24, 2021 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @csullivan. Let's keep these files under app and the minRPC server under src

@csullivan csullivan marked this pull request as ready for review November 23, 2021 00:27
@csullivan csullivan changed the title [WIP] Add minimal RPC server for host driven python execution on Hexagon Add minimal forwarding RPC server for host driven python execution on Hexagon Nov 23, 2021
@csullivan
Minimal proxy RPC server for hexagon. Functions by routing from
fc11b33 
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.
@csullivan
Apply clang-format.
49e3f7c
mehrdadh
mehrdadh reviewed Nov 24, 2021
View reviewed changes
Comment thread cmake/config.cmake Outdated
@csullivan csullivan requested a review from a team as a code owner November 29, 2021 18:03
jwfromm
jwfromm approved these changes Nov 30, 2021
View reviewed changes

@jwfromm jwfromm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Chris!

@jwfromm jwfromm merged commit 6c32f97 into apache:main Nov 30, 2021
masahi pushed a commit to masahi/tvm that referenced this pull request Dec 1, 2021
@csullivan @Lunderberg @masahi
Add minimal forwarding RPC server for host driven python execution on…
9f7936d 
… Hexagon (apache#9526)

* Minimal proxy RPC server for hexagon. Functions by routing from
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.

* Apply clang-format.

* Fix build to support building alongside Hexagon Launcher.

* Add readme.

* src/runtime/hexagon/rpc -> src/runtime/hexagon/proxy_rpc

* Added small refactors to hexagon test_matmul.py.

* Add skipif on additional env vars.

* Fix IOS build.

* Rename USE_HEXAGON_PROXY_RPC and add tvm_options entry.

* Add NDArray::Container deleters.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Dec 1, 2021
@csullivan @Lunderberg @mehrdadh
Add minimal forwarding RPC server for host driven python execution on…
e83354d 
… Hexagon (apache#9526)

* Minimal proxy RPC server for hexagon. Functions by routing from
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.

* Apply clang-format.

* Fix build to support building alongside Hexagon Launcher.

* Add readme.

* src/runtime/hexagon/rpc -> src/runtime/hexagon/proxy_rpc

* Added small refactors to hexagon test_matmul.py.

* Add skipif on additional env vars.

* Fix IOS build.

* Rename USE_HEXAGON_PROXY_RPC and add tvm_options entry.

* Add NDArray::Container deleters.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Dec 1, 2021
@csullivan @Lunderberg @mehrdadh
Add minimal forwarding RPC server for host driven python execution on…
3037423 
… Hexagon (apache#9526)

* Minimal proxy RPC server for hexagon. Functions by routing from
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.

* Apply clang-format.

* Fix build to support building alongside Hexagon Launcher.

* Add readme.

* src/runtime/hexagon/rpc -> src/runtime/hexagon/proxy_rpc

* Added small refactors to hexagon test_matmul.py.

* Add skipif on additional env vars.

* Fix IOS build.

* Rename USE_HEXAGON_PROXY_RPC and add tvm_options entry.

* Add NDArray::Container deleters.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
@TejashShah

TejashShah commented Dec 1, 2021

Copy link
Copy Markdown

Thanks, Chris! 🙌

ylc pushed a commit to ylc/tvm that referenced this pull request Jan 7, 2022
@csullivan @Lunderberg @ylc
Add minimal forwarding RPC server for host driven python execution on…
f753f57 
… Hexagon (apache#9526)

* Minimal proxy RPC server for hexagon. Functions by routing from
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.

* Apply clang-format.

* Fix build to support building alongside Hexagon Launcher.

* Add readme.

* src/runtime/hexagon/rpc -> src/runtime/hexagon/proxy_rpc

* Added small refactors to hexagon test_matmul.py.

* Add skipif on additional env vars.

* Fix IOS build.

* Rename USE_HEXAGON_PROXY_RPC and add tvm_options entry.

* Add NDArray::Container deleters.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
yangulei pushed a commit to yangulei/tvm that referenced this pull request Jan 11, 2022
@csullivan @Lunderberg @yangulei
Add minimal forwarding RPC server for host driven python execution on…
d3fb529 
… Hexagon (apache#9526)

* Minimal proxy RPC server for hexagon. Functions by routing from
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.

* Apply clang-format.

* Fix build to support building alongside Hexagon Launcher.

* Add readme.

* src/runtime/hexagon/rpc -> src/runtime/hexagon/proxy_rpc

* Added small refactors to hexagon test_matmul.py.

* Add skipif on additional env vars.

* Fix IOS build.

* Rename USE_HEXAGON_PROXY_RPC and add tvm_options entry.

* Add NDArray::Container deleters.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
yangulei pushed a commit to yangulei/tvm that referenced this pull request Jan 12, 2022
@csullivan @Lunderberg @yangulei
Add minimal forwarding RPC server for host driven python execution on…
2ff494c 
… Hexagon (apache#9526)

* Minimal proxy RPC server for hexagon. Functions by routing from
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.

* Apply clang-format.

* Fix build to support building alongside Hexagon Launcher.

* Add readme.

* src/runtime/hexagon/rpc -> src/runtime/hexagon/proxy_rpc

* Added small refactors to hexagon test_matmul.py.

* Add skipif on additional env vars.

* Fix IOS build.

* Rename USE_HEXAGON_PROXY_RPC and add tvm_options entry.

* Add NDArray::Container deleters.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
ylc pushed a commit to ylc/tvm that referenced this pull request Jan 13, 2022
@csullivan @Lunderberg @ylc
Add minimal forwarding RPC server for host driven python execution on…
ed492d1 
… Hexagon (apache#9526)

* Minimal proxy RPC server for hexagon. Functions by routing from
Android to Hexagon via QTI FastRPC calls. Interim solution until
Hexagon on-device RPC server is ready.

* Apply clang-format.

* Fix build to support building alongside Hexagon Launcher.

* Add readme.

* src/runtime/hexagon/rpc -> src/runtime/hexagon/proxy_rpc

* Added small refactors to hexagon test_matmul.py.

* Add skipif on additional env vars.

* Fix IOS build.

* Rename USE_HEXAGON_PROXY_RPC and add tvm_options entry.

* Add NDArray::Container deleters.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
@driazati driazati mentioned this pull request Jul 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mehrdadh mehrdadh mehrdadh left review comments

@jwfromm jwfromm jwfromm approved these changes

@areusch areusch Awaiting requested review from areusch

@comaniac comaniac Awaiting requested review from comaniac

@icemelon icemelon Awaiting requested review from icemelon

@jroesch jroesch Awaiting requested review from jroesch

@junrushao junrushao Awaiting requested review from junrushao

@kazum kazum Awaiting requested review from kazum

@liangfu liangfu Awaiting requested review from liangfu

@masahi masahi Awaiting requested review from masahi

@merrymercy merrymercy Awaiting requested review from merrymercy

@tmoreau89 tmoreau89 Awaiting requested review from tmoreau89

@tqchen tqchen Awaiting requested review from tqchen

@vinx13 vinx13 Awaiting requested review from vinx13

@yzhliu yzhliu Awaiting requested review from yzhliu

@zhiics zhiics Awaiting requested review from zhiics

@ZihengJiang ZihengJiang Awaiting requested review from ZihengJiang

+1 more reviewer

@kparzysz-quic kparzysz-quic kparzysz-quic left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@csullivan @TejashShah @mehrdadh @jwfromm @kparzysz-quic @Lunderberg