Skip to content

[NNVM][FRONTEND][ONNX] Fix the gemm conversion in onnx frontend #1241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tqchen merged 1 commit into from
Jun 11, 2018
Merged

[NNVM][FRONTEND][ONNX] Fix the gemm conversion in onnx frontend #1241

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions nnvm/python/nnvm/frontend/onnx.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ def _impl_v1(cls, inputs, attr, params):
inputs[0] = _sym.transpose(inputs[0], axes=(1, 0))
if not transB:
inputs[1] = _sym.transpose(inputs[1], axes=(1, 0))
inputs[0] = _sym.flatten(inputs[0])

@tqchen tqchen Jun 6, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we verify if flatten is necessary? i.e. check the dimension and only insert the flatten if necessary

@tqchen tqchen Jun 8, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nishi-t can you act on this comment?

@nishi-t nishi-t Jun 8, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying it, but I have not already done so well. Because, I'm struggling to get the input data's dimension in the _impl_v1 for checking necessity of flatten. I try to use _infer_shape auxiliary function to get the input[0]'s output_shape. But, it does not work when input[0] is pooling. Could you give me some advices?

@tqchen tqchen Jun 8, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is strange that you cannot get the output shape when the input[0] is pooling. Maybe can you look into what happens here?

@nishi-t nishi-t Jun 11, 2018 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your advice. I analyzed what happened here and found out a few things.
In the stage of nnvm.frontend.from_onnx, maybe infer_shape does not work completly, because the input's shape may not be decided.

When I putted the following code before the flatten and ran it, out_shapes was [[]]

g = _graph.create(inputs[0])
shape_dict = {k: v.shape for k, v in params.items()}
_, out_shapes = graph_util.infer_shape(g, **shape_dict)

In following lines called from above infer_shape, dshape.ndim() became 0 and returned false.

https://github.com/dmlc/tvm/blob/master/nnvm/src/top/nn/pooling.cc#L33
https://github.com/dmlc/tvm/blob/master/nnvm/src/top/nn/convolution.cc#L62

It seems difficult to use infer_shape for my purpose without making any changes. Please let me know what you think.

@tqchen tqchen Jun 11, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhreshold can you comment on this?

@zhreshold zhreshold Jun 11, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

infer shape is not available because input shape is not ready. Currently I cannot find a way to detect if flatten is necessary as well.

@tqchen tqchen Jun 11, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, maybe an easier way is to add an optimization to remove flatten in nnvm when it is not necessary. I am going to merge this in for now

return _sym.dense(
alpha * inputs[0], inputs[1], beta * inputs[2], units=channels)

Expand Down