Skip to content

[microNPU] Set output tolerance of codegen and network tests to 0#10675

Merged
Mousius merged 2 commits into
apache:mainfrom
lhutton1:reference-kernel
Apr 7, 2022
Merged

[microNPU] Set output tolerance of codegen and network tests to 0#10675
Mousius merged 2 commits into
apache:mainfrom
lhutton1:reference-kernel

Conversation

@lhutton1

Copy link
Copy Markdown
Contributor

After the recent upgrade of Tensorflow to 2.6, we are now able to use reference kernels in order to verify the output. Thus, removing the tolerances previously added.

Additionally, the network tests have been altered to use TFLite as a reference, rather than TVM.

cc @manupa-arm @ekalda @NicolaLancellotti @dchauhan-arm

@github-actions github-actions Bot requested a review from manupak March 18, 2022 16:15
@lhutton1 lhutton1 force-pushed the reference-kernel branch 2 times, most recently from cd65869 to 4797434 Compare March 24, 2022 10:16
@NicolaLancellotti

Copy link
Copy Markdown
Contributor

Can you check if we need these two output tolerances?

output_tolerance=1 if operator_type == "MAX" else 0,

ethosu_mod, input_data, output_data, accel_type, output_tolerance=1

@lhutton1

Copy link
Copy Markdown
Contributor Author

Thanks for pointing these out @NicolaLancellotti! The identity tolerance cannot be changed since the output can be slightly different compared to the numpy reference due to quantization (cc @ekalda). However, we should be able to remove the tolerance from max. I'll update after #10762 to save an additional rebase :)

After the recent upgrade of Tensorflow to 2.6, we are now able to use
reference kernels in order to verify the output. Thus, removing the
tolerances previously added.

Additionally, the network tests have been altered to use TFLite as a
reference, rather than TVM.

Change-Id: Idcf408b84c67456d16da5d61f4865eb8b6042554
@lhutton1

lhutton1 commented Apr 5, 2022

Copy link
Copy Markdown
Contributor Author

After #10889 this should now pass CI

@NicolaLancellotti NicolaLancellotti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Change-Id: I174cc48a47c75590f76e0ee2a9634547b2442e87

@Mousius Mousius left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Mousius Mousius merged commit 05bb482 into apache:main Apr 7, 2022
@Mousius

Mousius commented Apr 7, 2022

Copy link
Copy Markdown
Member

Nice to see zero tolerance @lhutton1 / @NicolaLancellotti 😸

pfk-beta pushed a commit to pfk-beta/tvm that referenced this pull request Apr 11, 2022
…ache#10675)

After the recent upgrade of Tensorflow to 2.6, we are now able to use
reference kernels in order to verify the output. Thus, removing the
tolerances previously added.

Additionally, the network tests have been altered to use TFLite as a
reference, rather than TVM.
mehrdadh pushed a commit to mehrdadh/tvm that referenced this pull request Apr 11, 2022
…ache#10675)

After the recent upgrade of Tensorflow to 2.6, we are now able to use
reference kernels in order to verify the output. Thus, removing the
tolerances previously added.

Additionally, the network tests have been altered to use TFLite as a
reference, rather than TVM.
altanh pushed a commit to altanh/tvm that referenced this pull request Apr 28, 2022
…ache#10675)

After the recent upgrade of Tensorflow to 2.6, we are now able to use
reference kernels in order to verify the output. Thus, removing the
tolerances previously added.

Additionally, the network tests have been altered to use TFLite as a
reference, rather than TVM.
@lhutton1 lhutton1 deleted the reference-kernel branch May 9, 2022 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants