[Test][TFLite] Add unit tests for LEAKY_RELU, HARD_SWISH ReLU_N1_to_1 and LOG_SOFTMAX#19388
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the TVM Relax TFLite frontend by renaming the leaky_relu operator to leakyrelu and updating the clip operator call. It also adds new test cases for leaky_relu, hard_swish, relu_n1_to_1, and log_softmax. The reviewer suggests using integer literals for the clip bounds in the hard_swish implementation and its associated test to ensure consistency with other activation functions and avoid potential overhead.
|
|
||
| def _relu6(data): | ||
| return relax.op.tensor.clip(data, 0.0, 6.0) | ||
| return relax.op.clip(data, 0.0, 6.0) |
There was a problem hiding this comment.
For consistency with other activation functions implemented with clip in this file (like convert_relu6 and convert_relu_n1_to_1), it would be better to use integer literals for the clip bounds. This avoids potential overhead from using float64 bounds on a float32 tensor and improves consistency.
| return relax.op.clip(data, 0.0, 6.0) | |
| return relax.op.clip(data, 0, 6) |
| lv1: R.Tensor((1, 30), dtype="float32") = R.clip( | ||
| lv, R.prim_value(T.float64(0.0)), R.prim_value(T.float64(6.0)) | ||
| ) |
There was a problem hiding this comment.
This PR adds unit test coverage for activation function operators
in the Relax TFLite frontend, as part of #18971
(tf.nn.log_softmax, R.nn.log_softmax)entry to elementwise unit testLEAK_RELUwith default alphaHARD_SWISHReLU_N1_to_1Bugs fixed:
relax.op.tensor.clip->relax.op.clipin_relu6helper ofconvert_hard_swishrelax.op.nn.leaky_relu->relax.op.nn.leakyreluinconvert_leaky_relu