Resize bilinear + NHWC layout requires scales to be specified

[masahi]

Describe the bug
The ONNX Resize op spec says "Only one of 'scales' and 'sizes' can be specified", but if the following conditions are met, I get an error saying "scales" must be specified even though I am specifying output "sizes" instead.

• The input is in NHWC layout
• The "mode" is linear

Urgency
None

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
• ONNX Runtime installed from (source or binary): pip install onnxruntime
• ONNX Runtime version: 1.0.0
• Python version: 3.7.3

To Reproduce
Describe steps/code to reproduce the behavior:

import onnx
from onnx import helper, TensorProto
import numpy as np
import onnxruntime.backend


def get_onnxruntime_output(model, x, dtype='float32'):
    rep = onnxruntime.backend.prepare(model, 'CPU')
    x = x.astype(dtype)
    ort_out = rep.run(x)[0]
    return ort_out


def make_constant_node(name, data_type, dims, vals):
    return helper.make_node('Constant',
                            inputs=[],
                            outputs=[name],
                            value=helper.make_tensor(name=name,
                                                     data_type=data_type,
                                                     dims=dims,
                                                     vals=vals))


def verify(ishape, oshape, scales, mode, coord_trans):
    nodes = [
        make_constant_node('roi', onnx.TensorProto.FLOAT, (0,), []),
        make_constant_node('scales', onnx.TensorProto.FLOAT, (len(scales),), scales)
    ]
    input_names = ['X', 'roi', 'scales']
    if oshape != []:
        nodes.append(make_constant_node('sizes', onnx.TensorProto.INT64, (len(oshape),), oshape))
        input_names.append('sizes')
    nodes.append(helper.make_node(
        'Resize',
        inputs=input_names,
        outputs=['Y'],
        mode=mode,
        coordinate_transformation_mode=coord_trans
    ))

    if oshape == []:
        oshape = [round(dim * scale) for (dim, scale) in zip(ishape, scales)]

    graph = helper.make_graph(nodes,
                              "resize_test",
                              inputs=[helper.make_tensor_value_info("X", TensorProto.FLOAT, ishape)],
                              outputs=[helper.make_tensor_value_info("Y", TensorProto.FLOAT, oshape)])

    model = helper.make_model(graph, producer_name='resize_test')
    x = np.random.uniform(size=ishape).astype('float32')

    get_onnxruntime_output(model, x, 'float32')

        
# NCHW + linear, works
verify([1, 16, 32, 32], [1, 16, 64, 64], [], "linear", "align_corners")
# NHWC + nearest, works
verify([1, 32, 32, 16], [1, 64, 64, 16], [], "nearest", "asymmetric")
# NHWC + linear, does not work 
verify([1, 32, 32, 16], [1, 64, 64, 16], [], "linear", "align_corners")

Expected behavior

2019-12-18 07:19:15.814947403 [E:onnxruntime:, sequential_executor.cc:165 Execute] Non-zero status code returned while running Resize node. Name:'' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/upsample.h:221 void onnxruntime::UpsampleBase::ScalesValidation(const std::vector<float>&, onnxruntime::UpsampleMode) const scales.size() == 2 || (scales.size() == 4 && scales[0] == 1 && scales[1] == 1) was false. 'Linear' mode only support 2-D inputs ('Bilinear') or 4-D inputs with the corresponding outermost 2 scale values being 1 in the Resize operator
Stacktrace:

Traceback (most recent call last):
  File "resize_bug.py", line 60, in <module>
    verify([1, 32, 32, 16], [1, 64, 64, 16], [], "linear", "align_corners")
  File "resize_bug.py", line 52, in verify
    get_onnxruntime_output(model, x, 'float32')
  File "resize_bug.py", line 10, in get_onnxruntime_output
    ort_out = rep.run(x)[0]
  File "/home/masa/anaconda3/lib/python3.7/site-packages/onnxruntime/backend/backend_rep.py", line 52, in run
    return self._session.run(None, inps, options)
  File "/home/masa/anaconda3/lib/python3.7/site-packages/onnxruntime/capi/session.py", line 136, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Resize node. Name:'' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/upsample.h:221 void onnxruntime::UpsampleBase::ScalesValidation(const std::vector<float>&, onnxruntime::UpsampleMode) const scales.size() == 2 || (scales.size() == 4 && scales[0] == 1 && scales[1] == 1) was false. 'Linear' mode only support 2-D inputs ('Bilinear') or 4-D inputs with the corresponding outermost 2 scale values being 1 in the Resize operator

[hariharans29]

I ll take a look.

[askhade]

When sizes is provided instead of scale we internally derive the scale from sizes. For linear and cubic modes there is a restriction on the scale - it needs to be [1, 1, <scale_h>, <scale_w>]
The last case which does not work for you violates this restriction … the scales are calculated as [1, 2, 2, 1] and therefore you are seeing this error.

Btw ONNX assumes NCHW input... if your input is NHWC then you need to convert it to NCHW format.

[masahi]

ok, I was assuming that ONNX is agnostic to layout. No problem for me.