Add dependency-free ONNX export for DNNs

[Compaile]

This adds inference-only ONNX export for dlib DNNs without adding a protobuf or ONNX dependency. The ONNX wire format is written by hand to avoid pulling
libprotobuf into the build.

Adds dlib/dnn/onnx.h and dlib/dnn/onnx_abstract.h, wires the exporter through dlib/dnn.h, adds a small ImageNet export example, and adds dependency-free
unit tests in test/dnn.cpp.

The exporter targets ONNX opset 17, IR version 8.

The default export mode takes the preprocessed NCHW tensor accepted by net.forward(). There is also a dlib_input_layer mode for supported input layers, so
RGB image preprocessing can be included in the ONNX graph.

Validated locally against ONNX Runtime CPU and CUDA:

• resnet34_1000_imagenet_classifier.dnn
• dlib_face_recognition_resnet_model_v1.dat
• semantic_segmentation_voc2012net_v2.dnn

The ResNet34 ImageNet path was also checked on a real dlib sample image, examples/mmod_cars_test_image.jpg, not only synthetic tensor input. Output
differences were within small floating point tolerances.

Supported layer coverage includes common inference layers such as convolutions, transposed convolutions, fully connected layers, affine/batchnorm
conversion, pooling, activations, softmax variants, residual skip layers, concat, reshape/flatten, resize/upsample, slice/extract, transpose, reorg,
normalization layers, embeddings, positional encodings, and fixed-shape tril masks.

Known unsupported cases:

• dropout
• image pyramid input layers
• input_rgb_image_pair
• adaptive computation time
• custom user layers
• detection/NMS postprocessing
• training/loss graph export beyond inference pass-through

The included tests are dependency-free and inspect the generated ONNX structure directly, so default dlib CI does not need ONNX Runtime or protobuf

ps feel free to add yourself to the copy right line at the top as it is standard in all dlib files. But it felt wrong to ad your name myelf without asking first :)

[Compaile]

While dlib is already fast, this gives us a possibility to use tensorrt as backend (via onnxruntime) and "free" fp16 mode which would be harder to implement into dlib

For the Resnet34 i build a small benchmarking script:

ONNX CUDA FP32:
verify max_abs=0.000128366 mean_abs=6.78675e-07 mismatches=0
500 calls: 671.751 ms total, 1.344 ms/call, 744.323 calls/s

ONNX TensorRT FP16:
verify max_abs=0.00305904 mean_abs=9.94151e-06 mismatches=0
500 calls: 282.627 ms total, 0.565 ms/call, 1769.117 calls/s

dlib CUDA:
500 calls: 1074.633 ms total, 2.149 ms/call, 465.275 calls/s

I love and prefer dlib for training, but having tensorRT Fp16 as runtime in prod is also nice

[pfeatherstone]

Wow this is cool! How about using the yolo example written by @arrufat as the default onnx export example? That's a bit harder than a classifier model. I think onnx supports NMS as native op. Adding that to the export graph would be cool.

[Compaile]

Thanks,

yes i think that could be added for sure. Kinda unsure since usually dlib samples are the most simple ones and the yolo example itself has way more loc. Maybe add it but dont change the default example?

[pfeatherstone]

Can I ask, what types of models are you training with dlib? Cheers

[davisking]

I left one question about the ONNX Resize optional inputs.

[davisking]

Should this use omitted optional input slots rather than named empty tensor initializers for roi and scales?

It looks like the current code may still specify both scales and sizes: scales_name is a real initializer and is passed as the third Resize input, while sizes_name is also passed as the fourth input. The ONNX Resize docs say one of scales and sizes must be specified and it is an error if both are specified. The same docs say that if sizes is needed, an empty string can be used as the scales input name: https://onnx.ai/onnx/operators/onnx__Resize.html

The ONNX IR spec also says static optional inputs can be left unspecified by using an empty string in place of an input name: https://onnx.ai/onnx/repo-docs/IR.html#input-and-output-values

So I think the node should probably leave both optional slots empty and provide only sizes:

Suggested change

	const std::string roi_name = ctx.next_value_name("resize_roi");
	const std::string scales_name = ctx.next_value_name("resize_scales");
	const std::string sizes_name = add_int64_initializer(ctx, "resize_sizes", output_shape);
	std::vector<float> empty;
	ctx.add_initializer(roi_name, {0}, empty);
	ctx.add_initializer(scales_name, {0}, empty);
	// dlib's bilinear resize samples source pixels with asymmetric
	// coordinate mapping, not ONNX's half_pixel convention.
	ctx.add_node("Resize", {input_name, roi_name, scales_name, sizes_name}, {output_name}, {
	make_attribute_string("mode", mode),
	make_attribute_string("coordinate_transformation_mode", "asymmetric")
	});
	const std::string sizes_name = add_int64_initializer(ctx, "resize_sizes", output_shape);
	// dlib's bilinear resize samples source pixels with asymmetric
	// coordinate mapping, not ONNX's half_pixel convention.
	ctx.add_node("Resize", {input_name, "", "", sizes_name}, {output_name}, {
	make_attribute_string("mode", mode),
	make_attribute_string("coordinate_transformation_mode", "asymmetric")
	});

[Compaile]

Can I ask, what types of models are you training with dlib? Cheers

Sure, for quite a lot actually. ranging from simple classification stuff but also a lot of metric learning etc, and also some custom loss layers (which i probably should pr some point like supcon and arcface) yolo never archived the same accuracy sadly then the "other" yolo repos and i did not have time to look into it further.

also i am on vacation currently and return in around a week. so srry for the late replys. also will look into the pr feedback ofc thanks for having a look

[pfeatherstone]

Yeah the metric learning loss is great