[Build] C++ API cannot be reliably linked with an program using CMake

[hkerma]

Describe the issue

Hello,
Thank you for maintaining this project.
I am having issues trying to link my target C++ program with ONNX Runtime using CMake.

Context: I want to statically link ONNX Runtime with a C++ inference example (for instance: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/c_cxx/MNIST/MNIST.cpp). I compiled ONNX runtime and its dependencies as static libraries (using CMake), and I am also building the target program using CMake. Below I added the CMake command I used to compile ONNX Runtime as well as my own building script.

Issue: It seems that ONNX Runtime does not generate any FindOnnxruntime.cmake file, nor a onnxruntimeConfig.cmake file. Because there are a tons of dependencies compiled and generated, this makes it very difficult for me to then compile my own program and statically link all required libraries. I was able to make one example work by manually adding a onnxruntimeConfig.cmake file, then manually add a lot of .a filepaths to my CMakeLists.txt, but this is very unpractical (for instance, whenever I change the model, new dependencies must be added.

Am I missing something? The ideal scenario would be that, given a model.onnx file, I compile just the required features (using the --include_ops_by_config flag), then simply use something like find_package(onnxruntime CONFIG REQUIRED) in my CMake file to automatically add and compile needed libraries. There is a probably a "simple" way to do that, but I spent quite some time on it and could not find it (probably due to my lack of experience with CMake projects).

Related issues:

• Can you guys provide a clear CMakeLists.txt example on how to include onnxruntime for a C++ project? #3124 , in particular this message, however I did not succeed to make it work
• CMake build doesn't generate a config file #10818

Any help appreciated! Thanks

Urgency

No response

Target platform

Linux x86_64

Build script

To compile ONNX Runtime I just run ./build.sh --config=Release --minimal_build --skip_tests --parallel 12 in the general case.

To compile my sample app, the only way I found was to:

• add a custom onnxruntimeConfig.cmake (see below) in ./build/Linux/Release (I forgot where I found this sorry)
• use the following CMakeLists.txt file (the amount/names of libraries to be imported, in particular from the absl lib, largely depends on the model compiled. I have attached the sample code corresponding to this particular CMakeLists.txt)
• simply run cmake . then make. If there are errors "undefined reference to ...", I jut have to manually find which library it comes from, and add it in the file.

onnxruntimeConfig.cmake

get_filename_component(PACKAGE_PREFIX_DIR "${CMAKE_CURRENT_LIST_DIR}/../../../" ABSOLUTE)

get_filename_component(CMAKE_CURRENT_LIST_DIR "${CMAKE_CURRENT_LIST_FILE}" PATH)
get_filename_component(onnxruntime_INSTALL_PREFIX "${CMAKE_CURRENT_LIST_DIR}/../../../" ABSOLUTE)

set(onnxruntime_INCLUDE_DIRS ${onnxruntime_INSTALL_PREFIX}/include)
set(onnxruntime_LIBRARIES onnxruntime)
set(onnxruntime_CXX_FLAGS "") # no flags needed


macro(set_and_check _var _file)
  set(${_var} "${_file}")
  if(NOT EXISTS "${_file}")
    message(FATAL_ERROR "File or directory ${_file} referenced by variable ${_var} does not exist !")
  endif()
endmacro()

macro(check_required_components _NAME)
  foreach(comp ${${_NAME}_FIND_COMPONENTS})
    if(NOT ${_NAME}_${comp}_FOUND)
      if(${_NAME}_FIND_REQUIRED_${comp})
        set(${_NAME}_FOUND FALSE)
      endif()
    endif()
  endforeach()
endmacro()

####################################################################################
#include("${CMAKE_CURRENT_LIST_DIR}/onnxruntimeTargets.cmake")

CMakeLists.txt

cmake_minimum_required(VERSION 3.5)
project(model)
set(CMAKE_CXX_STANDARD 17 )


set(ONNXRUNTIME "/home/vagrant/onnxruntime/build/Linux/Release")
list(APPEND CMAKE_PREFIX_PATH ${ONNXRUNTIME})
find_package(onnxruntime REQUIRED)

set(PROTOBUF "/home/vagrant/onnxruntime/build/Linux/Release/_deps/protobuf-build")
list(APPEND CMAKE_PREFIX_PATH ${PROTOBUF})
find_package(Protobuf REQUIRED)

include_directories(${onnxruntime_INCLUDE_DIRS})
include_directories(${onnxruntime_INCLUDE_DIRS}/onnxruntime/core/session)

set(onnxruntime_LIBRARIES
    -Wl,--no-whole-archive
    -Wl,--start-group
    ${ONNXRUNTIME}/libonnxruntime_common.a
    ${ONNXRUNTIME}/libonnxruntime_flatbuffers.a
    ${ONNXRUNTIME}/libonnxruntime_framework.a
    ${ONNXRUNTIME}/libonnxruntime_graph.a
    ${ONNXRUNTIME}/libonnxruntime_mlas.a
    ${ONNXRUNTIME}/libonnxruntime_optimizer.a
    ${ONNXRUNTIME}/libonnxruntime_providers.a
    ${ONNXRUNTIME}/libonnxruntime_session.a
    ${ONNXRUNTIME}/libonnxruntime_util.a
    ${ONNXRUNTIME}/libonnx_proto.a
    ${ONNXRUNTIME}/libonnx.a
    ${ONNXRUNTIME}/_deps/google_nsync-build/libnsync_cpp.a
    -Wl,--end-group
)

set(absl_LIBRARIES
    -Wl,--no-whole-archive
    -Wl,--start-group
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/base/libabsl_base.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/base/libabsl_raw_logging_internal.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/base/libabsl_spinlock_wait.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/base/libabsl_malloc_internal.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/base/libabsl_throw_delegate.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/container/libabsl_raw_hash_set.a
    # ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/container/libabsl_hashtablez_sampler.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/hash/libabsl_low_level_hash.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/hash/libabsl_hash.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/hash/libabsl_city.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/strings/libabsl_str_format_internal.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/strings/libabsl_strings.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/strings/libabsl_strings_internal.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/synchronization/libabsl_synchronization.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/synchronization/libabsl_kernel_timeout_internal.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/time/libabsl_time.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/time/libabsl_civil_time.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/time/libabsl_time_zone.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/numeric/libabsl_int128.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/debugging/libabsl_stacktrace.a
    ${ONNXRUNTIME}/_deps/abseil_cpp-build/absl/debugging/libabsl_debugging_internal.a
    -Wl,--end-group
)

set(other_LIBRARIES
    -Wl,--no-whole-archive
    -Wl,--start-group
    ${ONNXRUNTIME}/_deps/pytorch_cpuinfo-build/libcpuinfo.a
    ${ONNXRUNTIME}/_deps/re2-build/libre2.a
    -Wl,--end-group
)

message(STATUS "onnxruntime_INCLUDE_DIRS ${onnxruntime_INCLUDE_DIRS}")
message(STATUS "onnxruntime_LIBRARIES ${onnxruntime_LIBRARIES}")
message(STATUS "PROTOBUF_LIBRARIES ${PROTOBUF_LIBRARIES}")

add_executable(model model.cpp)

target_link_libraries(model
    ${onnxruntime_LIBRARIES}
    ${PROTOBUF_LIBRARIES}
    ${other_LIBRARIES}
    ${absl_LIBRARIES}
)

The code I use to generate model.onnx (this varies depending on the model I want, and so do the dependencies later required in CMakeLists.txt

import torch

class SimpleMLP(torch.nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(SimpleMLP, self).__init__()
        self.fc1 = torch.nn.Linear(input_size, hidden_size)
        self.relu = torch.nn.ReLU()
        self.fc2 = torch.nn.Linear(hidden_size, output_size)

    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        return out


# Instantiate and train your model
model = SimpleMLP(10,20,50)


# Train your model here

# Dummy input for the export
dummy_input = torch.randn(1, 10)

# Export the model
torch.onnx.export(model, dummy_input, "model.onnx", 
                  input_names=['input'], output_names=['output'],
                  dynamic_axes={'input': {0: 'batch_size'}, 'output': {0: 'batch_size'}})

Error / output

If I don't manually add a onnxruntimeConfig.cmake file, I get an error:

CMake Error at CMakeLists.txt:8 (find_package): By not providing "Findonnxruntime.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "onnxruntime", but CMake did not find one.
    onnxruntimeConfig.cmake
    onnxruntime-config.cmake

Once I add it, I usually get a lot of undefined reference to (<either absl, libprotobuf, re2, etc.>) errors, until I add it manually in my CMakeLists.txt. Once I added all required dependencies, it works well, until I want to use a different model.

Visual Studio Version

n.a.

GCC / Compiler Version

g++ 11.4 / cmake 3.30.0

[jordanozang]

Hi, just wanted to mention that I am working on this. See draft pull request #22173. Example repo coming soon.