[C#/Linux] 1.29.0 SIGSEGVs at OrtEnv initialization in chiseled/distroless containers, POSIX telemetry regression (ORT_DISABLE_TELEMETRY=1 works around it)

[mjolivet-lucca]

Describe the issue

Upgrading the Microsoft.ML.OnnxRuntime NuGet package from 1.28.0 to 1.29.0 makes any .NET application crash with SIGSEGV (exit code 139) on linux-x64 when running inside a chiseled container image (mcr.microsoft.com/dotnet/aspnet:10.0-noble-chiseled). The crash happens at OrtEnv initialization, before any SessionOptions, InferenceSession or model is involved, so every consumer of the package is affected. The process dies silently: no managed exception, no native error message on stderr.

Instrumented run with stage markers (flushed after each stage):

[stage] start — ORT assembly 1.29.0.0
[stage] touching native lib via OrtEnv.Instance()...
<SIGSEGV, exit 139 — "native lib loaded" is never printed>

Results, consistent across repeated CI runs on GitHub-hosted amd64 runners (Xeon, AVX-512):

Configuration	Result
1.27.1 / 1.28.0, chiseled container	OK
1.29.0, chiseled container	SIGSEGV at OrtEnv.Instance(), deterministic
1.29.0, chiseled container, ORT_DISABLE_TELEMETRY=1	OK (session + inference all fine)
1.29.0, chiseled container, ORT_INTRA_OP_NUM_THREADS=1 only	SIGSEGV
1.29.0, same binary, full Ubuntu 24.04 host (no container)	OK, so this is specific to chiseled/distroless environments

Since ORT_DISABLE_TELEMETRY=1 alone fixes it and 1.28.0 is unaffected, this points to the POSIX telemetry introduced in 1.29.0 (#27379, #29872). Its initialization presumably dereferences something unavailable in minimal images (no /bin/sh, no systemd/dbus, minimal /etc) instead of failing gracefully. The 1.29.0 release notes say telemetry is only active "when ONNX Runtime is built with telemetry enabled"; the behavior above suggests the official NuGet linux-x64 binaries ship with it enabled.

This took down a production Kubernetes service for us: the SIGSEGV killed the pod on the first request touching ONNX Runtime, with nothing in the logs since the process dies before log buffers flush.

To reproduce

Console app (net10.0) referencing only Microsoft.ML.OnnxRuntime:

dotnet publish -c Release -o dist                     # -p:OnnxVersion=1.28.0 to compare
curl -sL -o dist/squeezenet.onnx https://github.com/microsoft/onnxruntime/raw/main/csharp/testdata/squeezenet.onnx
docker build -t onnx-repro .
docker run --rm onnx-repro                            # 1.29.0: exit 139 (SIGSEGV at OrtEnv.Instance())
docker run --rm -e ORT_DISABLE_TELEMETRY=1 onnx-repro # OK: session created, inference runs

The model is irrelevant: the crash occurs before any model is loaded. Squeezenet from this repo's C# test data is only there to show that everything works once the env var is set.

Dockerfile:

FROM mcr.microsoft.com/dotnet/aspnet:10.0-noble-chiseled
WORKDIR /app
COPY ./dist .
ENTRYPOINT ["dotnet", "MinimalOnnx.dll"]

Program.cs

using Microsoft.ML.OnnxRuntime;
using Microsoft.ML.OnnxRuntime.Tensors;

// Stage markers flushed at each step to pinpoint where the SIGSEGV happens.
void Stage(string msg)
{
    Console.WriteLine($"[stage] {msg}");
    Console.Out.Flush();
}

Stage($"start — ORT assembly {typeof(InferenceSession).Assembly.GetName().Version}");

Stage("touching native lib via OrtEnv.Instance()...");
var env = OrtEnv.Instance();
Stage($"native lib loaded — ORT native version: {OrtEnv.Instance().GetVersionString()}");

Stage("creating SessionOptions...");
using var options = new SessionOptions();
Stage("SessionOptions created");

Stage("creating InferenceSession (squeezenet.onnx)...");
using var session = new InferenceSession("squeezenet.onnx", options);
Stage("InferenceSession created");

var (inputName, meta) = session.InputMetadata.First();
var dims = meta.Dimensions.Select(d => d <= 0 ? 1 : d).ToArray();
Stage($"input '{inputName}' dims [{string.Join(",", dims)}] — building tensor...");
var tensor = new DenseTensor<float>(dims);
Stage("tensor built, running inference...");

for (var i = 1; i <= 3; i++)
{
    using var results = session.Run([NamedOnnxValue.CreateFromTensor(inputName, tensor)]);
    Stage($"inference pass {i} OK ({results.Count} output(s))");
}

Stage("SUCCESS — no native crash");

MinimalOnnx.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <InvariantGlobalization>true</InvariantGlobalization>
    <OnnxVersion Condition="'$(OnnxVersion)' == ''">1.29.0</OnnxVersion>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.ML.OnnxRuntime" Version="$(OnnxVersion)" />
  </ItemGroup>

</Project>

Urgency

High: crashes any .NET workload on chiseled/distroless images, a common production baseline for containerized .NET. A workaround exists (ORT_DISABLE_TELEMETRY=1) but is hard to discover because the crash is silent.

Platform

Linux

OS Version

mcr.microsoft.com/dotnet/aspnet:10.0-noble-chiseled (Ubuntu noble chiseled)

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.29.0

ONNX Runtime API

C#

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[edgchen1]

bmehta001 would you mind taking a look?

[bmehta001]

Hi Mathieu Jolivet (@mjolivet-lucca), Thanks for raising this issue and for the clear repro. I tested a fix in a chiseled container, and it should be in the next release.

[mjolivet-lucca]

Thanks a lot bmehta001 🙏