Process.Kill can hang indefinitely on macOS arm64

[mthalman]

Description

On macOS arm64, Process.Kill(entireProcessTree: true) can block indefinitely while terminating process trees created by the dotnet-watch tests.

This appeared after consuming the runtime flow containing #128598, which introduced the two-phase stop-then-kill implementation on Unix.

Enable dotnet-watch tests on mac once fixed: dotnet/sdk#55679

Reproduction Steps

1. From a dotnet/sdk checkout at commit dc6cef6, run the dotnet-watch.Tests suite on macOS 15 arm64. In CI, this is Helix shard dotnet-watch.Tests.dll.2 on queue osx.15.arm64.open.

2. Allow tests to dispose their child processes using Process.Kill(entireProcessTree: true).

3. Observe that some calls never return.

A standalone runtime-only reproduction has not yet been reduced. The failure reproduces in the SDK CI shard linked below.

Expected behavior

Process.Kill(entireProcessTree: true) terminates the process tree and returns promptly.

Actual behavior

Instrumentation logged nine calls entering Process.Kill(entireProcessTree: true), but only three returned. Helix eventually detected the hang, collected dumps, and terminated the test process:

[TEST AwaitableProcess.cs:237] Killing process tree for process 18251
Hang dump timeout of '00:48:00' expired
Test application process didn't exit gracefully, exit code is '137'

One separate call returned from Kill, but its process still did not exit within 30 seconds.

Regression?

Observed after the SDK consumed the runtime flow containing #128598. That PR changed Unix process-tree termination to recursively stop the tree before killing it.

Confidence is high that the immediate hang occurs synchronously inside Process.Kill(entireProcessTree: true). Confidence that #128598 introduced the regression is moderate pending a reduced runtime-only reproduction.

Known Workarounds

None currently. Bounding the subsequent wait does not help because the call to Process.Kill itself does not return.

Configuration

• macOS 15
• arm64
• Helix queue: osx.15.arm64.open
• Work item: dotnet-watch.Tests.dll.2
• .NET 11 runtime consumed by the SDK flow

Other information

• SDK Known Build Error: [macOS arm64] dotnet-watch cleanup hangs in Process.Kill(entireProcessTree: true) sdk#55636
• Failing build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1542039
• Helix job: https://helix.dot.net/api/jobs/2393a91c-17df-4ae9-b32c-faab6d292cf1/details
• Suspected runtime change: Fix Process.Kill(entireProcessTree: true) when intermediate child has KillOnParentExit #128598

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process
See info in area-owners.md if you want to be subscribed.

[adamsitnik]

Hi @mthalman

Thank you for a detailed bug report. I've hit hangs on macOS in #128598 but I was able to find the solution (#128598 (comment)) and get the dotnet/runtime CI green. Since then it never got stuck on macOS.

Were you able to collect a dump locally? Or at least attach a debugger and just check which methods exactly are blocked?
Do you happen to know if dotnet-watch.Tests.dll.2 does something unusual in terms of process management like p/invoking some native code that signs up for the SIGCHILD?

I am asking these questions because I don't have an access to macOS arm64 machine (I will somehow try to get it, but it may take some time).

[mthalman]

This happens in Helix. I don't know much about Helix so I don't know if they capture a dump or not. But an example build is at https://dev.azure.com/dnceng-public/public/_build/results?buildId=1542039. The hang seems to be sporadic.

[adamsitnik]

My mac is useless as I can't netiher build dotnet/runtime nor run the installed dotnet executable (it's simply too old and not supported). I've tried creating a standalone repro, but I've failed to reproduce the problem: https://github.com/adamsitnik/macosrepro

I suspect dotnet watch does something unusual that interferes with signal handling. I'll try to search for someone to try to debug it who has a working mac.