`aspire run --start-debug-session` hangs while stopping an orphaned AppHost

[adamint]

Is there an existing issue for this?

• I searched existing issues. aspire ps --format Json hangs in PowerShell pipelines because an orphaned aspire-managed helper holds stdout open #18779 is similar at the symptom level, but that issue was an orphaned aspire-managed process holding pipeline stdout open. This repro has no helper child and blocks during the existing AppHost auxiliary-backchannel handshake.

Describe the bug

An AppHost launched by the VS Code extension survived after its owning aspire run --start-debug-session CLI exited. Later aspire run and aspire ps commands found its auxiliary socket, connected at the transport level, and then hung indefinitely waiting for the AppHost to answer the initial RPC handshake.

The final visible CLI log entry was:

[dbug] BundleNuGetPackageCache: NuGet search returned 177986 bytes

That search had completed and was not the blocker. The stuck CLI had no child process and never reached AppHost build or launch.

Observed timeline

• 21:40: aspire run --start-debug-session --nologo started as PID 16380.
• 21:48: PID 16380 logged a termination signal and Exit code: 0.
• Its AppHost PID 16554 remained active under PPID 1, along with DCP and application resources.
• 22:41: a new VS Code extension host launched aspire ps --follow --format json --nologo; it connected to the existing AppHost.
• Three later aspire run --start-debug-session processes and a non-following aspire ps --format json process all remained stuck.
• The existing AppHost, DCP, API, worker, and all five later CLI processes were still alive more than an hour after the original launch.

Example process state:

16554     1 S   dotnet .../DotnetProject.AppHost.dll
16603     1 Ss  dcp start-apiserver --monitor 16554 ...
16604 16603 Ss  dcp run-controllers ...
17237 16941 Ss  aspire ps --follow --format json --nologo
17502 16941 S   aspire run --start-debug-session --nologo
19764 16941 Ss  aspire ps --format json --nologo
19912 16941 S   aspire run --start-debug-session --nologo
20505 16941 S   aspire run --start-debug-session --nologo --debug

The later run logs show DotNetAppHostProject connecting to the old auxiliary socket, but no response from the first RPC. The non-following ps log similarly stops after connecting to the same socket.

Relevant code path

RunCommand.ExecuteAsync awaits FindAndStopRunningInstanceAsync before launching. Its comment says a failed stop should not block AppHost startup, but this chain has no bounded handshake:

RunCommand.ExecuteAsync
  -> DotNetAppHostProject.FindAndStopRunningInstanceAsync
  -> RunningInstanceManager.StopRunningInstanceAsync
  -> AppHostAuxiliaryBackchannel.ConnectAsync
  -> CreateFromSocketAsync
  -> GetAppHostInformationAsync / GetCapabilitiesAsync

If the socket accepts but the AppHost never answers JSON-RPC, the command does not reach the intended StopFailed result and startup never continues.

There also appears to be a preceding lifecycle problem: CliOrphanDetector was active and had the original CLI PID plus stable start time, but the AppHost did not terminate after that PID exited.

Expected behavior

• A foreground AppHost should terminate when its owning CLI exits, or be collected as orphaned.
• An unhealthy auxiliary socket should not hang aspire run, aspire ps, or aspire stop indefinitely.
• The initial RPC handshake and stop attempt should be bounded. A failed existing-instance stop should return StopFailed and preserve the flow documented in RunCommand.

Environment

• Aspire CLI: 13.5.0+25c30ef12379ea5802c74bdd7fd7cf63351d9115
• CLI build ID: 13.500.26.41106
• Channel: staging
• AppHost source SHA: f5f526ec248e49a84875cfe1671594bf27c1d145
• AppHost version: 13.6.0-dev
• macOS 26.6 (25G72), Apple Silicon
• .NET SDK 10.0.201
• Started through the VS Code extension with --start-debug-session

[joperezr]

🤖 Aspire triage pass — 2026-08-12 03:12 UTC

Area: area-cli
Kind: bug
Repro: none — detailed observed timeline and process-state diagnostics, but no deterministic repro (see the "Observed timeline" section of the body)
Affected version(s): 13.5.0 staging CLI / 13.6.0-dev AppHost (latest shipped: 13.4.0)
Severity: S3 — aspire run/ps/stop hang indefinitely, but only after an orphaned-AppHost edge state is entered; manually killing the orphaned AppHost is a workaround

Urgency: ⚪ backlog — no milestone
Reason: not a regression (no prior-working version cited; observed on staging/dev builds), edge-triggered with a manual workaround; robustness hardening (bounded handshake) that can wait while 13.5 stabilizes

Suggested owner(s): James Newton-King (@JamesNK) (score high), backup Mitch Denny (@mitchdenny)
Evidence:

• James Newton-King (@JamesNK) authored the bulk of recent area-cli PRs (e.g. Resolve dotnet paths in doctor checks #19232, Suppress CLI progress when console logging is enabled #19214, Show status while preparing dashboard bundle #18985) — 54 merged in the last 90d
• James Newton-King (@JamesNK) closed the most area-cli issues as completed (e.g. [13.5] Trace logging on aspire doctor causes weird indentation #19017, Use resolved dotnet paths in aspire doctor #19003) in the last 180d
• CODEOWNERS lists Mitch Denny (@mitchdenny) James Newton-King (@JamesNK) David Fowler (@davidfowl) for /src/Aspire.Cli

Confidence: medium
Needs human?: no

[adamint]

This still looks reproducible from the current source (93b14b405). ConnectAsync can use an unbounded token, and CreateFromSocketAsync passes it directly to the initial GetAppHostInformationAsync and capabilities RPCs without adding a deadline.

The 10-second timeout in RunningInstanceManager starts later, while waiting for process termination. A socket that accepts the connection but never answers the initial RPC can therefore still block before that timeout is reached. I didn't open a live socket for this check, but the unbounded path is still there.

[danegsta]

Filed #19792 independently while reviewing #19762, then found it duplicates the handshake half of this issue. Closing that one; porting its analysis here so it isn't lost.

The unbounded handshake is a single code site with two callers

AppHostAuxiliaryBackchannel.CreateFromSocketAsync awaits two RPCs with only the caller's CancellationToken:

• src/Aspire.Cli/Backchannel/AppHostAuxiliaryBackchannel.cs:183 — GetAppHostInformationAsync
• src/Aspire.Cli/Backchannel/AppHostAuxiliaryBackchannel.cs:189 — FetchCapabilitiesAsync

There are exactly two callers, and each reaches the same hang by a different route:

• AppHostAuxiliaryBackchannel.ConnectAsync (line 140) → RunningInstanceManager.StopRunningInstanceAsync → the aspire run --start-debug-session hang reported above.
• AuxiliaryBackchannelMonitor.TryConnectToSocketAsync (AuxiliaryBackchannelMonitor.cs:561) → ScanAsync.

The monitor path amplifies the blast radius

ProcessDirectoryChangesAsync awaits every connect task via Task.WhenAll. So a single AppHost that accepts the socket but never answers blocks ScanAsync itself — and therefore every command that scans (aspire ps, aspire stop, aspire describe, and the launcher's connection wait), not just commands targeting the unresponsive AppHost. That matches the observed timeline here, where several unrelated CLI processes were stuck simultaneously.

The connect retry budget does not help. It bounds ConnectAsync only; once the connection is accepted, the handshake is unbounded.

Second, independent problem in the same block

rpc.StartListening() runs at line 178, before both RPC calls, and nothing disposes rpc if either call throws. JsonRpc owns the NetworkStream, which owns the socket (ownsSocket: true), so any handshake exception leaks a socket. This needs fixing regardless of the timeout, and a timeout fix must handle it, since a timeout is exactly this failure path.

Fix options considered

1. Bound the handshake inside CreateFromSocketAsync with a linked CTS, disposing rpc on failure. Fixes the root cause for both callers. The monitor can treat a handshake timeout like an unreachable socket so the existing MarkUnreachable exponential backoff parks it. Open question is the budget value: too aggressive and a slow-but-healthy AppHost on a loaded machine gets parked, possibly for minutes under the backoff. Backing off only after repeated handshake timeouts, rather than the first, would mitigate that.
2. Bound it only at the monitor call site. Smaller blast radius, but leaves the ConnectAsync path — the one that produced the report above — unprotected, and still needs the disposal fix.
3. Bound the whole scan via Task.WhenAll(connectTasks).WaitAsync(budget), letting stragglers land in Connections on a later pass. Single choke point with no per-RPC latency guesswork, but it changes ScanAsync's contract — callers assume connections are established when it returns, so aspire stop could act on an incomplete set. That trades a visible hang for silently missing an AppHost.

Option 1 looks like the right shape, with 3 as possible defense-in-depth.

Note that the orphan-lifecycle problem in the original report — the AppHost surviving its owning CLI's exit while CliOrphanDetector was active — is a genuinely separate defect from the handshake hang, and fixing the handshake will not address it.