Skip to content

.NET: [Bug]: Stabilize Foundry recovery tests #7816

Description

Description

Two Foundry recovery tests have synchronization gaps.

  1. StoppedHost_RecoversWorkflowWithCompleteOrderedOutputAsync waits until the live HTTP response contains 6, 5, and 4, then immediately reads the durable response file. AgentServer exposes live output before processing the checkpoint control event, so the file can still contain only 6 and 5 when the host stops.
  2. BackgroundResponse_ProcessCrash_RecoversAndCompletesAsync calls Environment.Exit(70) immediately after the workflow input stage. The exit can race the first durable workflow boundary. The recovered response can reach Completed with workflow action items only and no CRASH-RECOVERED message. Polling during process replacement can also return HTTP 409 session_updating, which asks the caller to retry.

Expected behavior:

  • The unit test should stop only after AgentServer processes the expected checkpoint control event.
  • The live test should allow the workflow boundary to become durable before intentionally ending the process.
  • Recovery polling should retry temporary 409, 404, and 424 responses.

Steps to reproduce:

  1. Run either affected test repeatedly.
  2. The unit test can compare expected output 6, 5, and 4 with persisted output 6 and 5.
  3. The live test can return Completed with no message output, or fail polling with HTTP 409 session_updating.

Error Messages / Stack Traces

Assert.Equal() Failure: Collections differ
Expected: ["6", "5", "4"]
Actual:   ["6", "5"]

Assert.Contains() Failure: Sub-string not found
String:    null
Not found: "CRASH-RECOVERED:<token>:PROCESS-CHANGED"

HTTP 409 (ServiceError: session_updating)
Session '<id>' is currently updating. Please try again after some time.

Package Versions

Repository tests for Microsoft.Agents.AI.Foundry.Hosting.

.NET Version

.NET 10

Additional Context

Observed in these workflow jobs:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    .NETUsage: [Issues, PRs], Target: .Net

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions