.Net 9 RC1 - Cannot set reponse headers when returning IAsyncEnumerable in controller

[gabynevada]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Had similar issues on .NET 6 release, adding just in case it's related:

• .Net 6 Preview 7 - Cannot set reponse headers before returning IAsyncEnumerable in controller #35364
• .Net 6 RC1 - Cannot set reponse headers in ExceptionMiddleware returning IAsyncEnumerable in controller #36977 (Was fixed after this one)

In .Net 9 RC1 when trying to set a header before yielding the first value on an IAsyncEnumerable, I get the following error:

      System.InvalidOperationException: Headers are read-only, response has already started.
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.ThrowHeadersReadOnlyException()
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.System.Collections.Generic.IDictionary<System.String,Microsoft.Extensions.Primitives.StringValues>.Add(String key, StringValues value)

Implementation of Controller looks like this:

    [HttpGet("set-header-aysnc-enumerable-error")]
    public async IAsyncEnumerable<WeatherForecastData> GetLocal([EnumeratorCancellation] CancellationToken cancellationToken)
    {
        var results = await dataService.GetData();
        // Adding a response header before yielding first result errors out
        Response.Headers.Add("Test", results.Name);
        await foreach (var result in results.Data.WithCancellation(cancellationToken))
        {
            yield return result;
        }
    }

Expected Behavior

The header to be set without errors as is possible in .NET 8

Usecase

We use this mostly to set Pagination headers for paginated lists

Steps To Reproduce

Use this webapi minimal reproduction

https://github.com/gabynevada/.net6-iasync-enumerable-set-header-error

Call endpoint returning IAsyncEnumerable

GET http://localhost:5000/api/data/set-header-aysnc-enumerable-error

Can replicate both on RC1 and the daily build for RC2

Exceptions (if any)

Connection id "0HN6MDAM4CBVH", Request id "0HN6MDAM4CBVH:00000001": An unhandled exception was thrown by the application.
      System.InvalidOperationException: Headers are read-only, response has already started.
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.ThrowHeadersReadOnlyException()
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpHeaders.System.Collections.Generic.IDictionary<System.String,Microsoft.Extensions.Primitives.StringValues>.Add(String key, StringValues value)
         at SetResponseHeaders.Controllers.WeatherForecastController.GetLocal(CancellationToken cancellationToken)+MoveNext() in /Users/elvisnievesmiranda/RiderProjects/.net6-iasync-enumerable-set-header-error/Controllers/WeatherForecastController.cs:line 18
         at SetResponseHeaders.Controllers.WeatherForecastController.GetLocal(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
         at System.Text.Json.Serialization.Converters.IAsyncEnumerableOfTConverter`2.OnWriteResume(Utf8JsonWriter writer, TAsyncEnumerable value, JsonSerializerOptions options, WriteStack& state)
         at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryWrite(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)
         at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
         at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.SerializeAsync(PipeWriter pipeWriter, T rootValue, Int32 flushThreshold, CancellationToken cancellationToken, Object rootValueBoxed)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.SerializeAsync(PipeWriter pipeWriter, T rootValue, Int32 flushThreshold, CancellationToken cancellationToken, Object rootValueBoxed)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.SerializeAsync(PipeWriter pipeWriter, T rootValue, Int32 flushThreshold, CancellationToken cancellationToken, Object rootValueBoxed)
         at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.SerializeAsync(PipeWriter pipeWriter, T rootValue, Int32 flushThreshold, CancellationToken cancellationToken, Object rootValueBoxed)
         at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeResultAsync>g__Logged|22_0(ResourceInvoker invoker, IActionResult result)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
         at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

.NET Version

9.0.100-rc.2.24466.1

Anything else?

IDE: JetBrains Rider 2024.2.4 on Mac M1 Arm 64

dotnet --info:

dotnet --info
.NET SDK:
 Version:           9.0.100-rc.2.24466.1
 Commit:            c4104e5646
 Workload version:  9.0.100-manifests.792c81b2
 MSBuild version:   17.12.0-preview-24463-04+8500d97af

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  14.6
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/9.0.100-rc.2.24466.1/

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
There are no installed workloads to display.

Host:
  Version:      9.0.0-rc.2.24462.10
  Architecture: arm64
  Commit:       static

[gabynevada]

Update, with minimal api it works.

The problem appears to be in Controllers:

app.MapGet(
    "/minimal-api",
    async (
        [FromServices] IDataService dataService,
        HttpContext httpContext,
        CancellationToken cancellationToken
    ) =>
    {
        var results = await dataService.GetData();

        httpContext.Response.Headers.Append("Test", results.Name);
        return results.Data;
    }
);

Sadly since it's a legacy app it will take us a while to migrate completely to minimal apis.

[BrennanConroy]

#57924 will fix this

[BrennanConroy]

Merged for RC2, thanks for trying out RC1 and finding this issue!

[toupswork]

wanted to share the following. if you are using system.text.json with source generation, the json serializer will not optimize the serialization unless you specifically add the type that you are returning to the context with it wrapped in the outer IAsyncEnumerable. I discovered this and the performance difference it made was astonishing