System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingReceive_SendsReset failing in CI

[ahsonkhan]

https://dev.azure.com/dnceng/public/_build/results?buildId=316633 from dotnet/corefx#40446 on netcoreapp-Linux-Debug-x64-(Alpine.39.Amd64.Open)

System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingReceive_SendsReset(doRead: False)

System.IO.IOException : Unable to read data from the transport connection: Broken pipe.\n---- System.Net.Sockets.SocketException : Broken pipe

Stack trace
   at System.Net.Security.SslStream.<WriteSingleChunk>g__CompleteAsync|212_1[TWriteAdapter](ValueTask writeTask, Byte[] bufferToReturn) in /_/src/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 1236
   at System.Net.Security.SslStream.WriteAsyncInternal[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer) in /_/src/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 1486
   at System.Net.Test.Common.Http2LoopbackConnection.WriteFrameAsync(Frame frame) in /_/src/Common/tests/System/Net/Http/Http2LoopbackConnection.cs:line 88
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass62_0.<<Http2_PendingReceive_SendsReset>b__1>d.MoveNext() in /_/src/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1784
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 83
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 111
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 71
   at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Int32 timeout) in /_/src/Common/tests/System/Net/Http/Http2LoopbackServer.cs:line 186
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingReceive_SendsReset(Boolean doRead) in /_/src/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1736
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace -----

cc @davidsh, @wfurt

[wfurt]

we have 13 failures in last 90 days. First failure 7/23 so it predates dotnet/corefx#40123. Failing on Windows and Linux.

[ViktorHofer]

Just happened again: https://dnceng.visualstudio.com/public/_build/results?buildId=331643&view=ms.vss-test-web.build-test-results-tab&runId=9715232&resultId=158425&paneView=debug.

Seems to be happening even though we are using a Loopback Server. Any idea what could be causing this?

[stephentoub]

Failed again:
dotnet/corefx#40794
https://dev.azure.com/dnceng/public/_build/results?buildId=336431&view=ms.vss-test-web.build-test-results-tab&runId=9914380&resultId=101197&paneView=debug

Error message
System.IO.IOException : Unable to read data from the transport connection: Broken pipe.\n---- System.Net.Sockets.SocketException : Broken pipe

Stack trace
   at System.Net.Security.SslStream.<WriteSingleChunk>g__CompleteAsync|212_1[TWriteAdapter](ValueTask writeTask, Byte[] bufferToReturn) in /_/src/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 1236
   at System.Net.Security.SslStream.WriteAsyncInternal[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory`1 buffer) in /_/src/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 1486
   at System.Net.Test.Common.Http2LoopbackConnection.WriteFrameAsync(Frame frame) in /_/src/Common/tests/System/Net/Http/Http2LoopbackConnection.cs:line 88
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass62_0.<<Http2_PendingReceive_SendsReset>b__1>d.MoveNext() in /_/src/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1784
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 83
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 111
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 71
   at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Int32 timeout) in /_/src/Common/tests/System/Net/Http/Http2LoopbackServer.cs:line 186
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingReceive_SendsReset(Boolean doRead) in /_/src/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1736
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace -----

[AriNuer]

Same test System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_PendingReceive_SendsReset(doRead: False) failed here:

Error message
Assert.NotNull() Failure

Stack trace
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.<>c__DisplayClass62_0.<<Http2_PendingReceive_SendsReset>b__1>d.MoveNext() in /_/src/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1782
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 83
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 111
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 77
   at System.Net.Test.Common.Http2LoopbackServer.CreateClientAndServerAsync(Func`2 clientFunc, Func`2 serverFunc, Int32 timeout) in /_/src/Common/tests/System/Net/Http/Http2LoopbackServer.cs:line 186
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http2.Http2_PendingReceive_SendsReset(Boolean doRead) in /_/src/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http2.cs:line 1736
--- End of stack trace from previous location where exception was thrown ---

Details:
https://dnceng.visualstudio.com/public/_build/results?buildId=338052&view=ms.vss-test-web.build-test-results-tab&runId=9958168&resultId=156173&paneView=debug

[Anipik]

It failed in release/3.0 branch as well. do we consciously didnot port it to release/3.0 ?

[ViktorHofer]

We usually port test fixes in an aggregated manner. I did that about 10 days ago and will do it again soon.