Repository navigation
Write JSON results through a chunked pipe buffer - #10552
Merged
Merged
Conversation
The response formatters write each response into a 64 KB buffer that is copied to the response pipe when it is full or the formatter flushes, instead of writing every token straight into Kestrel's pipe. The buffer is held only while a payload is being written, a write of 4 KB or more goes to the pipe directly, and the writer is cached per thread so a response allocates nothing it did not allocate before.
Contributor
Patch coverage83.2% of changed lines covered (124/149)
Uncovered changed lines (JSON){
"sha": "c12b5734a56974c0f53e0aa99c7695a39c5d7c6a",
"files": [
{ "path": "src/HotChocolate/AspNetCore/src/Transport.Formatters/JsonResultFormatter.cs", "ranges": [[245, 248]] },
{ "path": "src/HotChocolate/AspNetCore/src/Transport.Formatters/ChunkedPipeBufferWriter.cs", "ranges": [[70, 73], [76, 77], [99, 99], [102, 106], [108, 110], [157, 159], [190, 191]] },
{ "path": "src/HotChocolate/AspNetCore/src/Transport.Formatters/JsonLinesResultFormatter.cs", "ranges": [[50, 50]] }
]
}Project coverage: 59.0% (311132/527557 lines) |
A message whose formatting throws is discarded from the buffer, so the next message of a batch or stream does not continue a truncated one. Advance is validated against the buffer like the pipe does. The multipart batch reuses its writer for streams instead of renting a second one.
This was referenced Oct 25, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The response formatters write every JSON token straight into Kestrel's response
PipeWriter, which takes a lock on eachGetSpanandAdvance, so a response pays thousands of lock operations.Every formatter now writes through a
ChunkedPipeBufferWriter: a 64 KB buffer that is copied to the pipe when it is full or the formatter flushes. The buffer is rented on the first write and returned as soon as it is copied, before the flush is awaited, so a stream holds nothing between events. A write of 4 KB or more goes to the pipe directly. The writer is cached per thread, so a response allocates nothing it did not allocate before. Flush points are unchanged: single results flush once at the end, streams per event.Benchmarks
Results are written into a plain pipe and into a pipe that locks on every buffer call like Kestrel's response writer. The 10 KB and 47 KB results are the gateway benchmark response with one and ten users, as a single result, a batch of five and an event stream of five. The small results are product lists from 292 bytes to 5.6 KB, and the ones below 4 KB also run as a batch of five and an event stream of five. The scalar results are four pre-encoded UTF-8 strings of the given size, written the way Fusion writes source result strings.
Small result, 292 bytes
mainmainSmall result, 1.1 KB
mainmainSmall result, 2.7 KB
mainmainSmall result, 5.6 KB
mainmainFour 2 KB scalars
mainmainSingle 10 KB result
mainmainSingle 47 KB result
mainmainBatch of five 292 byte results
mainmainEvent stream of five 292 byte results
mainmainBatch of five 1.1 KB results
mainmainEvent stream of five 1.1 KB results
mainmainBatch of five 2.7 KB results
mainmainEvent stream of five 2.7 KB results
mainmainBatch of five 10 KB results
mainmainEvent stream of five 10 KB results
mainmainBatch of five 47 KB results
mainmainEvent stream of five 47 KB results
mainmain