Skip to content

Send single JSON responses with a Content-Length - #10700

Merged
michaelstaib merged 4 commits into
ChilliCream:mainfrom
michaelstaib:perf/response-content-length
Oct 26, 2026
Merged

michaelstaib merged 4 commits into
ChilliCream:mainfrom
michaelstaib:perf/response-content-length

Conversation

@michaelstaib

Copy link
Copy Markdown
Member

Single GraphQL JSON responses were sent with chunked encoding, so after the body Kestrel had to send the closing chunk in a second, separate network write on every request. The server now formats the whole response into a pooled buffer first, sets the Content-Length header and writes the body in one go, so each response leaves in a single send and the separate end-of-response write is gone. Batched, streamed (defer, subscriptions, SSE, JSON lines) and multipart responses are unchanged, and a formatting error still aborts the request instead of sending a partial body.

Metric Before Optimization After Optimization
Throughput 4,468 req/s 4,657 req/s (+4.2%)
Latency p50 9.94 ms 9.61 ms
Latency p95 20.15 ms 18.88 ms
CPU per request 0.808 ms 0.786 ms
Memory allocated per request 32.5 KB 31.8 KB
GC collections per second 12.5 10.8
Memory usage (avg working set) 166.1 MB 171.1 MB

Measured on the federation benchmark twin machine with the benchmark's deep-recursion query (constant 50 VUs, gateway pinned to 4 logical cores), median of 4 interleaved 60 s runs against main. This applies to every HotChocolate server, not only the Fusion gateway: single JSON responses now carry a Content-Length header instead of Transfer-Encoding: chunked.

DefaultHttpResponseFormatter now formats single application/json and
application/graphql-response+json results into an internal pooled buffer,
sets Content-Length, starts the response and copies the body into the
response pipe (flushing at most every 1 MB). Kestrel then sends the body
without chunk framing and without the separate chunk terminator send.

A formatter exception still aborts the request instead of producing a 500.
OnStarting callbacks now run after the body is formatted. Batch, stream,
multipart, SSE and JSONL responses stay chunked.
Removes two comments that explained implementation details instead of contracts.
@michaelstaib
michaelstaib merged commit 74f28ce into ChilliCream:main Oct 26, 2026
152 checks passed
@michaelstaib
michaelstaib deleted the perf/response-content-length branch October 26, 2026 19:25
This was referenced Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant