Skip to content

server : enable gzip compression (zlib) - #28059

Open
jhen0409 wants to merge 2 commits into
ggml-org:masterfrom
jhen0409:jhen/server-zlib
Open

jhen0409 wants to merge 2 commits into
ggml-org:masterfrom
jhen0409:jhen/server-zlib

Conversation

@jhen0409

@jhen0409 jhen0409 commented Aug 31, 2026 •

Copy link
Copy Markdown
Member

Overview

Ref #27357 (gzip only, no brotli).

This adds a LLAMA_ZLIB CMake option (default ON, gracefully disabled when zlib is not found) that builds the vendored cpp-httplib with CPPHTTPLIB_ZLIB_SUPPORT. cpp-httplib already implements both directions:

  • Requests with Content-Encoding: gzip are decompressed automatically (decompressed size capped by CPPHTTPLIB_PAYLOAD_MAX_LENGTH, 100MB).
    This helps us save a lot of traffic in long, multi-turn conversations or even multimodal message conversations on compression API requests.
  • JSON responses are compressed per Accept-Encoding negotiation. SSE streaming is unaffected (text/event-stream is excluded by httplib).
  • SSE streaming is unaffected (text/event-stream is excluded by httplib)

The fix in cpp-httplib

Enabling zlib exposed a double-encoding bug in cpp-httplib: the web UI serves build-time-gzipped assets with Content-Encoding: gzip set by the handler, and httplib compresses them a second time (Safari renders raw gzip bytes). The patch skips compression when the handler already set Content-Encoding, deciding the coding once so headers and body agree.

I'm submitted this patch upstream (PR: yhirose/cpp-httplib#2575, merged) and will keep this PR as a draft until the upstream author agrees with the approach the cpp-httplib released new version.

Additional information

Validation:

  • gzip request body on /v1/chat/completions (text + image): same 200 and identical prompt_tokens as the uncompressed request; the gzip request even hits the prompt cache populated by the plain one, confirming the decompressed bytes are byte-identical.
  • Web UI loads correctly on Safari and Chrome (single Content-Encoding header, single gzip layer); Accept-Encoding: gzip on /props returns a compressed response; SSE streaming unaffected.

Possible follow-ups: compressing request bodies from the web UI, brotli.

Requirements

@jhen0409

Copy link
Copy Markdown
Member Author

Rebased master, now it's ready for review.

Also, I found it cannot set quality param for brotli, so it's significantly increases response time by default, I think the benefits of brotli support to be minimal at the moment.

@jhen0409
jhen0409 marked this pull request as ready for review September 12, 2026 13:00
@jhen0409
jhen0409 requested review from a team and ggerganov as code owners September 12, 2026 13:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Compilation issues server vendor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant