Skip to content

feat(rpc): Add request decompression - #20277

Merged
mattsse merged 1 commit into
paradigmxyz:mainfrom
evmKnows:request-decompression
Aug 13, 2026
Merged

feat(rpc): Add request decompression#20277
mattsse merged 1 commit into
paradigmxyz:mainfrom
evmKnows:request-decompression

Conversation

@evmKnows

@evmKnows evmKnows commented Dec 10, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Closes: #20237

  • Added HTTP request decompression layer
  • Added option for request decompression algorithms
  • Added option for response compression algorithms
  • Disabled brotli by default for response compression

Request-content negotiation is currently reactive: if a request uses an unsupported Content-Encoding, Reth returns 415 Unsupported Media Type with an Accept-Encoding header listing the supported encodings. RFC 9110 §12.5.3 also permits advertising this in successful responses, so proactive discovery can be added later if useful.

Edit
Compression alg selection split off to #26668

@github-project-automation github-project-automation Bot moved this to Backlog in Reth Tracker Dec 10, 2025
@evmKnows
evmKnows marked this pull request as ready for review December 10, 2025 21:46
@github-actions github-actions Bot added the S-stale This issue/PR is stale and will close with no further activity label Jan 1, 2026
@github-actions github-actions Bot removed the S-stale This issue/PR is stale and will close with no further activity label Jan 13, 2026
@emmajam

emmajam commented Feb 25, 2026

Copy link
Copy Markdown
Member

Hey! We're doing some spring cleaning on our PR backlog 🧹 Closing old PRs to keep things tidy. If this is still relevant, please feel free to re-open — we appreciate your contribution!

@emmajam emmajam closed this Feb 25, 2026
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Reth Tracker Feb 25, 2026
@pata-eth

pata-eth commented Aug 11, 2026

Copy link
Copy Markdown

hi, folks. I just ran into uplink issues where request compression would be handy. @evmKnows , any chance you'd be interested in re-opening the PR?

@evmKnows

evmKnows commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

@pata-eth sure np

cc: @emmajam @mattsse
cannot reopen myself

@emmajam emmajam reopened this Aug 12, 2026
@github-project-automation github-project-automation Bot moved this from Done to In Progress in Reth Tracker Aug 12, 2026
@mattsse

mattsse commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

adding this to my list, thanks for bumping and sorry for dropping

@evmKnows evmKnows mentioned this pull request Aug 12, 2026
Merged
@evmKnows
evmKnows force-pushed the request-decompression branch from d02e082 to 68cd09d Compare August 12, 2026 17:26
@evmKnows

evmKnows commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

@mattsse

Updated and split into two PRs

Note on perf: Decompressed bodies are buffered before jsonrpsee, so concurrent requests push RSS up; jemalloc keeps freed pages for reuse. An internal concurrency limit (e.g. derived from conn limit / 8) would be sensible.

For practical use (or if considering defaults) would recommend to roll with zstd; browser native support still absent for zstd.

Alloy might be a good place to plug into zstd directly, so that (de)compressed size hints are actually used + populated when compressing requests. Last time I checked zstd-rs is omitting those hints by default, though exposes an API to r/w hints, and tower (or related deps) aren't doing anything with either.

@mattsse mattsse added the M-changelog This change should be included in the changelog label Aug 13, 2026
@mattsse
mattsse force-pushed the request-decompression branch 2 times, most recently from 817c237 to 96ac221 Compare August 13, 2026 11:35
@mattsse
mattsse force-pushed the request-decompression branch from 96ac221 to 356dc3f Compare August 13, 2026 12:21
@mattsse
mattsse enabled auto-merge August 13, 2026 12:23
@mattsse
mattsse added this pull request to the merge queue Aug 13, 2026
Merged via the queue into paradigmxyz:main with commit ebab368 Aug 13, 2026
34 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Reth Tracker Aug 13, 2026
@decofe decofe mentioned this pull request Aug 14, 2026
Merged
teyrebaz33 pushed a commit to teyrebaz33/tempo that referenced this pull request Aug 25, 2026
@decofe @shekhirin
@mattsse @SuperFluffy
deps: update reth from main (2026-08-25) (tempoxyz#7142)
b110f4e 
Automated nightly update of reth dependencies from `paradigmxyz/reth`
main branch.

## Upstream reth changes


[`10aa6a5...00ff650`](paradigmxyz/reth@10aa6a5...00ff650)

🔗 Amp thread:
https://ampcode.com/threads/T-01a036f1-9f58-7640-b789-4e3dd9778ebe
- **Engine**
- Improved payload building across canonical ancestors, finality,
persistence handoffs, pending resolution, and responsive cancellation
([#26559](paradigmxyz/reth#26559),
[#26567](paradigmxyz/reth#26567),
[#26580](paradigmxyz/reth#26580),
[#26708](paradigmxyz/reth#26708),
[#26759](paradigmxyz/reth#26759)).
- Added Bogota Engine API support and fork-time validation, plus FOCIL
inclusion-list construction and stubs
([#26682](paradigmxyz/reth#26682),
[#26706](paradigmxyz/reth#26706),
[#26711](paradigmxyz/reth#26711),
[#26737](paradigmxyz/reth#26737)).
- Tightened payload and block-access-list validation, including
state-gas admission and malformed BAL rejection
([#26651](paradigmxyz/reth#26651),
[#26694](paradigmxyz/reth#26694),
[#26719](paradigmxyz/reth#26719)).
- Fixed little-endian cell bitvectors, Osaka `getBlobsV4`, and
prewarm-worker shutdown
([#26650](paradigmxyz/reth#26650),
[#26703](paradigmxyz/reth#26703),
[#26768](paradigmxyz/reth#26768)).
- Reused scratch buffers for faster BAL hash encoding
([#26701](paradigmxyz/reth#26701)).

- **RPC**
- Added `debug_traceChain`, Alloy trace-chain result types, block-level
EVM reuse, and raw block transactions on the auth server
([#26582](paradigmxyz/reth#26582),
[#26614](paradigmxyz/reth#26614),
[#26669](paradigmxyz/reth#26669),
[#26760](paradigmxyz/reth#26760)).
- Added configurable response compression and request decompression
([#26668](paradigmxyz/reth#26668),
[#20277](paradigmxyz/reth#20277)).
- Added Bogota Engine API stubs and Amsterdam system contracts to
`eth_config` ([#26691](paradigmxyz/reth#26691),
[#26705](paradigmxyz/reth#26705)).
- Fixed cancellation of payload-hash and blocking-I/O work
([#26569](paradigmxyz/reth#26569),
[#26776](paradigmxyz/reth#26776)).
- Corrected execution-witness block identifiers, optional receipt
conversion/caching, and access-list environment preparation
([#26572](paradigmxyz/reth#26572),
[#26596](paradigmxyz/reth#26596),
[#26599](paradigmxyz/reth#26599),
[#26598](paradigmxyz/reth#26598)).
- Fixed testing block gas limits, transaction gas-limit preservation,
timestamp overflow, and chain-ID validation
([#26632](paradigmxyz/reth#26632),
[#26743](paradigmxyz/reth#26743),
[#26767](paradigmxyz/reth#26767),
[#26782](paradigmxyz/reth#26782)).
- Added network-specific log responses and generic testing RPC handlers
([#26491](paradigmxyz/reth#26491),
[#26547](paradigmxyz/reth#26547)).

- **Networking**
- Added ingress limits, configurable no-op client versions, and outbound
`GetCells` support
([#26660](paradigmxyz/reth#26660),
[#26652](paradigmxyz/reth#26652),
[#26673](paradigmxyz/reth#26673)).
- Improved handshake and protocol safety through ECIES identity checks,
message-ID validation, negotiated-protocol assertions, and bad-message
handling ([#26639](paradigmxyz/reth#26639),
[#26654](paradigmxyz/reth#26654),
[#26659](paradigmxyz/reth#26659),
[#26671](paradigmxyz/reth#26671)).
- Fixed ping/pong validation and pacing
([#26698](paradigmxyz/reth#26698),
[#26702](paradigmxyz/reth#26702)).
- Made eth/72 blob-cell announcements interoperable with geth while
preserving availability masks
([#26573](paradigmxyz/reth#26573),
[#26670](paradigmxyz/reth#26670)).
- Shared the snap/2 slim-account codec between client and server
([#26587](paradigmxyz/reth#26587)).

- **Txpool**
- Added blob-cell availability tracking and exposure on pooled
transactions ([#25463](paradigmxyz/reth#25463),
[#26642](paradigmxyz/reth#26642)).
- Included blob-pool transactions in queued counts and listings
([#26677](paradigmxyz/reth#26677),
[#26679](paradigmxyz/reth#26679)).
- Allowed senders with empty code hashes, added consensus encoding, and
converted sender accessors to iterators
([#26644](paradigmxyz/reth#26644),
[#26739](paradigmxyz/reth#26739),
[#26681](paradigmxyz/reth#26681)).

- **Trie & State**
- Added partial trie unwind and persistence support, including
changeset-cache handling
([#26543](paradigmxyz/reth#26543),
[#26612](paradigmxyz/reth#26612)).
- Corrected witness construction to use depth-first node order
([#26707](paradigmxyz/reth#26707)).
- Simplified `HashedPostState` wipe handling and added trie-data
reference collectors
([#26524](paradigmxyz/reth#26524),
[#26752](paradigmxyz/reth#26752)).

- **Storage & Providers**
- Removed `ConsistentDbView`, relocated `OverlayStateProvider`, and
simplified provider bounds
([#26581](paradigmxyz/reth#26581),
[#26611](paradigmxyz/reth#26611),
[#26591](paradigmxyz/reth#26591)).
- Fixed storage-wipe handling during batched persistence
([#26750](paradigmxyz/reth#26750)).
- Made RocksDB tolerate unknown column families
([#26647](paradigmxyz/reth#26647)).
- Updated static-file consistency checks to respect prune checkpoints
([#26565](paradigmxyz/reth#26565)).

- **Chainspec & Consensus**
- Added Bogota hardfork support and validated block-access-list hashes
during import ([#26686](paradigmxyz/reth#26686),
[#26696](paradigmxyz/reth#26696)).
- Honored the genesis `slotNumber` instead of hardcoding zero
([#26680](paradigmxyz/reth#26680)).
- Defaulted unspecified payload attributes
([#26684](paradigmxyz/reth#26684)).

- **DNS**
- Fixed EIP-1459 discovery records by rejoining long TXT character
strings and ignoring unrelated TXT records
([#26602](paradigmxyz/reth#26602),
[#26603](paradigmxyz/reth#26603)).

- **Snapshots & CLI**
- Added base-URL resolution and exposed prepared snapshot context
([#26576](paradigmxyz/reth#26576),
[#26777](paradigmxyz/reth#26777)).
- Fixed history downloads when the final snapshot chunk is partial
([#26607](paradigmxyz/reth#26607)).
- Added bootnode configuration to `reth.toml`
([#26551](paradigmxyz/reth#26551)).

- **Testing & Development**
- Improved engine reorg tests with explicit finality management and
expanded execute-blob Hive coverage
([#26584](paradigmxyz/reth#26584),
[#26606](paradigmxyz/reth#26606)).
- Added a persistent-datadir testing node and made dev-mined blocks
canonical immediately
([#26774](paradigmxyz/reth#26774),
[#26761](paradigmxyz/reth#26761)).

- **Bench**
- Restored metrics visibility in benchmark run configurations
([#26461](paradigmxyz/reth#26461)).

- **Dependencies & Releases**
- Updated Alloy to 2.4.x, `alloy-hardforks`/`alloy-eip7928` to 0.4.8,
and released Reth 2.5.0–2.5.1
([#26663](paradigmxyz/reth#26663),
[#26666](paradigmxyz/reth#26666),
[#26685](paradigmxyz/reth#26685),
[#26687](paradigmxyz/reth#26687),
[#26700](paradigmxyz/reth#26700),
[#26771](paradigmxyz/reth#26771)).

## Migrations

🔗 Amp thread:
https://ampcode.com/threads/T-01a036f2-23b2-77c0-8959-5c3f0f9df6c5
- Upgraded Reth to `00ff650`, Alloy to `2.4.1`, and related dependencies
to match their latest APIs.
- Renamed workspace lint keys from kebab-case to snake_case for updated
Cargo lint syntax.
- Removed the no-longer-needed crate recursion limit.
- Migrated hashed storage construction from the removed `from_iter` API
to direct struct initialization.
- Reused Reth’s prepared snapshot manifest, base URL, and data
directory, removing Tempo’s duplicate manifest discovery, fetching,
parsing, and path-resolution logic.
- Updated snapshot planning to handle Reth’s new `(plan, prepared)`
return value and execution’s optional prepared manifest.
- Updated `HashedPostStateProvider` implementations and callers for its
new fallible `ProviderResult` return type.
- Added the required RPC log associated type and identity `convert_log`
implementation for the updated receipt converter trait.
- Updated pooled transaction construction for the reordered transaction
field and new `blob_cell_availability` field.

[GitHub
Workflow](https://github.com/tempoxyz/tempo/actions/runs/32804636065)

---------

Co-authored-by: Alexey Shekhirin <github@shekhirin.com>
Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com>
Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com>
Co-authored-by: Richard Janis Goldschmidt <701177+SuperFluffy@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattsse mattsse mattsse approved these changes
@Rjected Rjected Awaiting requested review from Rjected Rjected is a code owner
@klkvr klkvr Awaiting requested review from klkvr klkvr is a code owner
@gakonst gakonst Awaiting requested review from gakonst gakonst is a code owner

Assignees

No one assigned

Labels

M-changelog This change should be included in the changelog

Projects

Reth Tracker
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Support for HTTP request decompression + CLI flag to specify response compression algorithms

4 participants

@evmKnows @emmajam @pata-eth @mattsse