execution/state: cut the header walk and the per-slot probes in WriteSet.Normalize - #23027
Closed
AskAlexSharov wants to merge 8 commits into
Closed
AskAlexSharov wants to merge 8 commits into
AskAlexSharov wants to merge 8 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Optimizes the hot WriteSet.Normalize path in execution/state by avoiding AllHeaders() iteration overhead and eliminating an extra pass for collecting field-level dirty addresses, reducing per-tx normalization CPU time in the parallel-exec apply loop.
Changes:
- Reworks
WriteSet.Normalizeto range the per-path typed maps directly (dropping per-writeWriteHeadercopies and redundantGet*lookups) while collectingallAddressesduring the same walk. - Preserves existing filtering semantics (self-destruct handling, incarnation filtering, and storage no-op filtering) while changing iteration strategy.
- Extends
BenchmarkWriteSetNormalizewith a{3,1}shape to better reflect common mainnet write-set geometry.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| execution/state/writeset_normalize.go | Replaces AllHeaders() dispatch with direct per-path map loops; collects allAddresses during the walk to avoid a second pass. |
| execution/state/writeset_normalize_bench_test.go | Adds {3,1} benchmark case to represent typical mainnet write-set shape. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
AskAlexSharov
marked this pull request as draft
August 5, 2026 14:01
AskAlexSharov
force-pushed
the
alex/normalize_walk_37
branch
from
August 5, 2026 14:03
3cbbf00 to
39d9366
Compare
AskAlexSharov
force-pushed
the
alex/normalize_walk_37
branch
from
August 5, 2026 14:20
8ea8154 to
96b887c
Compare
AskAlexSharov
force-pushed
the
alex/normalize_map_walk_37
branch
from
August 5, 2026 14:31
dbaee57 to
392abc2
Compare
AskAlexSharov
force-pushed
the
alex/normalize_walk_37
branch
from
August 5, 2026 14:32
96b887c to
5073683
Compare
AskAlexSharov
marked this pull request as ready for review
August 5, 2026 15:04
…bes once per address
AskAlexSharov
force-pushed
the
alex/normalize_walk_37
branch
from
August 5, 2026 15:14
5073683 to
3d6da4c
Compare
AskAlexSharov
force-pushed
the
alex/normalize_map_walk_37
branch
from
August 5, 2026 15:14
392abc2 to
fbe4122
Compare
AskAlexSharov
marked this pull request as draft
August 5, 2026 15:21
Collaborator
Author
|
closing in favor of future Marks PR on the same field |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Normalizeruns once per tx result insideblockExecutor.nextResult— the single-threaded apply loop that is parallel execution's serialization point — so its cost is never spread across workers. On a mainnet profile it is 32% ofnextResult.Two optimizations:
1. Walk the per-path maps instead of
AllHeaders.AllHeaders()yields a 56-byteWriteHeaderper write and erases the typed value, so every arm of theswitch h.Pathlooked the write back up by address (GetStorage,GetBalance, …) — a second probe of the entry just visited. Ranging the maps hands each branch its write directly.WriteSet.Applyalready does this, with the reasoning atrw_v3.go:120.2. Resolve the storage no-op filter's self-destruct probes once per address.
ReadSelfDestructandAnyDoneSelfDestructEqualskey on the address alone but ran per slot; ranging storage per address resolves them once per slot group, lazily.The self-destruct drop rule moves into
dropForSelfDestruct(path, eip8246)— one exhaustive switch, so a new path must state its answer, instead of the predicate being repeated across loops. Measured free (+0.07%, p>=0.21).Numbers
BenchmarkWriteSetNormalizevs main, 6 interleaved rounds of-count=3, n=18:An earlier EPYC run put the same comparison at -15.52% geomean.
B/opandallocs/opunchanged — a copy-and-dispatch win, not an allocation one.A storage-only variant (move just the
StoragePatharm, leave the rest of the switch untouched) was built and measured as the smaller-diff alternative: it regresses +3.3% / +3.6% at the two narrow shapes, because storage then gets walked twice — once throughAllHeadersinto an empty arm, once in the grouped loop. Geomean -4.01%. Rejected.The two narrow benchmark shapes are added here; the pre-existing ones were both wider than mainnet, where a tx writes ~3 addresses with ~1 storage slot (
debug_traceBlockByNumber+prestateTracer, 2019-era replay and today's tip agree).Review
Each old
switcharm becomes one loop over the matching map, with its filter unchanged. Three things worth checking:dropForSelfDestruct— including the paths that deliberately do not drop (CreateContract,SelfDestruct,CodeSize,Address), which previously said so only by being absent from the switch.sdSet, whose raw storage writes are dropped, so the two can never collide.eachWriteHeaderOfyieldsvw.WriteHeader, so the old code addressed writes byh.Address/h.Key; the new loops do the same.Author comments are preserved verbatim, including the
AddressPathandCodeSizePathmarkers for the two paths that produce no output. Filter logic, fill loop, EIP-7702 recovery, EIP-161 removal and all error paths are unchanged.Green:
execution/state,execution/stagedsync,execution/tests, includingTestSelfDestructReceive,TestEIP161AccountRemoval,TestCVE2020_26265,TestDeleteRecreateAccount,TestDeleteRecreateSlots,TestDeleteRecreateSlotsAcrossManyBlocks.