execution/state: cache-free parallel execution via versionMap unification (noMaterialize) + warm-read caching - #22409
Merged
Merged
Conversation
…extcodehash bench Checkpoint of the warm-read investigation: profiling the 20x warm-extcodehash gap pinned it to Empty()'s whole-account refresh + per-read SelfDestruct probe, and the review pivoted (per review) to examining the 4 synthetic account-lifecycle paths (Address/SelfDestruct/Incarnation/CreateContract) for internal consistency rather than spot fixes. Adds BenchmarkWarmExtCodeHash (before/after gate, 471 ns/op baseline) and the lifecycle-path analysis doc (independent-fields finding, 3-way revival inconsistency, CreateContractPath redundancy candidate, no-regression approach).
…paths Extend the lifecycle-path review to the six real state fields (Balance/Nonce/Code/CodeHash/CodeSize/Storage): per-path writer sites, read function, validation class (value vs noValueRead), refresh variant and revival participation. Surfaces the second, distinct redundancy (the Code/CodeHash/ CodeSize write trio, intentional as a read-cost split), the CodeHash value-vs- noValue validation asymmetry, CodeHash's lifecycle double-role at createObject, and confirms refreshVersionedAccount is a partial Empty-oriented reconstruction. Adds the two new coverage gaps to guard before rationalizing.
…-vs-1
The two readers (getVersionedAccount, versionedStateReader.ReadAccountData)
compute revival identically (AddressPath>= OR {Bal,Nonce,CodeHash}>); only
validateReadImpl omits the AddressPath>= arm. The gap is specific to same-tx
metamorphic SD+CREATE2.
Characterization tests for the readers-vs-validator revival divergence. The two
readers (getVersionedAccount, versionedStateReader.ReadAccountData) revive on
AddressPath>= OR {Balance,Nonce,CodeHash}>; validateReadImpl omits the
AddressPath>= arm. The tests confirm this is unobservable on production
histories — a same-tx metamorphic SD+CREATE2 re-writes the read's own field, so
checkVersion invalidates a stale read before the revival arm is reached — and
only diverges in a synthetic AddressPath-only revival that createObject never
emits (it co-writes CodeHash). Locks the co-write invariant the missing
validator arm depends on before any rationalization.
…ndant CreateContractPath first looked redundant vs IncarnationPath (it appears only in validation exclusion lists), but the trace shows it is load-bearing on the apply side: rw_v3 consumes d.createContract to DomainDelPrefix storage before re-creation (mirroring Writer.CreateContract), and it prevents empty-account pruning of newly-deployed contracts. The test pins that contract creation records it while a plain account creation does not, guarding against a fold-into-IncarnationPath simplification silently dropping the storage-clear trigger. Corrects the earlier 'removal candidate' note in the review doc.
…ap 2) CodeHashPath validates as a value path (tiebreaker) while its co-written CodePath/CodeSizePath are noValueRead (version/status only). The test pins that the split is observable only for a StorageRead-sourced read colliding with a concurrent Done flush — for a MapRead the tiebreaker is bypassed, so map reads show no asymmetry. In the collision a still-matching CodeHash read survives while Code/CodeSize invalidate: benign (hash unchanged => read accurate; version-only checks conservative) and the mechanism that lets an EXTCODEHASH-only tx skip re-execution. An intentional read-cost optimization, not a bug.
Safety-net characterization before rationalizing the whole-account refresh Empty() drives: pins the EIP-161 verdict for warm empty/non-empty, absent (records the nil AddressPath read for OCC create/absent detection), cross-tx self-destruct, and the field-write-without-AddressPath edge that keys emptiness off AddressPath existence rather than the field cells. This is the invariant set the earlier naive per-field spot-fix violated.
…bsent path, not refresh The bench writes only field cells with an empty reader, so readAccount returns nil and Empty() short-circuits on the absent path before refreshVersionedAccount runs — verified that adding AddressPath alone still reads empty. Its docstring claimed to pin the over-refresh + per-read SD-probe cost; it does not. Reproducing the warm-refresh path needs a populated reader, not just versionMap cells. The 20x warm-extcodehash gap must be measured on the benchmarkoor cell, not this micro-bench; corrected the docstring and the review doc's baseline.
…t read path Profiling the warm-extcodehash benchmarkoor cell (the 20x peer-gap outlier) showed the versionMap RWMutex as the top non-GC cost: sync/atomic.(*Int32).Add at 19% flat, ~73% of it from RWMutex.RLock. A warm Empty()/refresh re-probes SelfDestruct once per field (four times for the same address), each acquiring the versionMap read lock, though the probe reads only prior-tx SD writes and is stable within one execution attempt (the tx's own SelfDestruct lives in versionedWrites, consulted separately). Memoize the probe per execution attempt, keyed by address and gated by an epoch bumped on Reset/SetTxContext so the cache is discarded across txs and re-executions with no per-tx map clear. Collapses the four refresh probes to one and removes the repeated lock acquisitions across a tx's reads of the same address. Behavior-identical: the memo returns the same value the locked read would, so recorded reads and SD gating are unchanged (state suite + -race + SD/recreate/revival/validation tests green).
…micity question Defer the lock-acquisition reduction (batched read / lock sharding) to a follow-on. Capture the load-bearing open question: whether FlushVersionedWrites' single-lock atomic flush is actually required given OCC validation reruns on the tx read-set in the verification thread and would catch a partial-flush read. Pin that OCC-catches-partial-reads invariant as a test before weakening the flush lock.
…te from field reads) refreshVersionedAccount reconstructs a whole *accounts.Account by overlaying the per-field versionMap cells onto the base record — work the field-oriented getters (GetBalance/GetNonce/GetCodeHash) already do directly, never touching it. Empty and Exist were the only field-oriented callers still going through the full reconstruction: - Exist used only readAccount != nil, discarding the entire 4-field overlay. - Empty needed just the EIP-161 verdict (balance/nonce/codeHash all zero). Split versionedAccountBase (existence + self-destruct/revival gate, records the OCC nil AddressPath read) out of getVersionedAccount (which still appends the refresh for the stateObject builders). Exist now stops at the base; Empty reads balance/nonce/codeHash per-field, short-circuiting on the first non-empty field, so it does fewer reads (drops the unused incarnation), skips the account allocation (account.Copy), and avoids the reconstruction. Behavior-identical: the per-field refresh reads apply the same self-destruct gate, and the nil-read OCC invariant is preserved (state suite + -race + Empty characterization + revival/SD/recreate tests green).
The primary Phase-2 question. Reads are already off the stateObject (getters have a versionMap parallel branch; Empty/Exist moved in Phase 1). Writes still materialize it via GetOrNewStateObject but the value is sourced from the versionMap (prev = getBalance) and carried by recordWrite* into the write-set — so.data has no parallel reader. The only load-bearing parallel use left is intra-tx revert (journal/RevertToSnapshot on so.data). Move revert onto the versioned write-set/journal and the stateObject materialization + refreshVersionedAccount sync become dead on the parallel path.
…t a materialized stateObject TouchAccount only needs the EIP-161 emptiness verdict, but went through GetOrNewStateObject -> getStateObject -> refreshVersionedAccount to read so.data.Empty(). On warm CALLs (the account exists) that whole-account reconstruction is pure overhead — profiling warm-call showed the write-path getStateObject->refreshVersionedAccount at 21% of exec, reached via Transfer/TouchAccount. For an existing account compute emptiness via the shared emptyFromVersionedFields helper (per-field balance/nonce/codeHash reads, short-circuiting) with no stateObject materialization. An absent account still falls through to GetOrNewStateObject so createObject records the AddressPath write that OCC create-detection relies on. Behavior-identical: state suite + -race + EIP-161/recreate/selfdestruct integration tests green. First step of the parallel-stateObject removal (Phase 2).
…ecise-measurement plan Record the balance-first vertical-slice approach for removing the parallel stateObject (journal-prev from the versionMap read, revert stays lazy, absent accounts keep createObject/AddressPath), the GetDelegatedDesignation fallback+no-record code-read need, and the plan to add pprof labels + KVReadLevelledMetrics to the hot versionedio paths so post-cleanup time attribution is measured, not inferred.
…ival definition)
The self-destruct/revival verdict is currently recomputed ad-hoc at three sites
(getVersionedAccount, versionedStateReader.ReadAccountData, validateReadImpl)
with two different definitions (the readers include an AddressPath >= arm the
validator omits — coverage gap 1). AccountLifecycle derives destroyed/
destroyedAt/revived once from the synthetic lifecycle paths with a single
definition (AddressPath >= destroyedAt catches same-tx metamorphic SD+CREATE2;
{Balance,Nonce,CodeHash} > destroyedAt otherwise), so the readers, validation,
and the write-path create decision can converge onto it and cannot diverge.
Pure addition, not yet wired into consumers. Unit-tested against the not-
destroyed / destroyed-no-revival / metamorphic-AddressPath / field-revival /
strict-> cases. Foundation for the parallel-stateObject write-path removal
(the create decision reads this verdict instead of materializing the object).
versionedAccountBase (getVersionedAccount) and versionedStateReader.ReadAccountData
each open-coded the identical SD/revival check (AddressPath >= destroyedAt OR
{Balance,Nonce,CodeHash} > destroyedAt). Replace both with the AccountLifecycle
resolver — behavior-identical (same definition), removing the duplication so the
readers can't drift. Full state suite + revival/SD/metamorphic/recreate/EIP-161
tests green.
… the floor The complete self-destruct verdict needs the tx's own field-level writes layered over the versionMap floor — the account-level analogue of what versionedReadCore does per field. accountLifecycle consults the write collection (versionedWriteSelfDestruct) and the versionMap only, never the stateObject (whose deleted flag is a redundant cache of the own SelfDestruct write). An own-tx SelfDestruct write is authoritative: true after a same-tx SD, false after a same-tx recreate; with no own write the floor's destroyed-and-not-revived verdict applies. Unit-tested for both layers. This is the verdict the write-path create decision reads instead of materializing the stateObject — removing the reason GetOrNewStateObject's decision 'had to stay'.
…e-set The parallel finalize task built its committed writes via MakeWriteSet(collector) from stateObject.data; migrate it to normalizeWriteSet over the finalize write-set (ivw), the same path regular txs use. ivw is already reconciled by VersionedWrites(checkDirty=true), and block finalize never creates+SELFDESTRUCTs a contract, so normalizeWriteSet reproduces the finalize commit without so.data. First step of sourcing the parallel commit solely from versionedWrites.
Categorize every FinalizeTx/MakeWriteSet/CommitBlock call site as versionMap!=nil (must be write-set-sourced) vs versionMap==nil (stays). Records that EXEC3_PARALLEL defaults true (so generation + import both run with a versionMap), that the executor system-tx/worker so.data outputs are non-authoritative for domains, and that block production (chain_makers + builder) is the confirmed remaining blocker. Design: relocate normalizeWriteSet into package state + a WriteSet->StateWriter adapter so all versionMap consumers share one write-set commit authority.
Move the write-set normalization (versionMap WriteSet -> clean committed WriteSet) out of package stagedsync into a method on WriteSet in package state, so block production (builder, chain_makers) and the executor can share one write-set commit authority. Pure mechanical relocation: state. qualifiers dropped, signature becomes a method receiver; callers updated to rawWrites.Normalize(...). Behavior-neutral.
Convert StateV3.applyVersionedWrites into (writes *WriteSet).Apply(domains, ...), taking the SharedDomains and trace flag as params instead of reading StateV3. Symmetric with WriteSet.Normalize and lets block production apply the write-set to domains without a StateV3 (which is becoming a thin domains-holder). StateV3. ApplyStateWrites now delegates to writes.Apply(rs.domains, ..., rs.trace.Load()). Behavior-neutral.
… write-set Block generation committed the whole block's so.data once via CommitBlock; when the block is versioned (EXEC3_PARALLEL default true, or Amsterdam) commit instead from the per-phase write-sets via WriteSet.Normalize+Apply — the same path the parallel executor uses — so generation matches import once the write path bypasses so.data. blockIO is now created for any versioned block (not only Amsterdam) so its Outputs() carry the phase write-sets. Legacy CommitBlock stays for versionMap==nil.
…ap write-set The production payload builder accumulated so.data across the block and committed it once via FinalizeBlockExecution -> CommitBlock. When the block is versioned (BAL mode) skip that CommitBlock and instead apply the per-phase write-sets captured in ba.BalIO() via WriteSet.Normalize+Apply to the SharedDomains, matching the parallel executor and block generation. FinalizeBlockExecution skips CommitBlock only for a versioned ibs; ExecuteBlockEphemerally (versionMap==nil) keeps the so.data path. Requires the hive/eest Amsterdam block-building gate for final validation.
…ite path AddBalance/SubBalance/SetBalance route through writeBalanceVersioned when a versionMap is present: an existing-alive account records the balanceChange + recordWriteBalance directly, without GetOrNewStateObject materializing/refreshing a stateObject; absent or destroyed-no-revival accounts still materialize so createObject records the AddressPath write OCC needs. The write is carried entirely by versionedWrites, but MakeWriteSet's revert reconciliation dropped it: SoftFinalise seeded the dirty set only for addresses with a materialized stateObject, so a fast-path write (no stateObject) was treated as reverted and deleted from versionedWrites (wrong SELFDESTRUCT-beneficiary/empty account balance). In the versionMap path dirtiness now comes from the journal (populated alongside every recordWrite), not stateObject existence.
Formatting fixups for the WriteSet.Apply/Normalize relocation commits: consolidate the orphaned applyVersionedWrites doc into the Apply method doc, drop a trailing blank line after MergeVersionedWrites.
…ise/MakeWriteSet The parallel (versionMap) worker no longer routes end-of-tx finalize through SoftFinalise + MakeWriteSet, which tracked dirtiness via stateObject materialization. FinalizeTxVersioned applies the EIP-6780 storage wipe, captures the reconciled write-set from the journal, and clears it. The serial path keeps the legacy methods.
…erts The record-level createObjectChange/resetObjectChange journal entries only deleted the stateObject on revert, orphaning the field-level versionedWrites that createObject/createAccount record (address/codeHash/balance/incarnation/ selfDestruct/createContract). The parallel finalize then swept those orphans via a dirty-set reconciliation keyed on journal.dirties. Fix at source: createObjectChange.revert drops the account's versioned writes (it did not exist before the create); resetObjectChange.revert restores them from a pre-create snapshot. versionedWrites now stays in step with the journal, so FinalizeTxVersioned no longer reconciles — it snapshots the write-set directly.
…ation With the create/reset journal reverts now maintaining versionedWrites, the write-set already contains exactly the surviving writes — the checkDirty reconciliation (drop non-dirty addresses + delete from the version map) is a no-op. Drop the checkDirty parameter and deleteAddrFromVersionMap; VersionedWrites now always returns the direct snapshot.
…xVersioned Route the block-init and block-end tasks through FinalizeTxVersioned on the versionMap path instead of FinalizeTx(NoopWriter)/MakeWriteSet. VersionedWrites no longer depends on the journal, so the init writes can be snapshotted directly after the switch. Serial path keeps FinalizeTx/MakeWriteSet.
The parallel finalize (EIP-6780 same-tx create+selfdestruct storage wipe + write-set snapshot) is now a functional WriteSet.Finalize — it derives the create+destruct pair from the write-set (createContract survives SELFDESTRUCT) instead of reading stateObjects, so it needs no IntraBlockState. IBS keeps only the thin VersionedWrites accessor and the journal reset. finalizeSystemTx drops its FinalizeTx(stateWriter) for FinalizeTxVersioned. This moves execution toward functional IO (call it, get the write-set back) and lets IntraBlockState shrink to an EVM context object rather than a general-purpose exported one.
Contributor
Author
|
Addressed all three review items (pushed in 848e925 + 6277097, on top of the main-merge 852b8ca):
Both new tests are red-first (verified failing without the fix). |
On the noMaterialize (parallel) path, committedCodeDirect and committedCodeSizeDirect read code straight from the address-keyed CodeDomain without consulting the account's codeHash. Code bytes are never deleted when a 7702 delegation is cleared — only the codeHash pointer is reset to empty — so a since-cleared delegation leaves stale ef0100.. bytes in the CodeDomain. The ungated read returns them, GetDelegatedDesignation sees a delegation on an empty-codehash EOA, and a plain value transfer to that account is charged as a call to delegated code: it runs out of gas after the intrinsic cost, flips the receipt status to failed, and diverges the receiptHash and trie root from serial (which gates via stateObject.CodeTyped: empty codehash -> empty code). Gate both reads on the committed codeHash, mirroring the materialized path. Reproduced on mainnet block 25580072 (three transfers to a since-delegated EOA): parallel exec failed the block, serial passed; fixed, both now agree.
AskAlexSharov
approved these changes
Jul 22, 2026
AskAlexSharov
enabled auto-merge
July 22, 2026 02:59
AskAlexSharov
added a commit
that referenced
this pull request
Jul 23, 2026
Reconcile the compact journal-entry representation (this branch) with main's versionMap-unification / noMaterialize revert semantics (#22409): - journal.go: port main's materialize-aware reverts and versioned write-set tracking (createObject deleteAddr; resetObject restoreCreateFields + noMaterialize; selfdestruct incarnation/balance restore; touch balance undo; storage versionMap-first ordering) onto the compact tagged-union entries; drop the trace scaffolding main removed. resetObject/selfdestruct/touch constructors gain the new fields (prevWrites, incarnation/balance capture). - intra_block_state.go / state_object.go: route the new versioned call sites (writeBalanceVersioned, selfdestructVersioned, versioned TouchAccount, reset snapshot) through the compact journal constructors.
mh0lt
added a commit
that referenced
this pull request
Jul 28, 2026
Brings in main's commitment features on top of this branch's framework: - checkpointStepsFromBAL (#22756) and the compute-ahead naming; kept the branch's framework (ownsChangeset fold-ahead gate removed, owned blocks compute ahead with finalizeFoldedChangeset filling their diffs). - self-destruct storage-enumeration removal (#22758): calc_state.go's storageEnum is now a test-only injection (nil in production); the branch's changeset_reconstruct.go already guards on storageEnum==nil. - noMaterialize (#22409) and the rest of main. Conflict resolutions: - rpc/jsonrpc tests, simulated.go, exec_module_test.go: kept the branch's OverlayDB()/WaitCommitsDrained reads, adopted main's new signatures (NewPrivateDebugAPI DebugApiConfig, newDebugApiForTest helper, pendingState). - committer.go: main's checkpointStepsFromBAL + compute naming, branch's fold-ahead framework. - Removed a merge-artifact WaitIdle(ctx) after New in TestReorgBackAndForwardIntoCanonicalChain (WaitIdle permanently stops the branch's commit worker; calling it at test start dead-locked bg-commit). - common.Copy -> bytes.Clone at branch-only call sites (main's //go:fix inline).
pull Bot
pushed a commit
to Dustin4444/erigon
that referenced
this pull request
Jul 29, 2026
… path (erigontech#22854) Re-executing mainnet from genesis computes a wrong trie root at block 2,675,119, in the Spurious Dragon empty-account clearing range: ``` computed 8dac6476e3444420de57f554980cddf9af57da6fa2fee3da7ce7a774133af2b0 expected 727fc87b6a13c3a4d1097b7cf8a1f183b0d4e944ef2abca1583250bbafaa76f8 ``` Tracing the block's commitment updates against the last-good commit shows the sets differ by exactly one operation out of 728 — the delete of RIPEMD-160 (`0x…03`). `touchAccount` bumps ripemd's journal dirty count so its touch outlives the transaction that rolled back, and the serial path therefore still sweeps the account under EIP-161. The versioned path builds its write set from `versionedWrites`, and the `kindTouch` revert dropped the touch's zero-balance write — so the sweep never happened and the account kept its trie leaf. Bisected to erigontech#22409, which moved parallel execution off resident `stateObject`s onto `versionMap + journal`. Only re-execution reaches this range; snapshot-synced nodes never execute these blocks, which is why it went unnoticed. ## Changes - `journal.go`: exempt ripemd from the `kindTouch` revert on the versioned path, mirroring the dirty-count bump serial already relies on. - Regression test covering ripemd (swept despite the revert) and an ordinary empty account (not swept), so the exemption stays specific.
pull Bot
pushed a commit
to Dustin4444/erigon
that referenced
this pull request
Aug 3, 2026
…#22956) `getStateObject` and `reconstructCellFlags` recomputed `keccak(code)` on every rebuild, to keep the rebuilt stateObject's `CodeHash` in step with the code cells. The hash was already available at every source that produced the bytes. ## Why it is hot Erigon's CodeDomain is keyed by **address**, not by code hash (geth/reth/gevm all key their code store by hash, so they never re-derive it on a load). Two independent cells — the account record's `CodeHash` and the code bytes — can therefore disagree, e.g. when a prior tx in the block sets an EIP-7702 delegation whose `AddressPath` record was published with the old hash. The recompute was a blunt way to make `obj.code` and `obj.data.CodeHash` agree. That was affordable while `getStateObject` cached its result. Under `noMaterialize` (erigontech#22409) it no longer does — the object is rebuilt on every call — so a per-load hash became a per-read hash of the entire contract, reached from balance, nonce and code-hash reads alike. In a 300s mainnet parallel-exec profile: | | cum | share | |---|---|---| | `getStateObject` -> `Keccak256Hash` | 15.01s | 83.6% of all keccak in the process | | `opKeccak256` (the `KECCAK256` opcode) | 2.73s | — | The internal resync cost **5.5x the opcode it exists to implement**. ## Fix `refreshCode` now returns the writer's hash alongside the bytes, and the caller resolves it per source: - **write-set / version-map hit** — the cell stores `accounts.Code`, which already carries the hash. - **read-set hit recorded from committed storage** — the account record's `CodeHash` is authoritative; the bytes cannot be newer than it. This produces exactly the pairing `stateObject.Code()` builds on its lazy path (`accounts.Code{Hash: so.data.CodeHash, Bytes: code}`). - **read-set hit recorded from the version map** — the probe that produced the hit already returned the cell, so its hash is carried through rather than re-derived. Version equality with the recorded read pins the same cell, so the hash pairs with the recorded bytes; the tiers that return before the probe leave it Nil and fall through to the rules above. One case still hashes on every rebuild: a **dirty address reading committed code**. `journal.dirties` bypasses the read-once gate, so the rebuild re-probes, misses the version map, and resolves through the matching-header branch with source `ReadSetRead` — which does not carry the account record's authority the way `StorageRead` does. Unimproved rather than regressed; resolving it means widening the account-record rule to a read-set source, which is a behaviour change of its own and belongs in a separate PR. `refreshCode` cannot just return `accounts.Code`: that type documents `INVARIANT: Hash == Keccak256(Bytes)`, and the read-set source stores bytes only. The new `refreshedCode` carries no such promise, so a caller cannot silently take a bogus hash for a real one. ## Numbers `BenchmarkGetStateObjectAfterCodeRead` — the state-object rebuild every account-field read falls through to. n0 (EPYC 4344P), interleaved binaries, benchstat n=8, all p=0.000: | code size | ns/op before | ns/op after | delta | B/op before | B/op after | allocs/op before | allocs/op after | |---|---|---|---|---|---|---|---| | 32 B | 530.3 ± 2% | 228.1 ± 2% | **-56.98%** | 528 | 528 | 5 | 5 | | 1024 B | 2,236.0 ± 1% | 207.5 ± 4% | **-90.72%** | 528 | 528 | 5 | 5 | | 24576 B | 36,977.5 ± 0% | 204.3 ± 3% | **-99.45%** | 528 | 528 | 5 | 5 | | geomean | 3.526us | 213.1ns | **-93.96%** | 528 | 528 | 5 | 5 | Before scales with bytecode length; after is flat. Allocations identical — this removes compute, not garbage. ## Behaviour One deliberate change: when the CodeDomain entry disagrees with the account record, the account record now wins. The CodeDomain is keyed by address, so it can hold bytes the account no longer owns — a cleared 7702 delegation leaves them behind — and the old code let those bytes overwrite `obj.data.CodeHash` / `obj.original.CodeHash`. `stateObject.Code()`'s lazy path already preferred the account record. `TestCommittedCodeHashComesFromAccountRecord`, `TestPriorTxCodeWriteHashComesFromTheCell` and `TestPriorTxCodeWriteHashSurvivesReadSetHit` all fail on main. Each gives its cell a hash that disagrees with `keccak(bytes)`, which is what makes the source of the hash observable — a real cell never lies, so on real input this changes cost, not values.
This was referenced Aug 4, 2026
This was referenced Aug 5, 2026
[r3.6] execution/stagedsync: cut allocations and redundant account reads in normalizeWriteSet
#23026
Merged
Sahil-4555
pushed a commit
to Sahil-4555/erigon
that referenced
this pull request
Aug 6, 2026
…igontech#22190) ## execution/state, execution/vm: zero re-executions and aborts for BAL-fed parallel execution; remove `VersionMap.HasBAL` ## Why EIP-7928 block access lists pre-populate the parallel executor's versionMap with per-field value cells, so every read on a BAL-fed block can resolve deterministically — in principle making optimistic execution conflict-free. In practice, residual races and validation blind spots still caused re-executions and dependency aborts: reads recorded from signals the BAL structurally cannot carry (the SELFDESTRUCT flag, account existence, incarnation metadata), version churn on unchanged values, and probes recorded as observations when they were only guesses. Each re-execution is wasted work and — worse — non-determinism. This PR drives BAL-fed parallel execution to **literal zero re-executions and zero aborts**, verified from genesis to tip on glamsterdam-devnet-6 (and previously bal-devnet-7), and removes `VersionMap.HasBAL` entirely: BAL and no-BAL blocks run through one uniform, value/cell-evidence-based validation — pre-populated cells simply make reads deterministic when present, with no flag-gated semantics anywhere. ## What Net delta vs main: ~1,618 insertions / 179 deletions over 20 files (execution/state, execution/stagedsync, execution/vm, common/dbg), 29 new tests. The eth/71 BAL fetcher is NOT in this PR — it was extracted and landed separately (erigontech#22715). ### Validation semantics (versionmap.go) - `ValidateVersion` gains an `eip161 bool`: nil-vs-EIP-161-empty record dead-equivalence applies only post-Spurious-Dragon (pre-161, existing-empty is gas-observable, so the strict existence-only form is kept). Deadness is assembled from sub-field floors (`accountLiveAt`): a creation-time-empty record next to a funded Balance/Nonce/Code cell is live (the "created then funded" phantom that once produced a wrong BAL at bal-devnet-7 block 115,591). - AddressPath record reads validate existence-only; every sub-field is recorded and value-validated as its own read (records churn per fee-paying tx; their values are what matter). - Value tiebreakers unified across the code trio: Code (byte compare), CodeHash, and CodeSize (upgraded here) all survive a value-matching cold-read/flush collision and still invalidate on a real change. - Sub-field reads with no dedicated cell fold onto the AddressPath record for validation; the fold now carries a `recordField`/`matchesRecord` value tiebreaker — fee-merge record re-stamps with an unchanged field no longer invalidate readers (coinbase nonce churn). - `destroyedAndUnrevived` relaxes the created-account incarnation check: a nil record read of a destroyed, unrevived account is correct on every fork. ### Read-path determinism (read_paths.go, intra_block_state.go) - Value-aware read-time relaxation (`readValueUnchanged`) at the RD/WR dependency checks: version-only churn with an unchanged value (existence-only for records, with the same sub-field-aware dead case as validation) is not a dependency — no abort. - `ProvisionalRead`: a mid-account-load nil record probe is marked provisional; a cell flushed between probe and re-probe is adopted (the nil was never exposed to the EVM) instead of false-positive aborting. Provisional markers are demoted to definitive reads at every absent-conclusion exit, so they never outlive their load. - `synthesizeCreatedAccountBase`: an account absent from both the AddressPath map and the DB resolves its existence from BAL-prepopulated sub-field cells (Done cells only; bails on SD floors, estimates, and EIP-161-empty results) — removing the read-after-create race with the creator's flush. Synthesis refuses to fire once the tx has consumed a definitive absence (a recorded non-provisional nil read): adopting cells then would fork the tx's view mid-execution invisibly to validation; commit-time validation re-executes instead. - Recorded-read reconciliation: a DB-resolved account load reconciles the nil map-read marker it recorded (otherwise a later record cell spuriously invalidates the nil against a live account), including for recordRead=false flows. - Read-set-served probes are never laundered into recorded reads: the cold account-field path resolves a `ReadSetRead` source back to the underlying entry's source/version, and `accountRead` skips re-recording a read-set-served read. (Without this, validation deterministically rejects the synthetic source and the tx livelocks — "too many validator-invalid retries".) - DB-loaded records are stamped `UnknownVersion` (pre-block state) so lower- versioned BAL cells overlay them instead of being shadowed. ### SELFDESTRUCT signal handling (vm/operations_acl.go, intra_block_state.go) The SD flag is the one signal a BAL cannot carry, so recorded SD reads race the destroyer's flush. Everywhere its consequences are already pinned by value-validated reads, the probe is derived without recording: - The SELFDESTRUCT refund gas probe runs only when `refundsEnabled` (pre-London): post-EIP-3529 the result is discarded, so the recorded read was pure race surface. - `CreateAccount` (CREATE2 re-creation flow) derives `destructed` from its own write set → cached state object → direct map probe, unrecorded; the value-carrying synthetic incarnation/balance reads it stamps make any stale conclusion invalidate by value. - Account-load SD gates decide deletion via cells-evidenced revival (`AccountLifecycle`: any cell after the destruct index proves life; same-tx re-creation uses >= on AddressPath). ### Incarnation is metadata, not an observation (read_paths.go) The no-cell incarnation refresh no longer records a read: with no cell, the "current" incarnation is pre-block state or a synthesis guess, it is EVM-invisible, staleness is caught by the real field reads, and write normalization resolves the final incarnation from the versionMap (`SetAccountFieldFromMap`). The decisive repro was the fund-then-deploy CREATE2 pattern (target funded a block earlier, deployed onto, read by the next tx). ### BAL pre-population (versionmap.go) `WriteChanges` derives `CodeSize` and `CodeHash` cells from each BAL code change: EXTCODESIZE/EXTCODEHASH readers resolve from the map instead of falling through to the DB and racing the creator's flush. ### Instrumentation (env-gated, off by default) `ERIGON_TRACE_REEXEC` (classified validation-invalid/abort dumps: VINV reason=, DEP-*, ABORT-*, TOOEARLY) and `ERIGON_TRACE_BAL_FEED` (per-block feed/miss + per-cell pre-population) — the measurement tooling behind every number below. ## Verification Every fix is TDD'd (red→green); 29 new tests pin the classes. Full batteries on the final tree: **EEST 355,601 / 0 failed** across five shards (blocktests-devnet 86,931; blocktests-stable-parallel and -sequential 70,864 each; statetests-devnet 66,976; statetests-stable 59,966), `GOGC=80 make test-all` exit 0, `make lint` 0 issues, plus main's erigontech#22409 lifecycle/SD/revival test battery. On-devnet acceptance (fresh datadir, from genesis to tip, external CL): | run | code state | span | post-Amsterdam re-execs / aborts | |---|---|---|---| | bal-devnet-7 baseline | pre-fix | 0 → 43.8k | 1,188 / 31 | | bal-devnet-7 final | fix set | 0 → 407,538 (tip) | **0 / 0** (0 bare blocks) | | gd6 merge baseline | post-main-merge, pre-fix | 0 → 15k | 78 / 0 | | gd6 acceptance | + 5 gd6 fixes | 0 → 149,990 (tip) | **0 / 0** | | gd6 re-acceptance | post-erigontech#22409 merge + seam fixes | 0 → 171,182 (tip) | **0 / 0** | Each acceptance also had 0 blocks executed without a BAL, 0 BAL-hash mismatches/bad blocks, and a canonical hash match at tip. For contrast, the pre-Amsterdam blocks of the same syncs (no BALs by protocol) ran ordinary Block-STM at a 46.8% re-execution rate on gd6's conflict-heavy fuzz traffic — that is the cost the BAL eliminates. The pre-Spurious-Dragon strictness and all no-BAL behavior changes are covered by the stable sequential+parallel EEST shards; a mainnet from-0 parallel re-execution remains the deferred final verification before enabling anything by default. ## Not in this PR - The eth/71 BAL fetcher hardening (extracted; landed as erigontech#22715 — this branch now consumes it from main). - Any spec change: EIP-7928 does not carry account existence, the SD flag, or incarnation; the residual classes those caused are eliminated by derivation/relaxation, not by extending the BAL.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes the parallel execution path cache-free by unifying all reads/writes on
the versionMap (the
noMaterializepath): the per-txstateObjectis removedfrom parallel exec and
IntraBlockStateresolves state asversionMap + journal,with the Block Access List and OCC read-set derived from the same versioned reads.
Removing the resident
stateObjectcost ~2.4x on warm reads, so this branch alsoadds a read-side caching layer that recovers it without reintroducing the
stale-read bug:
sync.Map+ per-AddressEntryRWMutexinstead of one global
RWMutex(removes reader-counter contention).clean address) instead of re-probing the versionMap; conflicts still caught at
commit by
ValidateVersion.BlockStateCache.committedAccounts→sync.Map(write-once-per-key, lock-free hits) + a per-tx memo of the committed fallback
in
versionedAccountBase(block-immutable, shared with read-only callers).Performance (benchmarkoor, 100M gas, parallel exec, 6 cores)
Baseline = #22154 (
currentcolumn from that PR).serial/parallel= thisbranch with serial- and parallel-commitment respectively. Both arms are 34/34
VALID. Peers are one consistent pandaops snapshot; rank is erigon's position
among the 6 clients.
serial → parallel commitment is a non-diff (median p/s = 0.987), so the two arms give the identical rank picture.
Headline: the
warm-*/ repeated-read family improves 1.5–6.7× over #22154,with 7 cells climbing a rank and no regressions. warm-extcodehash now beats
geth (2222 vs 1816).
vs #22154 (baseline = #22154
current; serial + parallel commitment arms; improvement / rank / gap computed from the parallel arm)Improving cells (summary: parallel vs #22154 + ranking movement)
Cells that climbed a client rank or improved ≥1.30× over baseline. Seven cells climb a rank; the warm-* read family is where the read-side caching pays off.
Scope / follow-ups (not in this PR)
noMaterialize),dropping the remaining
stateObjects/nilAccounts/balanceIncplumbing.-contract/*-bloated/*-missingcells (bottom of the tables, ~1.0×) are
NO_CACHErandom cold reads over abloated multi-GB state; they are cache-defeating by design and are the target of
a separate persistent-cache PR, not the in-exec caching here.
Testing
execution/state+execution/vmunit tests and-racegreen;make lintclean.eest_stable(max-failures=0). Theglamsterdam-devnetshard is known-WIP upstream (CI
max-failures: 3473).Follow-up work (single versionedio model, remove the stateObject) is tracked in #22458.