feat(stages): handle partial trie unwinds - #26543
Merged
mediocregopher merged 11 commits intoAug 4, 2026
Merged
Conversation
mediocregopher
marked this pull request as ready for review
August 3, 2026 09:08
mediocregopher
requested review from
Rjected,
klkvr,
mattsse and
shekhirin
as code owners
August 3, 2026 09:08
Member
Author
|
cyclops audit |
Contributor
|
Cyclops audit event published. View workflow run Config: config: |
Member
👁️ Cyclops Security Review
🧭 Auditing · mode=
⚙️ Controls
|
tempoxyz-bot
reviewed
Aug 3, 2026
tempoxyz-bot
left a comment
Member
There was a problem hiding this comment.
👁️ Cyclops Review
Found 3 actionable findings, all in the new partial-trie startup recovery orchestration. The trie-walker flag propagation appears targeted, but the startup unwind scheduler needs stronger crash-consistency, target-readability, and pruning-retention checks before this recovery path is safe.
Reviewer Callouts
- ⚡ Crash consistency of startup unwinds: The new recovery path relies on normal per-stage pipeline commits; review every crash point between
FinishStageandMerkleUnwindfor resumability. - ⚡ Partial persistence with pruning: Review how
num_state_masking_blocks, persistence thresholds, and account/storage prune modes interact so crash-recovery history cannot be pruned away. - ⚡ Combined recovery ordering: Re-check mixed partial-trie and storage/static-file inconsistency cases, especially Headers, so the first queued target is always readable and lower repair targets are not skipped.
- ⚡ Trie walker resumed stacks:
with_walk_all_changed_branch_childrendoes not recomputecan_skip_current_nodeafter constructing a walker from an existing stack; current partial recovery uses fresh walkers, but future resumed use of this mode should be audited.
joshieDo
approved these changes
Aug 4, 2026
This was referenced Aug 5, 2026
teyrebaz33
pushed a commit
to teyrebaz33/tempo
that referenced
this pull request
Aug 25, 2026
Automated nightly update of reth dependencies from `paradigmxyz/reth` main branch. ## Upstream reth changes [`10aa6a5...00ff650`](paradigmxyz/reth@10aa6a5...00ff650) 🔗 Amp thread: https://ampcode.com/threads/T-01a036f1-9f58-7640-b789-4e3dd9778ebe - **Engine** - Improved payload building across canonical ancestors, finality, persistence handoffs, pending resolution, and responsive cancellation ([#26559](paradigmxyz/reth#26559), [#26567](paradigmxyz/reth#26567), [#26580](paradigmxyz/reth#26580), [#26708](paradigmxyz/reth#26708), [#26759](paradigmxyz/reth#26759)). - Added Bogota Engine API support and fork-time validation, plus FOCIL inclusion-list construction and stubs ([#26682](paradigmxyz/reth#26682), [#26706](paradigmxyz/reth#26706), [#26711](paradigmxyz/reth#26711), [#26737](paradigmxyz/reth#26737)). - Tightened payload and block-access-list validation, including state-gas admission and malformed BAL rejection ([#26651](paradigmxyz/reth#26651), [#26694](paradigmxyz/reth#26694), [#26719](paradigmxyz/reth#26719)). - Fixed little-endian cell bitvectors, Osaka `getBlobsV4`, and prewarm-worker shutdown ([#26650](paradigmxyz/reth#26650), [#26703](paradigmxyz/reth#26703), [#26768](paradigmxyz/reth#26768)). - Reused scratch buffers for faster BAL hash encoding ([#26701](paradigmxyz/reth#26701)). - **RPC** - Added `debug_traceChain`, Alloy trace-chain result types, block-level EVM reuse, and raw block transactions on the auth server ([#26582](paradigmxyz/reth#26582), [#26614](paradigmxyz/reth#26614), [#26669](paradigmxyz/reth#26669), [#26760](paradigmxyz/reth#26760)). - Added configurable response compression and request decompression ([#26668](paradigmxyz/reth#26668), [#20277](paradigmxyz/reth#20277)). - Added Bogota Engine API stubs and Amsterdam system contracts to `eth_config` ([#26691](paradigmxyz/reth#26691), [#26705](paradigmxyz/reth#26705)). - Fixed cancellation of payload-hash and blocking-I/O work ([#26569](paradigmxyz/reth#26569), [#26776](paradigmxyz/reth#26776)). - Corrected execution-witness block identifiers, optional receipt conversion/caching, and access-list environment preparation ([#26572](paradigmxyz/reth#26572), [#26596](paradigmxyz/reth#26596), [#26599](paradigmxyz/reth#26599), [#26598](paradigmxyz/reth#26598)). - Fixed testing block gas limits, transaction gas-limit preservation, timestamp overflow, and chain-ID validation ([#26632](paradigmxyz/reth#26632), [#26743](paradigmxyz/reth#26743), [#26767](paradigmxyz/reth#26767), [#26782](paradigmxyz/reth#26782)). - Added network-specific log responses and generic testing RPC handlers ([#26491](paradigmxyz/reth#26491), [#26547](paradigmxyz/reth#26547)). - **Networking** - Added ingress limits, configurable no-op client versions, and outbound `GetCells` support ([#26660](paradigmxyz/reth#26660), [#26652](paradigmxyz/reth#26652), [#26673](paradigmxyz/reth#26673)). - Improved handshake and protocol safety through ECIES identity checks, message-ID validation, negotiated-protocol assertions, and bad-message handling ([#26639](paradigmxyz/reth#26639), [#26654](paradigmxyz/reth#26654), [#26659](paradigmxyz/reth#26659), [#26671](paradigmxyz/reth#26671)). - Fixed ping/pong validation and pacing ([#26698](paradigmxyz/reth#26698), [#26702](paradigmxyz/reth#26702)). - Made eth/72 blob-cell announcements interoperable with geth while preserving availability masks ([#26573](paradigmxyz/reth#26573), [#26670](paradigmxyz/reth#26670)). - Shared the snap/2 slim-account codec between client and server ([#26587](paradigmxyz/reth#26587)). - **Txpool** - Added blob-cell availability tracking and exposure on pooled transactions ([#25463](paradigmxyz/reth#25463), [#26642](paradigmxyz/reth#26642)). - Included blob-pool transactions in queued counts and listings ([#26677](paradigmxyz/reth#26677), [#26679](paradigmxyz/reth#26679)). - Allowed senders with empty code hashes, added consensus encoding, and converted sender accessors to iterators ([#26644](paradigmxyz/reth#26644), [#26739](paradigmxyz/reth#26739), [#26681](paradigmxyz/reth#26681)). - **Trie & State** - Added partial trie unwind and persistence support, including changeset-cache handling ([#26543](paradigmxyz/reth#26543), [#26612](paradigmxyz/reth#26612)). - Corrected witness construction to use depth-first node order ([#26707](paradigmxyz/reth#26707)). - Simplified `HashedPostState` wipe handling and added trie-data reference collectors ([#26524](paradigmxyz/reth#26524), [#26752](paradigmxyz/reth#26752)). - **Storage & Providers** - Removed `ConsistentDbView`, relocated `OverlayStateProvider`, and simplified provider bounds ([#26581](paradigmxyz/reth#26581), [#26611](paradigmxyz/reth#26611), [#26591](paradigmxyz/reth#26591)). - Fixed storage-wipe handling during batched persistence ([#26750](paradigmxyz/reth#26750)). - Made RocksDB tolerate unknown column families ([#26647](paradigmxyz/reth#26647)). - Updated static-file consistency checks to respect prune checkpoints ([#26565](paradigmxyz/reth#26565)). - **Chainspec & Consensus** - Added Bogota hardfork support and validated block-access-list hashes during import ([#26686](paradigmxyz/reth#26686), [#26696](paradigmxyz/reth#26696)). - Honored the genesis `slotNumber` instead of hardcoding zero ([#26680](paradigmxyz/reth#26680)). - Defaulted unspecified payload attributes ([#26684](paradigmxyz/reth#26684)). - **DNS** - Fixed EIP-1459 discovery records by rejoining long TXT character strings and ignoring unrelated TXT records ([#26602](paradigmxyz/reth#26602), [#26603](paradigmxyz/reth#26603)). - **Snapshots & CLI** - Added base-URL resolution and exposed prepared snapshot context ([#26576](paradigmxyz/reth#26576), [#26777](paradigmxyz/reth#26777)). - Fixed history downloads when the final snapshot chunk is partial ([#26607](paradigmxyz/reth#26607)). - Added bootnode configuration to `reth.toml` ([#26551](paradigmxyz/reth#26551)). - **Testing & Development** - Improved engine reorg tests with explicit finality management and expanded execute-blob Hive coverage ([#26584](paradigmxyz/reth#26584), [#26606](paradigmxyz/reth#26606)). - Added a persistent-datadir testing node and made dev-mined blocks canonical immediately ([#26774](paradigmxyz/reth#26774), [#26761](paradigmxyz/reth#26761)). - **Bench** - Restored metrics visibility in benchmark run configurations ([#26461](paradigmxyz/reth#26461)). - **Dependencies & Releases** - Updated Alloy to 2.4.x, `alloy-hardforks`/`alloy-eip7928` to 0.4.8, and released Reth 2.5.0–2.5.1 ([#26663](paradigmxyz/reth#26663), [#26666](paradigmxyz/reth#26666), [#26685](paradigmxyz/reth#26685), [#26687](paradigmxyz/reth#26687), [#26700](paradigmxyz/reth#26700), [#26771](paradigmxyz/reth#26771)). ## Migrations 🔗 Amp thread: https://ampcode.com/threads/T-01a036f2-23b2-77c0-8959-5c3f0f9df6c5 - Upgraded Reth to `00ff650`, Alloy to `2.4.1`, and related dependencies to match their latest APIs. - Renamed workspace lint keys from kebab-case to snake_case for updated Cargo lint syntax. - Removed the no-longer-needed crate recursion limit. - Migrated hashed storage construction from the removed `from_iter` API to direct struct initialization. - Reused Reth’s prepared snapshot manifest, base URL, and data directory, removing Tempo’s duplicate manifest discovery, fetching, parsing, and path-resolution logic. - Updated snapshot planning to handle Reth’s new `(plan, prepared)` return value and execution’s optional prepared manifest. - Updated `HashedPostStateProvider` implementations and callers for its new fallible `ProviderResult` return type. - Added the required RPC log associated type and identity `convert_log` implementation for the updated receipt converter trait. - Updated pooled transaction construction for the reordered transaction field and new `blob_cell_availability` field. [GitHub Workflow](https://github.com/tempoxyz/tempo/actions/runs/32804636065) --------- Co-authored-by: Alexey Shekhirin <github@shekhirin.com> Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com> Co-authored-by: Matthias Seitz <19890894+mattsse@users.noreply.github.com> Co-authored-by: Richard Janis Goldschmidt <701177+SuperFluffy@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extracts the staged pipeline unwind handling from #24100.
On startup, a lagging
partial_state_triefrontier first unwinds to the durable trie tip with full changed-branch walks. A lower RocksDB/static-file target then runs as a separate normal unwind.walk_all_changed_branch_children
This new flag is required when unwinding to partial_state_trie in order to handle the following scenario:
In this situation we would unwind 14->12. However when doing the unwind only N's child 4 will be in the prefix set which is used; 2 will not be in the prefix set because it was only updated in block 10. Because of this a stale hash for 2 would get used, and any ancestor cached hashes would end up incorrect as well.
The fix is to enable this
walk_all_changed_branch_childrenmode, under which the TrieWalker will visit all children of a branch when just one child of the branch is in the prefix set. This way any stale sibling hashes are also refreshed.A durable metadata marker records the original Finish and partial-state-trie frontiers before recovery starts. Interrupted recovery resumes from this marker, which is deleted only after the special unwind completes successfully.