Parallel exec: commitment "empty branch data during unfold" on reorg/unwind-to-genesis (Hive consensus UncleFromSideChain_Cancun)

[yperbasis]

Summary

Under the default parallel executor (EXEC3_PARALLEL=true, the default since #21591), importing a block that triggers a reorg/unwind back to genesis crashes the parallel commitment calculator:

got empty branch data during unfold  key=00 row=0 depth=1
[4/6 Execution] commitment: commitmentCalculator: hash sort failed: followAndUpdate:
    unfold: empty branch data read during unfold, compact prefix 00 nibbles

The reorg is aborted, so the canonical head never advances to the new chain. With EXEC3_PARALLEL=false (serial) the exact same import succeeds.

This is live on main (reproduced on 4155fdf41f).

Where it shows up

Hive ethereum/consensus suite, e.g. BlockchainTests/InvalidBlocks/bcMultiChainTest/UncleFromSideChain.json::UncleFromSideChain_Cancun:
https://hive.ethpandaops.io/#/test/generic/1782353287-7717aa7042e85bde42a372c6fd0cf7a9

It is not specific to that one test. The failing set fluctuates run-to-run (bcUncleTest, bcUncleHeaderValidity, bcUncleSpecialTests, bcMultiChainTest, plus some ValidBlocks) — all reorg/multi-chain tests — which points to a parallel-schedule-dependent commitment-on-unwind defect. UncleFromSideChain_Cancun reproduces deterministically and is the smallest repro.

(The suite's other persistent failure, the test file loader meta-test, is unrelated — it's a Hive simulator limitation, .meta/index.json parse + unknown Prague ruleset, and every client fails it.)

Root cause

• Crash site: execution/commitment/hex_patricia_hashed.go:1713 (unfoldBranchNode), driven by the parallel committer execution/stagedsync/committer.go.
• After Unwind Execution from=N to=0, the trie unfolds the root (zero-length prefix, compact 00) and reads empty branch data, but rootChecked is already true, so the legitimate-empty-root special case at hex_patricia_hashed.go:1702 is skipped and it errors. The committer already carries comments about a "blockNum=0 ambiguity in SeekCommitment" on unwind to genesis — this is that hazard biting.

Regression

erigon's history on this suite: 1 fail (loader only) through 2026-06-03, then 8 reorg/uncle failures on 2026-06-07, settling at 2. The bisect window (74fbe0d2..dcef8f74) contains the trigger:

011159cbd6  common/dbg: default EXEC3_PARALLEL=true (#21591)
            -  Exec3Parallel = EnvBool("EXEC3_PARALLEL", false)
            +  Exec3Parallel = EnvBool("EXEC3_PARALLEL", true)

This didn't introduce the bug — it exposed a pre-existing parallel-commitment-on-reorg defect by making parallel the default. #21591's soak validation was forward-sync-from-0, which never exercises an unwind-to-genesis. (#21513 "process all import files" is in the same window but is not the cause: serial and parallel use that identical path; only the executor mode changes the outcome.)

Why CI doesn't catch it

execution/tests/TestLegacyBlockchain runs this exact fixture and passes, even with parallel exec. Every in-process harness backs the raw chaindata KV with mdbx…InMem (db/kv/temporal/temporaltest/kv_temporal_testdb.go → memdb.NewTestDB). For these tiny blocks nothing collates to domain files (step size 390 625), so the commitment branches live in the raw KV, and the unwind-to-0 delete/read behaves differently in-memory vs on-disk. The bug is on-disk-only, so the unit suite is blind to it.

Reproduce

Genesis matches the Hive fixture (0xfc96de62…). 8 RLP blocks: chain A (1-3, head 0x3339…) vs chain B (forks at genesis, valid block 4 = 0xd73a…), plus one invalid post-Paris-uncle block.

erigon init   --datadir DD genesis.json
# default (parallel) — crashes, head stuck at block 3 (0x3339…):
erigon import --datadir DD --networkid 1337 0001.rlp … 0008.rlp
# serial — succeeds, head = 0xd73a4a15…d49347 (== fixture lastblockhash):
EXEC3_PARALLEL=false erigon import --datadir DD --networkid 1337 0001.rlp … 0008.rlp

Mode	Final head	Result
EXEC3_PARALLEL=true (default)	0x3339fd43… (block 3)	❌ reorg aborted
EXEC3_PARALLEL=false (serial)	0xd73a4a15… (block 4)	✅ correct

Workaround

EXEC3_PARALLEL=false for any reorg-heavy import/sync path until the parallel committer's unwind-to-genesis commitment-state restoration is fixed.

[VBulikov]

yperbasis added Imp2 as Importance