feat(test-fixtures): add blockchain_test_engine_reorg format and consume reorg simulator

docs/navigation.md

@@ -56,6 +56,7 @@
               * [Blockchain Engine X Tests](running_tests/test_formats/blockchain_test_engine_x.md)
               * [Transaction Tests](running_tests/test_formats/transaction_test.md)
               * [Blockchain Sync Tests](running_tests/test_formats/blockchain_test_sync.md)
+              * [Blockchain Engine Reorg Tests](running_tests/test_formats/blockchain_test_engine_reorg.md)
               * [Common Types](running_tests/test_formats/common_types.md)
               * [Exceptions](running_tests/test_formats/exceptions.md)
           * [Hive](running_tests/hive/index.md)

docs/running_tests/running.md

@@ -16,6 +16,7 @@ Both `consume` and `execute` provide sub-commands which correspond to different
 | [`consume engine`](#engine)             | Client imports blocks via Engine API `EngineNewPayload` in Hive                         | EVM, block processing, Engine API                            | Staging, Hive | System test                       |
 | [`consume enginex`](#enginex)           | Client imports blocks via Engine API in Hive, optimized by client reuse            | EVM, block processing, Engine API, chain reorgs (implicit\*\*) | Staging, Hive | System test                       |
 | [`consume sync`](#sync)                 | Client syncs from another client using Engine API in Hive                               | EVM, block processing, Engine API, P2P sync                  | Staging, Hive | System test                       |
+| [`consume reorg`](#reorg)               | Client is driven through a DAG of payloads and forkchoice updates in Hive                | EVM, block processing, Engine API, chain reorgs               | Staging, Hive | System test                       |
 | [`consume rlp`](#rlp)                   | Client imports RLP-encoded blocks upon start-up in Hive                                 | EVM, block processing, RLP import (sync\*)                   | Staging, Hive | System test                       |
 | [`build-block`](#block-building)        | Client builds blocks via `testing_buildBlockV1` in Hive, validated against fixture       | EVM, block production, Engine API (testing namespace)        | Staging, Hive | System test                       |
 | [`execute hive`](./execute/hive.md)     | Tests executed against a client via JSON RPC `eth_sendRawTransaction` in Hive           | EVM, JSON RPC, mempool                                       | Staging, Hive | System test                       |
@@ -173,6 +174,25 @@ The `consume sync` command:
 5. **Monitors sync progress** and validates that the sync client reaches the same state.
 6. **Verifies final state** matches between both clients.
 
+## Reorg
+
+| Nomenclature   |                                 |
+| -------------- | ------------------------------- |
+| Command        | `consume reorg`                 |
+| Simulator      | `eels/consume-reorg`            |
+| Fixture format | `blockchain_test_engine_reorg`  |
+
+The consume reorg method drives a client through a DAG of Engine API payloads and forkchoice updates to test chain reorganization behavior directly, rather than as a side effect of client reuse (see [Implicit Chain Reorg Coverage](#implicit-chain-reorg-coverage)). Each test describes a block DAG (side chains are first-class, identified by label) and an ordered, branching script of `engine_newPayloadVX`/`engine_forkchoiceUpdatedVX`/JSON-RPC steps together with every outcome the [execution-apis](https://github.com/ethereum/execution-apis) specification allows a conformant client to return.
+
+The `consume reorg` command:
+
+1. **Initializes the client under test** with genesis state.
+2. **Sends an initial forkchoice update** to genesis and verifies the client's genesis block hash via `eth_getBlockByNumber(0)`.
+3. **Runs the fixture's step script**: sends each request verbatim, selects the first outcome (of possibly several spec-legal ones) that matches the observed response, and runs that outcome's branch steps.
+4. **Fails** on the first step whose observed result matches none of its listed outcomes.
+
+Unlike `consume engine`, which sends a linear payload list and always follows each valid payload with a forkchoice update to it, `consume reorg` fixtures describe explicit forkchoice states (head, safe, finalized), client-built payloads bound to new labels via `engine_getPayloadVX`, and assertions of observable state (`eth_getBalance`, `eth_getLogs`, `eth_getTransactionReceipt`, ...) after each forkchoice update. See the [Blockchain Engine Reorg Tests](./test_formats/blockchain_test_engine_reorg.md) format page for the full fixture structure.
+
 ## Block Building
 
 | Nomenclature   |                          |

docs/running_tests/test_formats/blockchain_test_engine_reorg.md

@@ -0,0 +1,225 @@
+# Blockchain Engine Reorg Tests  <!-- markdownlint-disable MD051 (MD051=link-fragments "Link fragments should be valid") -->
+
+The Blockchain Engine Reorg Test fixture format tests are included in the fixtures subdirectory `blockchain_tests_engine_reorg`, and describe a DAG of Engine API payloads plus a branching script of Engine API / JSON-RPC steps to verify chain reorganization behavior.
+
+These are produced by the `ReorgTest` test spec.
+
+## Description
+
+Unlike [`BlockchainEngineFixture`](./blockchain_test_engine.md) (a linear payload list where the consumer always sends `forkchoiceUpdated(head=payload)` after each payload), this format lets a test describe side chains, explicit forkchoice states (head, safe, finalized), multiple legal outcomes per step, outcome-specific follow-up steps (branches), client-built payloads (`getPayload` binds the built payload to a new label), and transaction-pool observations.
+
+Every block in the DAG names its parent by label instead of relying on list order, so sibling blocks and blocks built on top of an invalid block are first-class. Every hash-valued step field is a label (`"genesis"` is reserved for the genesis block, `"zero"` for the zero hash, and `"latest"`/`"null"`/`"any"` are reserved RPC tags/special values valid only where a field's description says so; labels introduced by `getPayload.bind` are resolved at run time); the consumer resolves labels to hashes itself, so fixtures are byte-identical across clients.
+
+Each step's `expect` field is a list of legal outcomes (the [Engine API reference model](../../library/execution_testing_specs.md) fills it in at fill time for any step an author left unannotated, deriving the outcomes the [execution-apis](https://github.com/ethereum/execution-apis) specification allows a conformant client to return); the consumer selects the first outcome matching the observed response and runs that outcome's `branches` steps.
+
+Every outcome in a step's `expect`, model-filled or hand-authored, must be spec-permitted at that point, so a passing fixture establishes conformance. Where the specification is ambiguous, an outcome is marked `disputed` with a reference or rationale; a client's spec violation is never added as an alternative.
+
+A single JSON fixture file is composed of a JSON object where each key-value pair is a different [`HiveFixture`](#hivefixture) test object, with the key string representing the test name.
+
+## Consumption
+
+For each [`HiveFixture`](#hivefixture) test object in the JSON fixture file, perform the following steps:
+
+1. Start the client under test using:
+
+    - [`network`](#-network-fork) to configure the execution fork schedule according to the [`Fork`](./common_types.md#fork) type definition.
+    - [`pre`](#-pre-alloc) as the starting state allocation of the execution environment for the test.
+    - [`genesisBlockHeader`](#-genesisblockheader-fixtureheader) as the genesis block header.
+    - `HIVE_ENGINE_MAX_REORG_DEPTH`, derived from [`minReorgDepth`](#-minreorgdepth-optionalnumber), if present.
+
+2. Send an initial `engine_forkchoiceUpdatedVX` to the genesis block and verify it returns `VALID`; verify the client's genesis block hash via `eth_getBlockByNumber(0)`.
+
+3. Run [`steps`](#-steps-liststep) in order. For each step:
+
+    1. Resolve every label referenced by the step to a hash (or to the label itself, for a client-built payload not yet bound).
+    2. Send the request (or perform the RPC observation).
+    3. Select the first entry of the step's `expect` list whose constraints match the observed response; fail the test if none match.
+    4. Run the steps listed in `branches` under the matched outcome's `id`, if any (recursively).
+
+## Structures
+
+### `HiveFixture`
+
+#### - `network`: [`Fork`](./common_types.md#fork)
+
+Fork configuration for the test. It is guaranteed that this field contains the same value as `config.network`.
+
+#### - `config`: [`FixtureConfig`](./blockchain_test_engine.md#fixtureconfig)
+
+Chain configuration object to be applied to every client running the test.
+
+#### - `genesisBlockHeader`: [`FixtureHeader`](./blockchain_test.md#fixtureheader)
+
+Genesis block header.
+
+#### - `pre`: [`Alloc`](./common_types.md#alloc-mappingaddressaccount)
+
+Starting account allocation for the test. State root calculated from this allocation must match the one in the genesis block.
+
+#### - `blocks`: [`Mapping`](./common_types.md#mapping)`[String,`[`FixtureReorgBlock`](#fixturereorgblock)`]`
+
+The block DAG, keyed by label.
+
+#### - `steps`: [`List`](./common_types.md#list)`[`[`Step`](#step)`]`
+
+Ordered, branching script of Engine API / JSON-RPC steps.
+
+#### - `minReorgDepth`: [`Optional`](./common_types.md#optional)`[`[`Number`](./common_types.md#number)`]`
+
+Minimum side-chain reorg depth (in blocks) the client must apply without refusing for capacity reasons; the consumer sets the client's cap (`HIVE_ENGINE_MAX_REORG_DEPTH`) to it. `None` keeps client defaults.
+
+#### - `meta`: [`Mapping`](./common_types.md#mapping)`[String,``Any``]`
+
+Free-form metadata about the test (e.g. `class`,`reorgDepth`) for offline analysis; not consumed by the runner. A `reorgDepth` of `1` states a suite requirement: every client must apply a same-height sibling reorg.
+
+### `FixtureReorgBlock`
+
+#### - `parent`: `String`
+
+Label of the parent block (`"genesis"` for a block extending the genesis block).
+
+#### - `payload`: `FixtureNewPayloadRequest`
+
+The block's `engine_newPayloadVX` request: `params` (version-dependent parameter tuple, see [`FixtureEngineNewPayload`](./blockchain_test_engine.md#fixtureenginenewpayload)) and `newPayloadVersion`.
+
+### `Step`
+
+A step is one of the variants below, distinguished by its `type` field. Every variant shares:
+
+#### - `type`: `String`
+
+One of `newPayload`,`forkchoiceUpdated`,`getPayload`,`assertHead`,`assertCanonical`,`assertState`,`assertReceipt`,`assertLogs`,`sendRawTransaction`,`assertTxStatus`.
+
+#### - `description`: [`Optional`](./common_types.md#optional)`[String]`
+
+Human-readable description of the step, for logging.
+
+#### `newPayload`
+
+- `block`: `String` — label of the block (or a `getPayload`-bound label) to send via `engine_newPayloadVX`.
+- `expect`: [`List`](./common_types.md#list)`[`[`Outcome`](#outcome)`]` — legal outcomes.
+- `branches`: [`Mapping`](./common_types.md#mapping)`[String,`[`List`](./common_types.md#list)`[`[`Step`](#step)`]]` — follow-up steps per matched outcome id.
+
+#### `forkchoiceUpdated`
+
+- `head` / `safe` / `finalized`: `String` — labels; `safe`/`finalized` default to `"zero"`.
+- `version`: [`Number`](./common_types.md#number) — `engine_forkchoiceUpdatedVX` version; when unset, that of the payload attributes' fork for a build request, else of the head block's fork.
+- `payloadAttributes`: [`Optional`](./common_types.md#optional)`[`[`PayloadAttributes`](#payloadattributes)`]` — if set, a payload build is requested.
+- `expect` / `branches`: as above.
+
+#### `getPayload`
+
+- `bind`: `String` — new label for the built payload.
+- `version`: [`Number`](./common_types.md#number) — `engine_getPayloadVX` version.
+- `parent`: `String` — expected parent of the built payload.
+- `transactionsInclude` / `transactionsExclude`: [`List`](./common_types.md#list)`[`[`TxRef`](#txref)`]` — transactions that must (or must not) be in the built payload.
+
+The wait before `engine_getPayloadVX` is the consumer's `--get-payload-wait-time` option, not a fixture field.
+
+#### `assertHead`
+
+- `latest` / `safe` / `finalized`: [`Optional`](./common_types.md#optional)`[String]` — expected labels, checked via `eth_getBlockByNumber`.
+
+#### `assertCanonical`
+
+- `blocks`: [`Mapping`](./common_types.md#mapping)`[`[`HexNumber`](./common_types.md#hexnumber)`,`[`Optional`](./common_types.md#optional)`[String]]` — expected label (or `None` for "no block") at each height.
+
+#### `assertState`
+
+- `at`: `String` — block label or `"genesis"` (that block's own state; it must be canonical when the step runs), or `"latest"`. Default `"latest"`.
+- `accounts`: [`Mapping`](./common_types.md#mapping)`[`[`Address`](./common_types.md#address)`,`[`AccountExpectation`](#accountexpectation)`]`.
+
+#### `assertReceipt`
+
+- `tx`: [`TxRef`](#txref).
+- `block`: [`Optional`](./common_types.md#optional)`[String]` — expected receipt block label, or `None` for no receipt.
+- `status`: [`Optional`](./common_types.md#optional)`[`[`HexNumber`](./common_types.md#hexnumber)`]`.
+
+#### `assertLogs`
+
+- `address`: [`Optional`](./common_types.md#optional)`[`[`Address`](./common_types.md#address)`]`.
+- `fromBlock` / `toBlock`: [`HexNumber`](./common_types.md#hexnumber)`|String` — defaults `"earliest"` / `"latest"`.
+- `blocks`: [`List`](./common_types.md#list)`[String]` — labels whose logs must appear, one entry per expected log.
+
+#### `sendRawTransaction`
+
+- `tx`: [`TxRef`](#txref).
+- `expect`: [`List`](./common_types.md#list)`[String]` — subset of `["accepted", "rejected"]`; default `["accepted"]`.
+
+#### `assertTxStatus`
+
+- `tx`: [`TxRef`](#txref).
+- `expect`: [`List`](./common_types.md#list)`[String]` — subset of `["included", "pending", "dropped"]`.
+- `includedIn`: [`Optional`](./common_types.md#optional)`[String]` — required block label when `included` matches.
+
+### `Outcome`
+
+One legal outcome of an Engine API step. Every set field is a constraint; unset fields are not checked.
+
+#### - `id`: `String`
+
+Identifier; selects the `branches` entry to run when matched.
+
+#### - `disputed`: [`Optional`](./common_types.md#optional)`[String]`
+
+If set, the specification is ambiguous about this outcome; the value is a reference (e.g. an issue URL). A disputed outcome still passes.
+
+#### - `status`: [`Optional`](./common_types.md#optional)`[String]`
+
+Expected `payloadStatus.status` (`VALID`,`INVALID`,`SYNCING`,`ACCEPTED`,`INVALID_BLOCK_HASH`).
+
+#### - `latestValidHash`: [`Optional`](./common_types.md#optional)`[String]`
+
+Expected `latestValidHash` as a block label, `"null"`, or `"any"`.
+
+#### - `errorCode`: [`Optional`](./common_types.md#optional)`[`[`Number`](./common_types.md#number)`]`
+
+Expected JSON-RPC error code (e.g. `-38002`,`-38006`).
+
+#### - `anyError`: [`Optional`](./common_types.md#optional)`[``Bool``]`
+
+If `true`, any JSON-RPC error matches (for uncoded errors).
+
+#### - `headMoved`: [`Optional`](./common_types.md#optional)`[``Bool``]`
+
+`forkchoiceUpdated` only: whether `latest` equals the requested head right after the call. Distinguishes an applied update from a no-op when both answer `VALID` with the same `latestValidHash`.
+
+#### - `validationError`: [`Optional`](./common_types.md#optional)`[String]`
+
+`"required"` or `"none"`: whether `validationError` must be present or absent.
+
+#### - `payloadId`: [`Optional`](./common_types.md#optional)`[String]`
+
+`"nonNull"` or `"null"`, for a `forkchoiceUpdated` with payload attributes.
+
+### `PayloadAttributes`
+
+Payload attributes sent with `forkchoiceUpdated` to start a build; fields match the Engine API's `PayloadAttributesVX`.
+
+### `AccountExpectation`
+
+#### - `balance` / `nonce`: [`Optional`](./common_types.md#optional)`[`[`HexNumber`](./common_types.md#hexnumber)`]`
+
+#### - `storage`: [`Optional`](./common_types.md#optional)`[`[`Mapping`](./common_types.md#mapping)`[`[`Hash`](./common_types.md#hash)`,`[`Hash`](./common_types.md#hash)`]]`
+
+### `TxRef`
+
+Reference to a transaction of a fixture block.
+
+#### - `block`: `String`
+
+#### - `index`: [`Number`](./common_types.md#number)
+
+Default `0`.
+
+## Differences from Blockchain Engine Tests
+
+1. **Block DAG, not a list**: blocks name their parent by label, so side chains, invalid-block descendants and re-convergence are first-class.
+2. **Branching step script**: `steps` is not a flat payload list; a step's `expect` can list several spec-legal outcomes and `branches` continues down the outcome that was actually observed.
+3. **Explicit forkchoice state**: `forkchoiceUpdated` steps set `head`/`safe`/`finalized` independently instead of always following the last payload.
+4. **Client-built payloads**: `getPayload` retrieves and binds a payload the client built itself, which can then be delivered back via `newPayload`.
+5. **State observations**: `assertState`/`assertReceipt`/`assertLogs`/`assertTxStatus` steps check observable RPC state after a forkchoice update, instead of a single fixture-wide `post` allocation.
+
+## Fork Support
+
+Blockchain Engine Reorg Tests are only supported for post-merge forks (Paris and later), as they rely entirely on the Engine API.

packages/testing/src/execution_testing/cli/pytest_commands/consume.py

@@ -51,9 +51,12 @@ def get_command_logic_test_paths(command_name: str) -> List[Path]:
             / "simulator_logic"
             / f"test_via_{test_command}.py"
         ]
-    elif command_name == "sync":
+    elif command_name in ["sync", "reorg"]:
         command_logic_test_paths = [
-            base_path / "simulators" / "simulator_logic" / "test_via_sync.py"
+            base_path
+            / "simulators"
+            / "simulator_logic"
+            / f"test_via_{command_name}.py"
         ]
     elif command_name == "direct":
         command_logic_test_paths = [
@@ -132,6 +135,12 @@ def sync() -> None:
     pass
 
 
+@consume_command(is_hive=True)
+def reorg() -> None:
+    """Client consumes Engine API reorg tests (payload DAG + step script)."""
+    pass
+
+
 @consume.command(
     context_settings={"ignore_unknown_options": True},
 )

packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/simulators/base.py

@@ -67,6 +67,7 @@ def check_live_port(test_suite_name: str) -> Literal[8545, 8551]:
         "eels/consume-engine",
         "eels/consume-enginex",
         "eels/consume-sync",
+        "eels/consume-reorg",
         "eels/build-block",
     }:
         return 8551