trace-interop

Reproducible evidence for a proposed Ethereum trace_* specification, with per-client change reports.

Each bar is one client's development build across every decision in the ledger, regenerated with each capture. See progress by client for the counts, trend and definitions.

Why this exists

Several Ethereum clients expose the Parity trace_* methods, but the same request can mean different things or return different shapes. Tracing tools carry client-specific workarounds, and maintainers have no shared contract to test against.

The execution-apis standardization discussion (#890) needs a concrete proposal and a clear view of its impact on each client. This repository supplies the draft specification, runnable cases on every client, and reports that explain what would change, why, and where to look in the code. Observed behavior stays separate from recommended changes, and the draft remains a proposal for review.

Join the trace working group on Telegram to discuss the proposal and coordinate client work.

Where to start

Client maintainers

1. Your client's report: each difference from the draft, with a minimal example, the proposed change and links into your source. 🛠️ marks differences already covered by a submitted fix.
2. Decision pages: the recommendation, rationale and cross-client observations behind each proposed change.
3. Related pull requests: open and merged fixes, with the decisions each one addresses.
4. Changes since the previous matrix: verdicts that changed for each client build since the last capture.

Specification reviewers

• Draft specification PR #895 and its source branch, which this repository pins rather than duplicating.
• Decision ledger: every open question, its policy status and per-client verdicts.
• Source review: the draft compared with Parity, current client source and the sibling eth_* methods; its recommendations are adopted in the ledger and draft.
• Upstream acceptance: what execution-apis requires to accept the proposal, and lessons from recent reviews.
• Geth draft implementation: an experimental fork implementing the draft, evaluated separately from the native clients.

Contributors

• Reproduce and review: builds, frozen chains, captures, report generation and evidence storage.
• Reviewing a proposed rule: how to separate API decisions from execution defects and record a disagreement.

How it works

1. Fixtures. Frozen generated chains (fixtures/chains) and request corpora (fixtures/corpora) define each case, its setup controls and its independent expectations.
2. Capture. Hive runs the corpora against the latest release and development builds of Besu, Erigon, Nethermind and Reth, plus the Geth draft fork, after a freshness preflight. Stateful scenarios cover reorg restoration and pruned history.
3. Evidence. Each run keeps its requests, raw responses, client logs and build lock under evidence/, checksummed and never rewritten.
4. Assessment. Responses are checked against the pinned draft schemas and against independent models of execution, gas, fees and state. Assertion models explain what each status means and what remains unassessed.
5. Reports. scripts/build_reports.py regenerates the client reports, decision pages and verdict matrix from the evidence selected by reports.lock.json.

Run it

uv sync --locked
uv run trace-interop verify
uv run python scripts/check_schema.py
uv run python scripts/build_reports.py

scripts/check.sh runs the whole CI check; see usage for optional prek hooks.

On Linux with Docker, capture a current comparison of the latest stable releases, development images and Geth draft head:

uv run python scripts/run_matrix.py --output runs/current-matrix

Scope and boundaries

All nine traditional methods are covered: trace_call, trace_callMany, trace_rawTransaction, trace_replayTransaction, trace_replayBlockTransactions, trace_block, trace_transaction, trace_get and trace_filter, with the call-trace, stateDiff and vmTrace output families.

The Geth fork is a candidate implementation of the draft, reported as go-ethereum_trace. It is not upstream Geth support or an independent vote for the proposal.

This is not a public testnet, performance benchmark, general EVM fuzzer or client ranking. Tests use disposable generated chains, and no mainnet synchronization is required.

Documentation index

Methodology: reproduce and review · stateful scenarios · assertion models · reviewing a proposed rule

Decision studies: H13 signed transaction validation · mined transaction probes · H15 unsigned simulation fees · Nethermind truncated validation responses

Reviews and audits: source review · historical records: harness assertion audit · H17 assessment audit · measurement and draft review corrections

Upstream work: related pull requests · upstream acceptance · Geth draft implementation