common/race: fix darwin -race crashes from file mmaps in the TSAN heap window

[yperbasis]

Problem

go test -race on darwin (Apple Silicon) flakily dies in mdbx-heavy packages — reproduced at 6/6 on main with EXEC3_PARALLEL=true go test -race ./execution/execmodule ./execution/state on an M-series Mac — with either of:

fatal error: runtime: split stack overflow        (sigpanic → racecall, no DATA RACE report)
fatal error: too many address space collisions for -race mode

The same packages pass the Linux race CI legs, which long disguised this as an environment flake. It isn't — both fatals share one root cause.

Root cause (caught live in lldb)

Catching the original fault under lldb (before the Go runtime mangles it into "split stack overflow") shows:

thread #26, stop reason = EXC_BAD_ACCESS (code=1, address=0x21882000bbb0)
  frame #0: __tsan_read + 44
  frame #2: txnprovider/txpool.(*TxPool).fromDB
(lldb) memory region `($x1 - 0x200000000000)/2`     ← app address for that shadow
[0x000000c410004000-0x000000c810000000) r--          ← a 16GiB mdbx data map
(lldb) memory region $x1                             ← its TSAN shadow
[0x21884800bbb0-0x219a00000000) ---                  ← unmapped

1. Go's race-mode heap lives in TSAN's Go/darwin window [0x00c0…, 0x00e0…); shadow (shadow = app*2 + 0x2000_0000_0000) is mapped per heap arena.
2. Each in-mem test env reserves a 16GiB VA map (InMem geometry upper bound); with dozens of parallel testers the kernel's bottom-up placement exhausts low VA and drops mdbx maps between Go heap arenas.
3. The runtime's racecalladdr validity filter checks one coarse interval [racearenastart, racearenaend) (min/max over arenas) — a sandwiched map passes the check with no shadow → the first instrumented read (txpool fromDB) faults inside __tsan_read. The SEGV lands while racecall is on the g0 stack, so the runtime dies with the misleading split-stack throw. The same squatting also makes heap-arena reservation collide repeatedly → the "too many address space collisions" fatal.

Linux is unaffected because mmap(NULL) there places file maps near 0x7f…, far from the heap window, so they always fail the racecalladdr filter and are simply (silently) invisible to TSAN.

Two repair strategies were tried and rejected with evidence before the final one:

• Calling the runtime's own __tsan_map_shadow per mapping verifiably does nothing here: compiler-rt's Go-mode MapShadow tracks a monotonic ctx->mapped_shadow_* interval and silently returns for requests inside it — interior holes (precisely this case) are skipped. Confirmed in compiler-rt source and empirically (crash inside a region the call had "covered").
• Direct shadow mmaps per mdbx env / per unix.Mmap, locating regions via mach_vm_region + proc_regionfilename, fixed the crash but serialized an O(all-VM-regions) walk into every env open — execution/tests (thousands of env opens) went from 9 minutes to a 1h timeout.

Fix

common/race (linked via blank imports from db/kv/mdbx and common/mmap; everything is compiled out unless race && darwin):

• At init, fill every unmapped gap in the heap window's shadow [app*2+0x2000…, …) with zeroed MAP_FIXED|MAP_ANON|MAP_NORESERVE mappings, leaving existing arena shadow untouched. One-time cost of a few mmaps; zero per-open cost; covers every file mapping the kernel ever places in the window — current or future, mdbx or otherwise. Zero shadow is valid "no prior access" TSAN state, so races on such mappings also become detectable where they were previously fatal — for plain reads/writes; Go atomics on such mappings would still fault, since meta shadow is not pre-mapped.
• The hard-coded layout (window bounds + shadow formula) is self-checked at init against a live heap allocation (it must be in-window with mapped shadow); on mismatch the package disables itself with a warning, restoring old behavior.

Plus: InMem test geometry 16GiB → 1GiB upper bound (only when a testing.TB is supplied, and not for benchmarks, which run sequentially and can need the full map). Unit tests never approach 1GiB per env; this removes the TB-scale VA squatting that pushes file maps into the heap window in the first place and is what triggers the "address space collisions" fatal.

Verification

• Repro rate of EXEC3_PARALLEL=true go test -race ./execution/execmodule ./execution/state (sequential, quiet M-series machine): main 6/6 fatal → this branch 0/8.
• EXEC3_PARALLEL=true go test -race ./execution/tests completes in normal time (the rejected per-open design hung it — kept as a regression gate).
• go test -race ./common/race: asserts the layout self-check and that shadow is actually mapped across the whole window.
• Full EXEC3_PARALLEL=true go test -race ./execution/... green — re-verified after merging main (2026-07-08, -count=1: 52 ok / 0 fail); make lint clean; make erigon integration builds; stubs cross-compile (linux/windows).
• The post-merge sweep initially caught ./execution/engineapi dying 6/6 with the "address space collisions" fatal — a sibling mechanism, not a regression of this fix: the txpool DB's hard-coded 1TB geometry (txnprovider/txpool/assemble.go) cannot fit below the 768GiB heap-window base on darwin, so each tester node's txpool map spanned the window (caught live in vmmap: two 1TB txpool/mdbx.dat maps starting 320MB above the window base) and burned the runtime's 32 race-mode arena hints, which are discarded permanently on collision; the fatal then hits a later innocent allocation. Fixed by flowing the tester's existing 1GB MdbxDBSizeLimit into the txpool config: 6/6 fatal → 6/6 green. Linux is unaffected (top-down mmap places the reservation near 0x7f…).

TDD note: natural occurrence depends on kernel VM placement (hence the flakiness), but the crash is deterministically reproducible — golang/go#80292 carries a standalone ~50-line reproducer (mmap with an address hint into the heap window + heap ballast + one instrumented read, 100% fatal). In-repo, a deterministic crash test would need a separate crashing subprocess (the fault kills the whole test binary), so the unit test pins the fix's load-bearing properties (layout constants, window coverage) and the repetition harness above is the end-to-end gate; each design iteration was validated live in lldb.

This is arguably a Go runtime/TSAN deficiency (coarse interval in racecalladdr, interior-skip in Go-mode MapShadow) — reported upstream as golang/go#80292 with a deterministic standalone reproducer; this PR makes erigon's macOS race runs work today.

Forensics trail: first reported as a suspected environment flake on #21605 (#21605 (comment)).

[Copilot]

Pull request overview

This PR addresses flaky/crashing go test -race runs on macOS (darwin, Apple Silicon) caused by large file mmaps landing inside Go/TSAN’s heap window without corresponding mapped shadow memory. It adds a darwin+race-only initializer to pre-map missing TSAN shadow ranges and reduces MDBX in-mem test VA reservations to avoid exacerbating address-space collisions.

Changes:

• Add common/race darwin+race implementation that fills unmapped TSAN shadow gaps for the Go heap window, plus a targeted unit test.
• Ensure the common/race side-effect init runs for relevant mappings by blank-importing it from MDBX and unix mmap codepaths.
• Reduce default InMem(tb!=nil) MDBX map size upper bound from 16GiB to 1GiB to mitigate macOS VA pressure during parallel tests.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
db/kv/mdbx/kv_mdbx.go	Blank-import common/race; reduce default in-mem test map size to lower VA pressure under parallel test runs.
common/race/shadow_fallback.go	No-op stub for non-race && darwin builds to keep imports safe across platforms.
common/race/shadow_darwin.go	darwin+race TSAN-shadow hole-filling implementation via mach_vm_region + mmap(MAP_FIXED, ...), with runtime layout self-check.
common/race/shadow_darwin_test.go	darwin+race unit test asserting the layout self-check passes and shadow is mapped across the heap window.
common/mmap/mmap_unix.go	Blank-import common/race so mmap users also trigger shadow pre-mapping on darwin race builds.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.