db/state, db/config3: erigondb.toml as source of truth for commitment branch referencing - #21452
Merged
Merged
Conversation
…ranches Mechanical rename of the commitment-branch referencing flag (field + the ForTest* toggle) repo-wide; no behavior change. Existing tests are the safety net. Behavioral decoupling of the merge-scheduling guards is a later task.
…gatorSqueezeCommitmentValues
… aggregator Apply the erigondb.toml-resolved commitment-refs flag to the global statecfg.Schema and the live commitment domain via Aggregator.applyReferencesInCommitmentBranches, called from ReloadErigonDBSettings and from the WithErigonDBSettings -> Open builder path (covering all standalone entry points).
Add a version.Version field to FilesItem, populated in Domain.openDirtyFiles from the version parsed by MatchVersionedFile, and expose it via a visibleFile.Version() accessor. This makes a per-file regime marker reachable at the commitment deref/merge sites (Tasks 7/8) via the existing range-match loop, without extending getLatestFromFiles.
…sion (Task 6) Bump commitment domain DataKV.Current to v2.1 in versions.yaml (regenerated version_schema_gen.go) so the read ceiling accepts both v2.0 referenced and v2.1 plain commitment files (range [v1.0, v2.1]). Add Domain.kvWriteVersion: the commitment .kv write version is decoupled from Current — v2.0 when ReferencesInCommitmentBranches is on, v2.1 when off. Other domains keep writing at DataKV.Current.
…write flag (Task 8)
…nditionally (Task 9) Decouple the commitment merge transformer's expand and re-shorten decisions so flipping references_in_commitment_branches off on a populated datadir stays correct: - vt closure: expand input short keys to plain whenever the input file is referenced (its own version+range), independent of the live flag; re-shorten into the merged output only when the flag is on AND the output range reaches the threshold. Never return a referenced valBuf as-is — that copied stale offsets into the merged file. - evaluate the threshold against the input file range for expansion and the output range for re-shortening (previously conflated on the output range). - SqueezeCommitmentFiles now writes its re-referenced output to the flag-derived kvNewFilePath (v2.0) rather than reusing the input filename. A rebuild writes plain v2.1 files during its flag-off window; reusing that name for squeezed referenced content produced a v2.1-named file with short keys, which reads as garbage after a disk reload. Tests (commitment_merge_version_test.go): - TestCommitmentMergeFlagOffExpandsReferencedInputs: disjoint early-only keys keep v2.0 referenced branches as merge winners; flag-off merge must produce a v2.1 plain file with no short offsets. - TestCommitmentRebuildSqueezeReadableAfterReload: full rebuild→squeeze cycle reopened from disk reads back to the same root with version/content consistent.
…cks (Task 10) Integrity checks decided whether a commitment file carried short references by range alone (ValuesPlainKeyReferencingThresholdReached). With v2.0-referenced and v2.1-plain files coexisting, a v2.1 file at/above the threshold was wrongly treated as referencing and silently under-verified. Expose Version() on the kv.VisibleFile interface, export state.CommitmentBranchReferenced, and route the three integrity decision sites through commitmentFileReferencing(file, stepSize) so they consult the file's own version+range.
- db/state/aggregator.go: gate applyReferencesInCommitmentBranches writes on actual change so the live commitment-domain flag read by background merges is mutated only at first resolution (before any merge spawns), never concurrently - db/state/merge.go: stamp merged commitment file's in-memory version from kvWriteVersion so freshly-merged files are classified by regime without a folder reopen (was zero-version, over-approximating as referenced until restart) - db/state/commitment_merge_version_test.go: add TestCommitmentReadDerefsReferencedFileWithFlagOff (reads a referenced v2.0 branch with the flag off through the real deref path and asserts short refs expand to plain — the central safety claim, previously only covered by self-consistent root equality) and TestMergedCommitmentFileVersionStampedInMemory; both mutation-verified red->green - docs/plans: correct the downgrade caveat — an old binary hard-rejects v2.1 via MustSupport panic, it does not silently misread
integrateDirtyFiles left FilesItem.version at the zero value, unlike mergeFiles and openDirtyFiles. The version-aware commitment read/merge logic treats a zero version as referenced (below v2.1), so a freshly built plain commitment file misclassified as referenced until restart, forcing the read down the deref/range-lookup path it should skip.
Contributor
There was a problem hiding this comment.
Pull request overview
Migrates the commitment "references in branches" setting from a build-time constant (AggregatorSqueezeCommitmentValues) to a per-datadir field in snapshots/erigondb.toml (references_in_commitment_branches, defaults to true). Reads become per-file version-driven (a commitment .kv is referenced iff its version < v2.1 and its range reaches the referencing threshold), while writes follow the live flag. The commitment DataKV version is bumped v2.0 → v2.1, with the read window accepting both and the write version flag-derived.
Changes:
- Add
*boolReferencesInCommitmentBranchestoErigonDBSettings, thread it viaWithErigonDBSettings/ReloadErigonDBSettingsintostatecfg.Schemaand the live commitment domain; renameDomainCfg.ReplaceKeysInValues → ReferencesInCommitmentBranchesand the test helper. - Decouple commitment
.kvwrite version fromDataKV.Current(flag-on → v2.0, flag-off → v2.1); persist per-file parsed version onFilesItem, expose viaVisibleFile.Version(), stamp newly merged/integrated files; squeeze writes to a flag-derivedkvNewFilePathrather than reusing the input name. - Make read deref, integrity checks, and merge planning/transformer use the per-file version+range predicate (
CommitmentBranchReferenced), so referenced inputs are always expanded even with the flag off and plain v2.1 files are never re-dereferenced.
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| db/config3/config3.go | Adds DefaultReferencesInCommitmentBranches = true. |
| db/state/erigondb_settings.go | Adds the new TOML field, accessor with default, normalizes/writes it in resolution paths. |
| db/state/erigondb_settings_test.go | New tests for roundtrip, absent-field normalization, legacy/fresh write behavior. |
| db/state/aggregator2.go | Plumbs referencesInCommitmentBranches through AggOpts/Open and WithErigonDBSettings. |
| db/state/aggregator.go | Renames ForTest… helper, adds applyReferencesInCommitmentBranches, gates merge planning on resolved-referencing predicate. |
| db/state/aggregator_refs_test.go | New tests for reload/open propagation of the flag. |
| db/state/aggregator_test.go | New tests for commitmentMergeInputsReferenced / commitmentVisibleFilesReferenced. |
| db/state/aggregator_fuzz_test.go | Rename of ForTestReplaceKeysInValues. |
| db/state/statecfg/statecfg.go | Renames ReplaceKeysInValues field on DomainCfg. |
| db/state/statecfg/state_schema.go | Removes AggregatorSqueezeCommitmentValues, wires default into the commitment domain. |
| db/state/statecfg/state_schema_test.go | Asserts default and schema agree. |
| db/state/statecfg/versions.yaml, version_schema_gen.go | Bump commitment DataKV current to v2.1. |
| db/state/domain.go | Adds kvWriteVersion(); stamps integrated file version. |
| db/state/domain_test.go | New tests for write-version selection and acceptance window. |
| db/state/domain_committed.go | New CommitmentBranchReferenced predicate plus visible/merge/file helpers; version-aware deref and merge transformer. |
| db/state/domain_committed_test.go | New table tests for CommitmentBranchReferenced. |
| db/state/dirty_files.go | FilesItem.version field; populated on open; visibleFile.Version() implementation. |
| db/state/dirty_files_test.go | New tests for visibleFile.Version() and openDirtyFiles version population. |
| db/state/merge.go | Stamps merged output version. |
| db/state/squeeze.go | Renames flag usage; squeeze output now uses kvNewFilePath. |
| db/state/squeeze_test.go, squeeze_concurrent_rebuild_test.go | Rename of ForTest… helper. |
| db/state/commitment_merge_version_test.go, commitment_version_testutil_test.go | New mixed-regime / merge / rebuild+squeeze tests and shared helpers. |
| db/test/aggregator_ext_test.go | Helper rename. |
| db/state/trie_reader_integration_test.go | Helper rename. |
| db/kv/visible_file.go | Adds Version() to VisibleFile interface. |
| db/integrity/commitment_integrity.go | Switches integrity decisions to version-aware predicate; updated log messages. |
| db/integrity/commitment_version_test.go, commitment_version_integration_test.go | New unit + integration coverage of version-regime behavior. |
| cmd/integration/commands/commitment.go | Helper rename in rebuild path. |
| db/agents.md | Documents snapshots/erigondb.toml fields and read/write semantics. |
| .claude/skills/erigondb-sync-integration-test-plan/SKILL.md | Updates plan with new field, producer workflow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Resolve 3 conflicts from main's overlapping refactors: - db/state/aggregator.go: drop frozenBlocks/FrozenBlocksProvider (removed repo-wide by #21415 block-catchup-recovery), keep commitmentRefsOverride. - db/state/domain_committed.go: keep both the PR's reshorten gate and #21303's per-merge findShortenedKey caches; the vt closure uses all three. - db/state/squeeze.go: keep the PR rename ForTestReferencesInCommitmentBranches and the wantsReferencesInBranches guard over main's old ForTestReplaceKeysInValues.
…ersion End-to-end guard for the two upgrade scenarios: an in-place upgrade (erigondb.toml without references_in_commitment_branches -> default true) keeps producing v2.0 referenced commitment files; a downloaded plain snapshot set (references=false) produces v2.1 plain files. Existing tests covered each link (resolution->live-domain bool, and domain bool->kvWriteVersion) separately but not the datadir-shape->produced-version chain.
If a datadir ends up with both a v2.0 (referenced) and v2.1 (plain) commitment file for the same step range (e.g. a download landing the other regime's file), openDirtyFiles must collapse them to one dirty item resolved via MatchVersionedFile to the highest version (v2.1), leaving the v2.0 twin on disk but unopened. Asserted order-independent.
…mmitment (fix Windows CI) wipeCommitment dir.RemoveFile'd commitment .kv files while the aggregator still mmapped them. Unix allows unlink-while-open, but Windows refuses (file in use), failing TestCommitmentRebuildSqueezeReadableAfterReload on the windows-2025 test legs. Release the handles via agg.Close() before deleting, then reopen a fresh agg over the wiped folder and return the new (db, agg); all 5 call sites updated to capture them.
kvWriteVersion special-cased the commitment domain by name and inlined the v2.0/v2.1 literals. Replace it with an optional KVWriteVersion func(*DomainCfg) on DomainCfg: the generic method delegates to the hook (else DataKV.Current), and the commitment regime->version mapping is declared once next to the schema. The hook reads the live ReferencesInCommitmentBranches flag, preserving single-source-of-truth (the flag is toggled at runtime during rebuild/squeeze, so the version must stay derived, not stored).
yperbasis
requested changes
Jun 24, 2026
yperbasis
left a comment
yperbasis
left a comment
Member
There was a problem hiding this comment.
Requesting changes — one blocking correctness issue, one that compounds it, and a doc fix. The version-gate design is otherwise sound; these are the exceptions.
1. Data race + silent corruption: commitmentKVWriteVersion reads the refs flag unlocked
db/state/statecfg/state_schema.go:194
The merge captures the content regime under commitmentRefsMu (aggregator.go:1915, via referencesInCommitmentBranches()) and shortens content from that snapshot, but the output filename is stamped later through merge.go:422 → kvNewFilePath → kvWriteVersion → commitmentKVWriteVersion, which reads c.ReferencesInCommitmentBranches directly and unlocked on the merge goroutine. ReloadErigonDBSettings (stage_snapshots.go:209, concurrent with background merges) writes that field under the lock.
In the plain-set + downloader path the flag flips true→false once when the toml arrives. If that lands between the two reads, content is shortened (snapshot = true) but the file is stamped v2.2 (live = false) → a v2.2 file carrying short offset keys. On read, CommitmentBranchReferenced(v2.2, …) is false → deref skipped → offsets misread as plain 20/52-byte keys → wrong commitment root. The reverse flip stamps a plain file v2.1 and the reader tries to deref it.
This is the exact race the new mutex was added for — the comment at aggregator.go:273 already states merge/squeeze paths must go through the locked accessor instead of touching the field directly, and kvWriteVersion is the one path that doesn't. TestReferencesInCommitmentBranchesConcurrent doesn't cover it.
Fix: derive the write version from the same captured snapshot used for the content decision (single source of truth) rather than re-reading the field, so the stamp and the content can never disagree.
2. Converter no longer cross-checks version vs content (compounds the above)
db/state/commitment_convert.go:151 (gate at :273)
detectFileState now derives squeezed solely from the file version, and buildValueTransformer returns no transformer when detectedSqueezed == targetSqueezed. A v2.2-named file that actually holds short keys (e.g. one produced by the race above) is classified plain and, with TargetSqueeze=false, copied verbatim — propagating the stale offsets. The removed detectSqueezeState content sampler would have caught the version/content disagreement. Trusting the stamp is only safe while the stamp is always correct, which the first issue violates.
3. Wrong versions in the squeeze comment and PR body
db/state/squeeze.go:228
The comment says the squeezed output is stamped v2.0 and the rebuild-window input is a "plain v2.1" file, but the actual scheme is referenced = v2.1 / plain = v2.2 (DataKV.Current = v2.2). The PR description carries the same inverted mapping (true→v2.0, false→v2.1). Please correct both so the on-disk version contract isn't misdocumented.
Member
Author
|
Thanks for the thorough review — the critical one is fixed, and walking through the rest. Critical (
|
AskAlexSharov
added a commit
that referenced
this pull request
Jun 27, 2026
…files are referenced OpenSequentialView gains a separateReadahead bool: true keeps the separate MADV_SEQUENTIAL mmap (isolated aggressive readahead); false shares the decompressor's mmap with MADV_NORMAL (Close is then a no-op). AggregatorRoTx.mergeFiles passes !commitmentVisibleFilesReferenced() into the domain-values and standalone inverted-index merge input-read loops: when the datadir has referenced commitment files, concurrent dereference does random reads on the .kv values, so the separate sequential view's deactivate-behind eviction would evict the pages those deref reads need - fall back to MADV_NORMAL. History and its inverted-index merges always use the separate sequential view: their .v/.ef files are never dereferenced. Accessor builds (fresh merged output) and forkable/block-snapshot merges likewise keep it. Depends on #21452 (commitmentVisibleFilesReferenced / CommitmentBranchReferenced).
| // CommitmentBranchReferenced reports whether a commitment file at fileVersion over from..to carries | ||
| // shortened key references — a property of the file (version+range), independent of the live write flag. | ||
| func CommitmentBranchReferenced(fileVersion version.Version, stepSize, from, to uint64) bool { | ||
| return fileVersion.Less(version.V2_2) && ValuesPlainKeyReferencingThresholdReached(stepSize, from, to) |
Collaborator
There was a problem hiding this comment.
and version and sampling?
Member
Author
There was a problem hiding this comment.
if >= v2.2 - definitely not referenced;
and threshold is as it was - reference >2 steps only
…ed key Return a sentinel error from the ReplacePlainKeys callback as soon as a shortened key is found instead of scanning the whole branch; any error still reads as referenced, preserving the conservative semantics.
…mmitment files are referenced (#22072) ## Summary Stacked on **#21452** (commitment branch referencing). Two commits: 1. **Cherry-pick of #21482** (`alex/mmap_merge_performance`) — run merges and index builds on a separate `MADV_SEQUENTIAL` mmap instead of `MadvSequential()` on the shared mmap, so concurrent random RPC readers aren't disturbed by merge readahead. (This is also PR #22046 against `main`; included here because Part 2 builds directly on these call sites.) 2. **Make that readahead optional, gated on commitment dereferencing.** `OpenSequentialView` gains a `separateReadahead bool`: - `true` → separate `MADV_SEQUENTIAL` mmap (isolated aggressive readahead), as in commit 1. - `false` → share the decompressor's mmap with `MADV_NORMAL`; `Close` is a no-op. `AggregatorRoTx.mergeFiles` computes `!commitmentVisibleFilesReferenced()` once and threads it into the domain / history / inverted-index merge **input-read loops**. When the datadir has referenced commitment files, concurrent dereference (`replaceShortenedKeysInBranch`) does **random** reads on those files, so the separate sequential view's *deactivate-behind* eviction would evict the very pages those deref reads need — fall back to `MADV_NORMAL` instead. Accessor builds (fresh merged output, no concurrent deref) and forkable / block-snapshot merges (no commitment) keep the separate sequential view (`true`). ## Behaviour matrix (per Q1) | datadir state | merge input reads | |---|---| | referenced commitment files present (deref) | shared mmap, `MADV_NORMAL` (don't `MADV_RANDOM`) | | no referenced files | separate mmap, `MADV_SEQUENTIAL` | ## Notes - Detection uses #21452's `commitmentVisibleFilesReferenced()` / `CommitmentBranchReferenced` (per-file version + range), hence the dependency. - `db/seg`: added `TestOpenSequentialView` asserting both modes read identical words and that the shared-view `Close` is a no-op on the decompressor's mmap. - Pre-existing failing tests on the #21452 tip (`TestCommitmentKvVersionAcceptance`, a `v2.2` `MustSupport` panic, and a commitment-merge `index out of range`) reproduce identically without this change; they are not introduced here.
…ainValues sticks GenesisToBlock resolves erigondb settings against the real datadir to configure its in-memory aggregator, creating erigondb.toml with the default regime before SetUpBlockReader's flag-aware resolve runs. That later resolve then finds the file already present and drops --commitment.plainValues. Resolve the first-start regime once in New() before genesis computation so the flag is honored.
Comment on lines
344
to
357
| for _, d := range cList { | ||
| if err := d.WithReadAhead(func() error { | ||
| g := d.MakeGetter() | ||
| for g.HasNext() { | ||
| word, _ = g.Next(word[:0]) | ||
| if err := f.AddWord(word); err != nil { | ||
| return err | ||
| } | ||
| } | ||
| return nil | ||
| }); err != nil { | ||
| view, err := d.OpenSequentialView(true) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| defer view.Close() | ||
| g := view.MakeGetter() | ||
| for g.HasNext() { | ||
| word, _ = g.Next(word[:0]) | ||
| if err := f.AddWord(word); err != nil { | ||
| return nil, err | ||
| } | ||
| } | ||
| } |
Comment on lines
102
to
+117
| meta := NumMetadata{} | ||
| for _, item := range filesToMerge { | ||
| var word = make([]byte, 0, 4096) | ||
| startRootNum, endRootNum := item.src.Range() | ||
| compression := f.isCompressionUsed(RootNum(startRootNum), RootNum(endRootNum)) | ||
|
|
||
| if err = item.src.decompressor.WithReadAhead(func() error { | ||
| reader := f.PagedDataReader(item.src.decompressor, compression) | ||
| var k, v []byte | ||
| var fmeta NumMetadata | ||
| if err := fmeta.Unmarshal(reader.GetMetadata()); err != nil { | ||
| return err | ||
| } | ||
| if meta.Count == 0 { | ||
| meta.First = fmeta.First | ||
| } | ||
| meta.Last = fmeta.Last | ||
| meta.Count += fmeta.Count | ||
|
|
||
| for reader.HasNext() { | ||
| k, v, word, _ = reader.Next2(word[:0]) | ||
| if err = writer.Add(k, v); err != nil { | ||
| return err | ||
| } | ||
| p.Processed.Add(1) | ||
| } | ||
| return nil | ||
| }); err != nil { | ||
| view, viewErr := item.src.decompressor.OpenSequentialView(true) | ||
| if viewErr != nil { | ||
| err = viewErr | ||
| return | ||
| } | ||
| defer view.Close() | ||
| reader := seg.NewPagedReader( | ||
| seg.NewReader(view.MakeGetter(), f.cfg.Compression), | ||
| f.cfg.ValuesOnCompressedPage, compression, | ||
| ) |
Comment on lines
+85
to
+97
| // TestCommitmentFileReferencing pins that the integrity referencing decision is taken from a FULL | ||
| // content scan of the file, independent of the file's version stamp — so it catches a file whose | ||
| // version says plain (v2.1 below) yet whose content carries shortened keys. | ||
| func TestCommitmentFileReferencing(t *testing.T) { | ||
| t.Run("referenced content is referencing even when version says plain", func(t *testing.T) { | ||
| f := fakeVisibleFile{path: writeCommitmentKV(t, true), endTxNum: 20, version: version.V2_1} | ||
| require.True(t, commitmentFileReferencing(f)) | ||
| }) | ||
| t.Run("plain content is not referencing even when version says referenced", func(t *testing.T) { | ||
| f := fakeVisibleFile{path: writeCommitmentKV(t, false), endTxNum: 20, version: version.V2_0} | ||
| require.False(t, commitmentFileReferencing(f)) | ||
| }) | ||
| } |
AskAlexSharov
enabled auto-merge
July 3, 2026 01:45
…mments Copilot review: several docs and comments still described the old mapping (referenced v2.0 / plain v2.1). The implemented gate is referenced v2.1 / plain v2.2, read as referenced iff version < v2.2. Update db/agents.md, the erigondb-sync-integration-test-plan skill, the squeeze and first-start-regime comments. Also narrow the backend seed comment: it pre-empts GenesisToBlock's erigondb.toml creation only on legacy datadirs.
mh0lt
added a commit
that referenced
this pull request
Jul 3, 2026
Bring the cache stack current with main (parallel/streaming commitment correctness fixes #22184/#22113, nibblized-keccak cache #22185, erigondb.toml commitment referencing #21452, trie io.Writer trace #21859, etc.). One conflict in commitment_context.go's trieContext: keep both this branch's probeSd/probeTx (adaptive trunk-pin probe) and main's traceW (io.Writer trace).
pull Bot
pushed a commit
to Dustin4444/erigon
that referenced
this pull request
Jul 13, 2026
…d branch (erigontech#21376) Enables the plain-commitment regime by flipping the compiled default; the refs-flag machinery is already on main (erigontech#21452). Three changes: - `config3.DefaultReferencesInCommitmentBranches` → `false` — produce/consume plain commitment by default - `EmbeddedWebseeds` → erigon36 buckets (all chains) - `DefaultSnapshotGitBranch` → `release/3.6` **[dnm]** until the erigon36 buckets are fully published — they currently serve v2.x state but 404 on the newest block ranges, so a fresh sync can't complete. Note: release/3.6 uses `step_size=390625`; a legacy `1562500` datadir is incompatible and needs a step-rebase first (erigontech#21905 fixes the 3.4 code default).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
erigondb.toml'sreferences_in_commitment_branches(*bool, absent →true) is the source of truth for the commitment "references in branches" regime, replacing theAggregatorSqueezeCommitmentValuesbuild const. The regime ships with the snapshots, not the binary.Default
true⇒ no behavior change — safe to merge to main as-is. With the default, merges write byte-identicalv2.1referenced files to current main. Turning the plain regime on by default (erigon36 webseeds + release/3.6) is a separate switch, #21376.Versioning:
v2.1= referenced (== main),v2.2= plainMain (#21780) bumped the commitment
.kvtov2.1while still writing referenced (short-offset) content, so on mainv2.1means referenced. This PR keeps that: the default/referenced regime writesv2.1(byte-identical to main), and the opt-in plain regime writes a freshv2.2. The read gate isversion < v2.2 && range ≥ threshold, sov1.x/v2.0/v2.1are referenced and onlyv2.2is plain — a current-main datadir reads correctly, no migration, no misread.Regime is keyed off the commitment
.kvfile versionThe earlier content-sampling detector is gone. A commitment
.kv's regime is decided by its file version plus range:references_in_commitment_branchespicks the version stamped on new merged files —true→v2.1(referenced / short keys, == main),false→v2.2(plain). Read ceilingDataKV.Current = v2.2accepts both;MinSupported = v1.0, sov1.0…v2.2all load.CommitmentBranchReferenced): a file is referenced iffversion < v2.2AND range ≥ the referencing threshold. Sov2.1≥ threshold derefs;v2.2, or anything below threshold, reads plain. Per-file, so mixedv2.1/v2.2files coexist and converge to the live regime lazily through merges. Deterministic — no sample, no false-negative, no per-branch read guard.v2.1) only when the flag is on AND output range ≥ threshold, else writes plain (v2.2).The flag governs new writes only, never reads. It is resolved once at the
erigondb.tomlsettings read and is not mutated during execution or merging, so the version-stamping read always observes a stable value;commitmentRefsMuguards the runtime accessors, and a mid-run regime flip is not a supported operation.--commitment.plainValuesfirst-start flagConvenience flag to pick the regime when a fresh datadir's
erigondb.tomlis created, without hand-editing the file:references_in_commitment_branches = !plainValues, applied only at creation (fresh / legacy / no-downloader paths).erigondb.tomlwins; the flag is logged as ignored.config3.DefaultReferencesInCommitmentBranches). It is the initial-write default only — not a persistent override, no MDBX state.Behavior
true: keeps writingv2.1referenced (== main), derefs old. No migration.erigondb.toml(false) wins; merges producev2.2plain.DataKV.Current < v2.2) rejectsv2.2viaMustSupport— loud panic, not silent misread.Validation
DataKV.Current = v2.1withAggregatorSqueezeCommitmentValues = true(db/state/statecfg: bump commitment domain kv/kvi to v2.1 #21780,c2fcc75255), i.e. main writesv2.1-referenced; the< v2.2gate reads those as referenced.references_in_commitment_branches = false: re-executed the gap, caught up, follows live tip — derefs referenced inputs, produces plain merges, 0 crit / root-mismatch / panic over 1d+; soaked under-race(0 races).--commitment.plainValueson the arb-sepolia host (real binary, fresh datadirs): flag on →references=false, off →true, existing toml wins. Unit + integration coverage for the version gate (CheckStateVerify+CheckCommitmentKvDeref, both regimes) and the flag wiring.main: reconciled the version gate with the commitment converter (erigon commitment convert, [3.6] cmd:erigon commitment convert#21933). The converter follows the version gate on the referenced axis both ways — reads referenced state from the version (CommitmentBranchReferenced) and stamps output by squeeze target per run (v2.1referenced /v2.2plain) — so converter and gate agree by construction. Nibbles V1/V2 stays content-detected. Fulldb/state+execution/commitmentgreen,-raceclean.Known / open
ReplaceKeysInValues → ReferencesInCommitmentBranchesrename is mechanical.erigondb.tomlvs auto-converge-on-restart (db/state: move to noref commitment files production, works with 3.4 ref commitment snapshots #21814) — separate.