Problem:
Erigon can't remove old (frozen) files. Not blocks files, not state files.
It's problem for --prune.mode=minimal with big uptime - it only creating-merging files but never delete old history files.
[x] Step 1: move from refcnt per-file to refcnt per visibleFiles set (#20462, #21397)
[x] Step 2: remove frozen field - optimization of refncts is not needed anymoer (#22126)
[x] Step 3: prune state history files in --prune.mode=minimal (#22123)
[x] Step 4: to not download --prune.include-commitment-history (#21200)
[x] Step 5: prune custom data: --prune.include-commitment-history
[x] Step 6: to not download and prune --persist.receipt (#22349)
[x] Step 7: per-visibleFiles refcnt (Step 2) for blocks files (and caplin files) (#22246, #22365)
[x] Step 8: do long-living View use for blocks files (#22343, #22353, #22445, ...)
[x] Step 9: do Step 3 for blocks files (maybe don't - will see)
[x] Step 10: move-away "agg.Retire() -> dirtyFiles.Lock()" from hot-path (from block execution. probably just to background). Same about blkfiles.Retir() -> dirtyFiles.Lock()"
[x] Step 11: do re-test that node synced on Archive node - can be switched to Full and cleanup older files
[x] Step 12: do re-test what happen if file never comes to visible list (dangling) - like 0-4.acc.v when all node files are actually starting from 128-256.v
[] Step 13: re-visit download "black list logic" (switch to "white list"). Postponed.
[] Step 14: follow-up on long-living View of blockFiles - to remove fallback. Postponed.
Related:
Problem:
Erigon can't remove old (frozen) files. Not blocks files, not state files.
It's problem for
--prune.mode=minimalwith big uptime - it only creating-merging files but never delete old history files.[x] Step 1: move from refcnt per-file to refcnt per visibleFiles set (#20462, #21397)
[x] Step 2: remove
frozenfield - optimization of refncts is not needed anymoer (#22126)[x] Step 3: prune state history files in
--prune.mode=minimal(#22123)[x] Step 4: to not download
--prune.include-commitment-history(#21200)[x] Step 5: prune custom data:
--prune.include-commitment-history[x] Step 6: to not download and prune
--persist.receipt(#22349)[x] Step 7: per-visibleFiles refcnt (Step 2) for blocks files (and caplin files) (#22246, #22365)
[x] Step 8: do long-living View use for blocks files (#22343, #22353, #22445, ...)
[x] Step 9: do Step 3 for blocks files (maybe don't - will see)
[x] Step 10: move-away "agg.Retire() -> dirtyFiles.Lock()" from hot-path (from block execution. probably just to background). Same about
blkfiles.Retir() -> dirtyFiles.Lock()"[x] Step 11: do re-test that node synced on Archive node - can be switched to
Fulland cleanup older files[x] Step 12: do re-test what happen if file never comes to visible list (dangling) - like
0-4.acc.vwhen all node files are actually starting from128-256.v[] Step 13: re-visit download "black list logic" (switch to "white list"). Postponed.
[] Step 14: follow-up on long-living View of blockFiles - to remove fallback. Postponed.
Related: