Skip to content

perf: archive, unarchive and delete threads with one index write - #585

Closed
Tryanks wants to merge 1 commit into
mainfrom
feat/storage-batch-writes
Closed

Tryanks wants to merge 1 commit into
mainfrom
feat/storage-batch-writes

Conversation

@Tryanks

@Tryanks Tryanks commented Oct 4, 2026 •

Copy link
Copy Markdown
Owner

Summary

First slice of the storage work in #535. Closes #520 and #521. It is the prerequisite for the redb index (#523 / #588), which needs the store writer to be the only owner of index writes.

Archive, unarchive and delete become batch commands, and every lifecycle path persists its index once instead of once per thread.

 protocol Command
-  ArchiveSession { session_id }
-  UnarchiveSession { session_id }
-  DeleteSession { session_id, remove_worktree }
+  ArchiveSessions { session_ids }
+  UnarchiveSessions { session_ids }
+  DeleteSessions { session_ids, remove_worktrees }
 on(archive / unarchive / delete / delete project / settle cascade)
-  for each thread
-    update meta
-    persist index            # N writes
+  collect every affected meta
+  persist index once         # 1 write
 import
-  convert each thread and write it
-  host re-reads the index from disk      # dropped metas still queued in the writer
+  prepare_import converts without writing
+  store writer installs each log atomically
+  one batched meta upsert when the run finishes

Also: a project deleted mid-import no longer gets sessions written against its id (their logs are removed), and a batch naming an unknown thread is rejected whole with unknown_session, as the single commands were.

Not changed: lifecycle semantics. Deleting a thread still leaves descendants with a dangling parent (#518). One side effect: Command::session_id() is None for batch commands, so a thread whose only pending command is an archive or delete no longer appears in the outbox's pending-thread list.

Evidence

Index writes counted with temporary counters (removed) at 200 threads:

Action Before After
Archive all N 1
Unarchive all N 1
Delete 100 threads N 1 index + 1 settings
Delete project N 2 index + 1 settings
  • Test: lifecycle_batches_apply_exact_timestamp_cascades_and_persist (renamed and extended) drives the batched commands and DeleteProject through the pipe handler and asserts the reopened store. Making RemoveSessions persist only its first id fails it.
  • The import test no longer asserts index length after skips, because prepare_import cannot write; within-run dedupe stays covered by import_finalizes_the_index_before_finished_even_after_the_initiator_disconnects.
cargo fmt / clippy --workspace -D warnings / machete   ✓
cargo nextest run --workspace --locked                 886 passed, 6 skipped
Web (wasm32) and iOS simulator with -D warnings        ✓

Android not run locally (no NDK). No visual change.

Merge Danger

Door: two-way

No on-disk format change. Reverting restores the per-thread commands.

Blast Radius: protocol

The protocol version bumps and the single-id commands are removed rather than wrapped, so every client updates with the host. The branch is currently conflicting with main on the version constant (#584 and #586 bumped it since); it needs a rebase to 10 before merge.

…ore writer

Archive, unarchive and delete now take a list of threads and persist with
one index write per command; deleting a project and the settle cascades do
the same. External import prepares each thread off the host loop, installs
its log through the store writer and indexes the whole run in one batched
upsert, so nothing outside the writer rewrites sessions.json while the
host runs and the in-memory index is no longer replaced from disk.

PROTOCOL_VERSION 9: ArchiveSessions, UnarchiveSessions and DeleteSessions
replace the single-thread commands.

Closes #520
Closes #521
@Tryanks Tryanks changed the title perf: batch thread lifecycle commands and route import through the store writer perf: archive, unarchive and delete threads with one index write Oct 6, 2026
@Tryanks

Tryanks commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

Folded into #591, which carries this commit unchanged as one PR for the whole storage phase.

@Tryanks Tryanks closed this Oct 6, 2026
@Tryanks
Tryanks deleted the feat/storage-batch-writes branch October 6, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

External import races the store writer on sessions.json

1 participant