Repository navigation
Conversation
Opening a cold thread, the first event of an uncached thread, and history pages or full outputs of a thread nobody has open each parsed and folded the whole log on the host mailbox, stalling every other session for as long as that took. A session now has at most one hydration. The store writer opens the log in its queue order and records its length, so the read holds exactly the appends queued before it; the parse and fold run on the blocking pool. Records accepted meanwhile are held in order and take the positions after the snapshot's end. When the log is resident the waiting subscription windows go out first, each ending where the snapshot ends, then the held records, so a subscriber sees every record once. A subscription's acknowledgement waits for its window. The log is kept only if the session is still resident. Part of #524
Owner
Author
|
Folded into #591, which carries this commit unchanged as one PR for the whole storage phase. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First slice of #524, stacked on #589. Storage format, fold, compaction and protocol are unchanged.
Loading a cold thread's log (read, parse every record, fold) no longer runs on the host mailbox, so other sessions keep responding while a large thread opens.
pendingand not in the snapshot. Two clients opening the same cold thread share one read.Evidence
Release build, synthetic compacted logs, real
spawn_hostdriven over its protocol channel. Round trips of an unrelated session while a cold open is in progress, median / max:End-to-end open latency is unchanged (large log: 666–720 ms before, 653–689 ms after); the work is the same and the fold dominates (read ≈ 100 ms, fold ≈ 520 ms). Peak RSS unchanged. The machine was under other load during the runs.
SessionLog, where it moved.Merge Danger
Door: two-way
Runtime scheduling only; nothing on disk or on the wire changes.
Blast Radius: host-runtime
history_paging_bench(ignored, needs a log path), Android, Windows, Linux.