Skip to content

fix: guard replay reads against mid-hydration prefixes across backend respawns - #301

Merged
william0wang merged 1 commit into
mainfrom
fix/replay-hydration-prefix-guard
Oct 6, 2026
Merged

william0wang merged 1 commit into
mainfrom
fix/replay-hydration-prefix-guard

Conversation

@william0wang

Copy link
Copy Markdown
Owner

Problem

Phone wake-reconnect could enter a conversation whose replay ended in the middle; re-entering the same conversation replayed it whole. Root cause (diagnosed across bridge + client, source-verified against upstream): the backend's session/messages returns a PREFIX while a resumed session's store is still hydrating, and three bridge-side gaps let an unguarded plain read ship that prefix (or an empty read) to clients:

  1. backendLoadedSessions stamps survived backend respawns (ensureBackend cleared hydration bookkeeping but not the stamps) — isBackendSessionLive stayed true for a process that never loaded the session, so session/load's alreadyLive check skipped the resume RPC.
  2. A settle poll that exits "stable" inside a hydration gap never arms hydrationUnsettled — later plain reads had no protection at all.
  3. runOneTurn's finally stamped backend-loaded unconditionally — a backend lost mid-turn with a FAILED recovery left a fresh stamp vouching for a session the respawned backend never loaded (ghost stamp, 5-min TTL of empty replays). Found in adversarial review.

Fix

  • resetBackendGeneration() (extracted from ensureBackend's spawn branch) now also voids backendLoadedSessions.
  • fetchMessagesForReplay settle-polls the first replay read of a backend generation (no hydrationWatermark entry — presence, not value, is the observed-settled marker); raiseWatermark materializes the entry at 0; create/fork seed 0 (a fresh store is complete by construction). The turn-path baseline opts out (guardFirstRead:false) — no-watermark is unreachable there once stamps are void, and the settle's 2×300ms gaps must not sit on the prompt path.
  • The turn finally re-stamps only while the subscribe-proven instance (residentBackend) is still the live backend.

Residual, accepted: a read landing above the watermark while a false-stable hydration still grows can still ship a prefix — the client-side last-message-id reconciliation is that defense (separate repo).

Verification

  • Full suite 2087/2087 (three consecutive runs), tsc, eslint, prettier clean.
  • New regression tests: no-watermark alreadyLive load re-settles a mid-hydration ladder (resume-single-flight); respawn voids stamps (session-eviction); failed backend-lost recovery leaves no stamp — verified red on the pre-fix code (turn-state).

… respawns

Three aligned guards against the wake-reconnect truncated-history report
(first entry after phone wake replayed a conversation that ended in the
middle; re-entry was fine once hydration finished):

- resetBackendGeneration() now also voids backendLoadedSessions on every
  spawn: a stamp that survived a respawn kept isBackendSessionLive true
  for a process that never loaded the session, so session/load skipped
  the resume RPC and replayed empty or mid-re-hydration prefix.
- fetchMessagesForReplay settle-polls the first replay read of a backend
  generation (no hydrationWatermark entry); raiseWatermark materializes
  the entry at 0 and create/fork seed it - a fresh store is complete by
  construction. The turn-path baseline opts out (guardFirstRead:false).
- The turn finally re-stamps backend-loaded only while the
  subscribe-proven instance is still the live backend - a failed
  backend-lost recovery must not vouch for a session the respawned
  backend never loaded (ghost stamp, found in review).
@william0wang
william0wang merged commit a09fd43 into main Oct 6, 2026
2 checks passed
@william0wang
william0wang deleted the fix/replay-hydration-prefix-guard branch October 6, 2026 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant