Skip to content

ai-chat: double STREAM_RESUMING replay duplicates the streaming assistant text part #1733

Closed

Description

@advaitpaliwal

Summary

When a useAgentChat client connects while a turn is mid-stream (e.g. the WebSocket opens after the turn started), the server sends CF_AGENT_STREAM_RESUMING twice to the same connection:

  1. proactively in onConnect (packages/ai-chat/src/index.ts ~L744), and
  2. again in response to the client's CF_AGENT_STREAM_RESUME_REQUEST (~L1059) — the request the React client sends once its message handler is registered (resume defaults to true).

_notifyStreamResuming has no per-connection dedupe and the RESUME_ACK handler has no double-replay guard, so the client ACKs both notifies and replayChunks re-sends the full chunk buffer twice.

The client does have a reset for replays — resetMatchingHydratedAssistantForReplay, gated on pendingReplayResumeRequestIdsRef — but the pending id is deleted on the first replay's start chunk (packages/ai-chat/src/react.tsx ~L1936-1949). When both RESUMING frames arrive before the first replay's start (the common ordering: both notifies are queued back-to-back on the socket), the second replay's start finds the pending id already deleted, skips the reset, passes the status === "observing" gate, and its text-start opens a second text part. All subsequent live deltas feed the new part while the first freezes — the user sees the same reply streaming as two stacked text blocks. The persisted message is clean, so the duplication snaps away on refresh/finalize.

Repro

  1. Agent with an active streamed turn; client connects mid-stream with resume: true (default). Easiest real-world trigger: start the turn over a detached HTTP request and let the WS connect a second later.
  2. Log WS frames: you'll see two cf_agent_stream_resuming, two ACKs, and the turn's start chunk replayed twice.
  3. If text deltas were already buffered when the second replay runs, the assistant message ends up with two step-start/text pairs while streaming.

Fix that worked for us

Client-side, make replay idempotent — reset the matching trailing assistant message on every replayed start, not only when the request id is still pending:

// react.tsx, both the localRequestIds branch and the broadcast branch
if (data.replay && typeof chunkData.messageId === "string" && chunkData.type === "start") {
  pendingReplayResumeRequestIdsRef.current.delete(data.id);
  resetMatchingHydratedAssistantForReplay(chunkData.messageId);
}

We're carrying this as a patch on @cloudflare/ai-chat@0.8.4 in production-like staging and it eliminates the duplication under all orderings we could produce.

Note: we first tried deduping server-side (skip _notifyStreamResuming when the connection is already in _pendingResumeConnections) and recommend against that: a client reconnecting mid-turn on its own request receives the proactive notify but intentionally does not ACK it (localRequestIdsRef early-return), so suppressing the second notify starves the reconnectToStream recovery path (5s timeout, no replay). The client-side idempotent reset is the safe fix.

Happy to send a PR if useful.

Activity

  1. 13 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions