perf(chat): derive the recovery progress marker from the stream log by mattzcarey · Pull Request #2223 · cloudflare/agents

mattzcarey · 2026-09-09T11:06:52Z

What

The chat recovery forward-progress marker is derived from the stream log instead of being a KV counter bumped per credited chunk.

Both hosts (Think, AIChatAgent) used to call bumpChatRecoveryProgress on the streaming hot path: a Durable Object KV get and put per credited chunk. Per turn that is two per tool call, one per text or reasoning segment start, one per five seconds of deltas, and one per five seconds of forwarded sub-agent output. The chunk log those same hosts flush to is already the durable record of produced content, so the counter was a second copy of information the log holds.

ResumableStream.progressMarker() now reads it from the log:

a live stream contributes its log tail; a settled stream its final cursor, stamped exact at settlement;
when a stream's rows are deleted (the cutover from feat(streams): rollover block log and atomic stream → session cutover #2216, the reclaim on the next start(), clearAll()) its segments are folded into a retired total first, in the same synchronous block, so the sum never moves when rows go away;
a parent forwarding a sub-agent's output credits one unit through creditProgress(), throttled as before, because that output goes to clients and not to the parent's own log.

The invariants the counter carried hold by construction. A reconnect replay or a recovery re-persist reads the log without appending, so neither can fake progress (#1637). Compaction rewrites the transcript, not the log (#1628). The marker only grows.

The Streams sync aperture gains cursor(streamId) and onDelete(hook). The retire is a property of deletion, not of the caller: every path that removes a stream's rows (the public delete(), the aperture's deletes, a cutover's discard) runs the hook with the row and its cursor first, in the same synchronous block and, for a cutover, the same transaction. The chat adapter registers one hook that retires chat rows, so a chat stream deleted through the public capability can no longer lower the marker. The hook is owned per capability: a host whose startup retried constructs the adapter again, and the new adapter replaces the earlier hook rather than adding to it, so a deletion is retired once.

AIChatAgent now flushes a settled tool result (tool-output-available / -error / -denied) to SQLite the moment it is stored, as Think already did. It captures a completed, often non-idempotent side effect that an eviction before the next packed flush would lose, and under the derived marker the flush is what makes a settled result count as progress immediately, as it did under the per-chunk counter. Streaming deltas keep the buffer's packing.

Also in this PR

Three follow-ups from triaging the Devin review of #2216 against the merged code. They touch the same cutover and stream-adapter paths, so they ride here.

A Think agent-tool child kept nothing for its parent. Think's cutover never passed a discard option, so a child turn's stream rows were deleted with its message and a parent attaching after completion replayed nothing. ai-chat already kept child rows; Think now does the same through the run map, and the child's next start() reclaims them. Devin's note on feat(streams): rollover block log and atomic stream → session cutover #2216 marked this resolved, which was true only for ai-chat.
The legacy-table flag survived a rolled-back cutover. The settle's tail read can fold a v1 stream's rows inside the cutover transaction and, if they were the last, drop the legacy table and clear the in-memory flag. A throwing commit rolls the table back but not the flag, and the restored rows became invisible. The flag is now re-derived from the schema when the transaction leaves it changed. Not reachable from shipped code paths, since an append folds first, but cheap to close.
discardCompleted had no caller since the cutover deletes rows itself. Removed.

Cost

	Before	After
Per credited chunk	KV get + KV put	nothing
Per stream retired (cutover, reclaim, clear)	—	one row write alongside the deletion, plus one KV put mirroring the total for rollback
Per five seconds of forwarded child output	KV get + KV put	one row write
Reading the marker (incident evaluation)	KV get	stream rows + one log-tail row per live stream + one progress row

Pinned on real DO SQLite by progress-marker.test.ts: the marker counts flushed segments, is unchanged by a replay, a cutover, a reclaim and a clear, and a sequence that stores 36 chunks across three streams writes under 40 rows in total. The streams write-parity bench moves from +2 to +3 rows per reclaimed turn, which is the retire row.

Upgrade

The unit changed from "credited chunk" to "durable segment": one per ten packed streaming chunks, one per settled tool result, one per explicit credit. For delta-heavy turns that is a finer measure than the old counter, which credited a long text segment once per five seconds, so DEFAULT_CHAT_RECOVERY_MAX_WORK moves from 1 000 to 10 000 to stay as generous as before in the new unit while remaining finite. The no-progress window is unaffected since it only asks whether the marker moved.

The old KV counter is read once per isolate and folded into its own column of the progress row by max, beside the retired total, so it can never swallow segments retired before the seed landed and a repeated seed is idempotent. The hosts memoize the seed as a promise so two concurrent reads both wait for it, and an incident's work baseline is never captured from an unseeded marker; a failed read is not cached, so a transient storage error does not block recovery for the isolate's life.

Rollback stays safe in the other direction too. The hosts mirror the durable part of the marker back to the KV key after each retire and each credit (one put per stream retired, none per chunk), so a build rolled back to the counter never reads a marker lower than an incident recorded under this one. bumpChatRecoveryProgress stays exported and deprecated.

One known one-time effect: the first post-upgrade read counts an in-flight stream's live segments on top of the credits the old counter already banked for them, which reads as progress once and hands an in-flight incident one extra no-progress window.

Tests

agents: legacy-fold.test.ts gains the rolled-back cutover case (restored v1 rows stay visible; the next append continues at the old cursor). New progress-marker.test.ts (segment counting across cutover, reclaim, clear and replay; a chat row deleted through the public capability; the host mirror fires per retire and per credit; an adapter constructed three times retires a stream once; the legacy seed beside retired segments, idempotent); streams suites, chat unit suites, and the session migration suite pass with the parity pin updated.
@cloudflare/think: agent-tools.test.ts gains a parent that attaches after the child completed and still gets its chunks. Both hosts: the reconnect-immunity, N9 forwarding, silent-child, compaction-immunity and work-budget tests now drive and observe the derived marker. The two discussion: recovery/repair ergonomics so products don't need a full onChatRecovery override (persist-false footgun, ask_user preserve-as-text, onExhausted payload) #1631 exhaustion tests seed their incident with the marker as it stands, since the seeded stream's segments now count. One ai-chat concurrency test registered its done listener after awaiting a broadcast on a second socket and relied on cross-socket delivery order; it now registers first. Both full workers projects pass.
npm run typecheck, oxfmt --check ., oxlint clean.

Follows #2216: the cutover it introduced is where a stream's segments are retired into the total.

The chat recovery budget keys off one monotonic number: did the turn produce new durable content since the last attempt, and how much work has it done since the incident opened. Both hosts maintained it as a Durable Object KV counter bumped per credited chunk — a get and a put per tool call and per text or reasoning segment, on the streaming hot path — even though the chunk log they flush to already is the durable record of produced content. ResumableStream.progressMarker() now reads it from the log: a live stream contributes its log tail, a settled stream its final cursor, and a stream whose rows are deleted (the cutover, the reclaim on the next start, a clear) has its segments folded into a retired total first, so the sum never moves when rows go away. A parent forwarding a sub-agent's output credits one unit explicitly, since that output goes to clients and not to the parent's own log. Nothing is written per chunk; one row is written per stream retired and per explicit credit. A reconnect replay or a recovery re-persist reads the log without appending, so neither can fake progress (#1637); compaction rewrites the transcript, not the log (#1628). The old KV counter is read once per isolate and seeded into the retired total by max, so an in-flight incident never sees the marker read lower after the upgrade; the key is left in place so a rollback resumes from a value no lower than it left. AIChatAgent now flushes a settled tool result the moment it is stored, as Think already did: the result is durable before the next packed flush and counts as progress immediately.

changeset-bot · 2026-09-09T11:06:57Z

🦋 Changeset detected

Latest commit: f7a442d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
agents	Patch
@cloudflare/ai-chat	Patch
@cloudflare/think	Patch
@cloudflare/agent-think	Patch
@cloudflare/channels	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

agent-think · 2026-09-09T11:09:00Z

🟡 agents import sizes

Measured 265 runtime imports as minified bundles. The primary size is gzip; raw minified size is included for diagnosis. An existing import growing by more than 10% is marked red. This report is informational.

Red	Yellow	Green	Unchanged	New	Removed
0	43	6	216	0	0

Compared d2b51308 with f7a442db. Open workflow run.

Changed imports (49)

Status	Import	Base gzip	Head gzip	Delta
🟡	`agents/chat#ResumableStream`	4.6 KiB	5.1 KiB	+461 B (+9.74%)
🟡	`agents/streams#Streams`	4.1 KiB	4.2 KiB	+104 B (+2.46%)
🟡	`agents/chat#createChatStreams`	6.3 KiB	6.4 KiB	+102 B (+1.59%)
🟡	`agents/streams#sseResponse`	855 B	857 B	+2 B (+0.23%)
🟡	`agents/chat#CHAT_RECOVERY_STABLE_RETRY_DELAY_SECONDS`	2.3 KiB	2.3 KiB	+2 B (+0.08%)
🟡	`agents/chat#KV_DELETE_MAX_KEYS`	2.3 KiB	2.3 KiB	+2 B (+0.08%)
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_TERMINAL_MESSAGE`	2.4 KiB	2.4 KiB	+2 B (+0.08%)
🟡	`agents/chat#bumpChatRecoveryProgress`	2.4 KiB	2.4 KiB	+2 B (+0.08%)
🟡	`agents/chat#repairInterruptedToolParts`	2.6 KiB	2.6 KiB	+2 B (+0.07%)
🟡	`agents/chat#CHAT_RECOVERING_FLAG_TTL_MS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_RECOVERY_INCIDENT_TTL_MS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#aiSdkRecoveryCodec`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#applyChunkToParts`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#AutoContinuationController`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_MESSAGE_TYPES`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_RECOVERY_ALARM_DEBOUNCE_MS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_NO_PROGRESS_TIMEOUT_MS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#normalizeToolInput`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_MAX_ATTEMPTS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_MAX_OOM_RETRIES`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_MAX_WORK`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_STABLE_TIMEOUT_MS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#AGENT_TOOL_STREAM_PROGRESS_BUMP_THROTTLE_MS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_STREAM_PROGRESS_CREDIT_THROTTLE_MS`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_RECOVERING_KEY`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_RECOVERY_PROGRESS_KEY`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_LAST_TERMINAL_KEY`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#wrapChatFiberSnapshot`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#CHAT_RECOVERY_INCIDENT_KEY_PREFIX`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#readChatRecoveryProgress`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#pendingChatTerminal`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#clearChatTerminal`	2.3 KiB	2.3 KiB	+1 B (+0.04%)
🟡	`agents/chat#recordChatTerminal`	2.4 KiB	2.4 KiB	+1 B (+0.04%)
🟡	`agents/chat#ChatStreamStalledError`	2.4 KiB	2.4 KiB	+1 B (+0.04%)
🟡	`agents/chat#AgentToolStreamProgressThrottle`	2.4 KiB	2.4 KiB	+1 B (+0.04%)
🟡	`agents/chat#StreamProgressCreditThrottle`	2.4 KiB	2.4 KiB	+1 B (+0.04%)
🟡	`agents/chat#buildChatRecoveringFrame`	2.4 KiB	2.4 KiB	+1 B (+0.04%)
🟡	`agents/chat#classifyAgentToolChildRecovery`	2.4 KiB	2.4 KiB	+1 B (+0.04%)
🟡	`agents/chat#parseProtocolMessage`	2.5 KiB	2.5 KiB	+1 B (+0.04%)
🟡	`agents/chat#resolveChatRecoveryConfig`	2.6 KiB	2.6 KiB	+1 B (+0.04%)
🟡	`agents/chat#TurnQueue`	2.6 KiB	2.6 KiB	+1 B (+0.04%)
🟡	`agents/chat#ResumeHandshake`	3.0 KiB	3.0 KiB	+1 B (+0.03%)
🟡	`agents/chat#enforceRowSizeLimit`	3.5 KiB	3.5 KiB	+1 B (+0.03%)
🟢	`agents/chat#reconcileOrphanPartial`	2.4 KiB	2.4 KiB	-1 B (-0.04%)
🟢	`agents/chat#partAwaitsClientInteraction`	2.4 KiB	2.4 KiB	-1 B (-0.04%)
🟢	`agents/chat#buildInClauseStrings`	2.4 KiB	2.4 KiB	-1 B (-0.04%)
🟢	`agents/chat#setChatRecovering`	2.5 KiB	2.5 KiB	-1 B (-0.04%)
🟢	`agents/chat#AbortRegistry`	2.5 KiB	2.5 KiB	-1 B (-0.04%)
🟢	`agents/chat#PreStreamTurns`	2.6 KiB	2.6 KiB	-1 B (-0.04%)

All 265 current runtime imports

Status	Import	Gzip	Raw minified
⚪	`agents#__DO_NOT_USE_WILL_BREAK__agentContext`	258.6 KiB	1130.0 KiB
⚪	`agents#__DO_NOT_USE_WILL_BREAK__withInvocationScope`	258.6 KiB	1130.0 KiB
⚪	`agents#Agent`	258.6 KiB	1130.0 KiB
⚪	`agents#AGENT_TOOL_MILESTONE_PART`	258.6 KiB	1130.0 KiB
⚪	`agents#AGENT_TOOL_PROGRESS_PART`	258.6 KiB	1130.0 KiB
⚪	`agents#buildAgentPath`	259.1 KiB	1132.3 KiB
⚪	`agents#buildAgentUrl`	259.3 KiB	1132.7 KiB
⚪	`agents#callable`	258.6 KiB	1130.1 KiB
⚪	`agents#camelCaseToKebabCase`	258.6 KiB	1130.0 KiB
⚪	`agents#createHeaderBasedEmailResolver`	258.8 KiB	1130.4 KiB
⚪	`agents#DEFAULT_AGENT_STATIC_OPTIONS`	258.6 KiB	1130.0 KiB
⚪	`agents#DurableObjectOAuthClientProvider`	258.6 KiB	1130.0 KiB
⚪	`agents#getAgentByName`	258.6 KiB	1130.0 KiB
⚪	`agents#getCurrentAgent`	258.6 KiB	1130.0 KiB
⚪	`agents#getSubAgentByName`	258.9 KiB	1130.7 KiB
⚪	`agents#isDurableObjectCodeUpdateReset`	258.6 KiB	1130.0 KiB
⚪	`agents#isDurableObjectMemoryLimitReset`	258.6 KiB	1130.0 KiB
⚪	`agents#isDurableObjectStorageReset`	258.6 KiB	1130.1 KiB
⚪	`agents#isPlatformTransientError`	258.6 KiB	1130.0 KiB
⚪	`agents#MCP_SERVER_ID_MAX_LENGTH`	258.6 KiB	1130.0 KiB
⚪	`agents#MessageType`	258.8 KiB	1130.3 KiB
⚪	`agents#normalizeServerId`	258.6 KiB	1130.0 KiB
⚪	`agents#parseSubAgentPath`	258.6 KiB	1130.0 KiB
⚪	`agents#routeAgentEmail`	258.9 KiB	1130.7 KiB
⚪	`agents#routeAgentRequest`	259.2 KiB	1131.9 KiB
⚪	`agents#routeSubAgentRequest`	258.8 KiB	1130.6 KiB
⚪	`agents#SqlError`	258.6 KiB	1130.0 KiB
⚪	`agents#StreamingResponse`	258.6 KiB	1130.0 KiB
⚪	`agents#SUB_PREFIX`	258.6 KiB	1130.0 KiB
⚪	`agents#unstable_callable`	258.7 KiB	1130.2 KiB
⚪	`agents/agent-tools#agentTool`	112.5 KiB	538.2 KiB
⚪	`agents/browser#BrowserConnector`	50.5 KiB	176.6 KiB
⚪	`agents/browser#browserContent`	36.3 KiB	127.4 KiB
⚪	`agents/browser#browserExtract`	36.3 KiB	127.4 KiB
⚪	`agents/browser#browserLinks`	36.3 KiB	127.4 KiB
⚪	`agents/browser#browserMarkdown`	36.3 KiB	127.4 KiB
⚪	`agents/browser#browserPdf`	36.3 KiB	127.3 KiB
⚪	`agents/browser#BrowserRenderingError`	36.0 KiB	126.7 KiB
⚪	`agents/browser#browserScrape`	36.3 KiB	127.4 KiB
⚪	`agents/browser#browserScreenshot`	36.3 KiB	127.3 KiB
⚪	`agents/browser#browserSnapshot`	36.3 KiB	127.4 KiB
⚪	`agents/browser#CdpSession`	37.2 KiB	129.8 KiB
⚪	`agents/browser#CodemodeRuntime`	39.6 KiB	139.0 KiB
⚪	`agents/browser#connectBrowser`	37.8 KiB	131.4 KiB
⚪	`agents/browser#connectBrowserSession`	37.5 KiB	130.4 KiB
⚪	`agents/browser#connectUrl`	37.6 KiB	130.5 KiB
⚪	`agents/browser#createBrowserSession`	36.3 KiB	127.5 KiB
⚪	`agents/browser#DEFAULT_EXEC_SWEEP_IDLE_MS`	36.0 KiB	126.6 KiB
⚪	`agents/browser#DEFAULT_SWEEP_IDLE_MS`	36.0 KiB	126.6 KiB
⚪	`agents/browser#deleteBrowserSession`	36.1 KiB	126.9 KiB
⚪	`agents/browser#DurableBrowserSessionStore`	36.4 KiB	127.6 KiB
⚪	`agents/browser#getBrowserRecording`	36.2 KiB	127.1 KiB
⚪	`agents/browser#listBrowserTargets`	36.1 KiB	126.9 KiB
⚪	`agents/browser#loadCdpSpec`	36.6 KiB	128.3 KiB
⚪	`agents/browser#runQuickAction`	36.0 KiB	126.6 KiB
⚪	`agents/browser/ai#createBrowserRuntime`	146.0 KiB	630.3 KiB
⚪	`agents/browser/ai#createBrowserTools`	146.0 KiB	630.3 KiB
⚪	`agents/browser/ai#createQuickActionTools`	122.5 KiB	554.3 KiB
⚪	`agents/browser/tanstack-ai#createBrowserTools`	161.7 KiB	699.2 KiB
🟢	`agents/chat#AbortRegistry`	2.5 KiB	8.9 KiB
🟡	`agents/chat#AGENT_TOOL_STREAM_PROGRESS_BUMP_THROTTLE_MS`	2.3 KiB	8.2 KiB
⚪	`agents/chat#AgentToolProgressEmitter`	2.7 KiB	9.5 KiB
🟡	`agents/chat#AgentToolStreamProgressThrottle`	2.4 KiB	8.3 KiB
🟡	`agents/chat#aiSdkRecoveryCodec`	2.3 KiB	8.2 KiB
⚪	`agents/chat#applyAgentToolEvent`	3.2 KiB	11.0 KiB
🟡	`agents/chat#applyChunkToParts`	2.3 KiB	8.2 KiB
⚪	`agents/chat#applyToolUpdate`	2.4 KiB	8.4 KiB
🟡	`agents/chat#AutoContinuationController`	2.3 KiB	8.2 KiB
⚪	`agents/chat#awaitWithDeadline`	2.4 KiB	8.4 KiB
⚪	`agents/chat#broadcastTransition`	3.2 KiB	11.4 KiB
🟡	`agents/chat#buildChatRecoveringFrame`	2.4 KiB	8.4 KiB
🟢	`agents/chat#buildInClauseStrings`	2.4 KiB	8.4 KiB
🟡	`agents/chat#bumpChatRecoveryProgress`	2.4 KiB	8.3 KiB
⚪	`agents/chat#byteLength`	2.5 KiB	8.4 KiB
🟡	`agents/chat#CHAT_LAST_TERMINAL_KEY`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_MESSAGE_TYPES`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_RECOVERING_FLAG_TTL_MS`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_RECOVERING_KEY`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_RECOVERY_ALARM_DEBOUNCE_MS`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_RECOVERY_INCIDENT_KEY_PREFIX`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_RECOVERY_INCIDENT_TTL_MS`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_RECOVERY_PROGRESS_KEY`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_RECOVERY_STABLE_RETRY_DELAY_SECONDS`	2.3 KiB	8.2 KiB
⚪	`agents/chat#CHAT_RECOVERY_TASK_NAME`	2.3 KiB	8.2 KiB
🟡	`agents/chat#CHAT_STREAM_PROGRESS_CREDIT_THROTTLE_MS`	2.3 KiB	8.2 KiB
⚪	`agents/chat#ChatRecoveryEngine`	4.4 KiB	15.3 KiB
⚪	`agents/chat#chatRecoveryTaskRunOptions`	2.4 KiB	8.6 KiB
🟡	`agents/chat#ChatStreamStalledError`	2.4 KiB	8.3 KiB
🟡	`agents/chat#classifyAgentToolChildRecovery`	2.4 KiB	8.5 KiB
🟡	`agents/chat#clearChatTerminal`	2.3 KiB	8.3 KiB
⚪	`agents/chat#clientResolvableToolNames`	2.3 KiB	8.3 KiB
⚪	`agents/chat#ContinuationState`	2.7 KiB	9.8 KiB
⚪	`agents/chat#createAgentToolEventState`	2.3 KiB	8.3 KiB
⚪	`agents/chat#createChatFiberSnapshot`	2.5 KiB	8.6 KiB
⚪	`agents/chat#createChatRecoveryTaskDefinition`	2.6 KiB	9.0 KiB
🟡	`agents/chat#createChatStreams`	6.4 KiB	22.0 KiB
⚪	`agents/chat#createChatTurnTaskDefinition`	2.7 KiB	8.9 KiB
⚪	`agents/chat#createToolsFromClientSchemas`	114.3 KiB	545.4 KiB
⚪	`agents/chat#crossMessageToolResultUpdate`	2.4 KiB	8.6 KiB
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_MAX_ATTEMPTS`	2.3 KiB	8.2 KiB
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_MAX_OOM_RETRIES`	2.3 KiB	8.2 KiB
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_MAX_WORK`	2.3 KiB	8.2 KiB
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_NO_PROGRESS_TIMEOUT_MS`	2.3 KiB	8.2 KiB
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_STABLE_TIMEOUT_MS`	2.3 KiB	8.2 KiB
🟡	`agents/chat#DEFAULT_CHAT_RECOVERY_TERMINAL_MESSAGE`	2.4 KiB	8.3 KiB
⚪	`agents/chat#dispatchChatRecoveryToHandoff`	3.0 KiB	9.9 KiB
⚪	`agents/chat#drainInteractionApplies`	2.3 KiB	8.3 KiB
🟡	`agents/chat#enforceRowSizeLimit`	3.5 KiB	11.2 KiB
⚪	`agents/chat#hasIncompleteToolBatch`	2.4 KiB	8.6 KiB
⚪	`agents/chat#interceptAgentToolBroadcast`	2.5 KiB	8.7 KiB
⚪	`agents/chat#isPlatformFailure`	2.6 KiB	8.9 KiB
⚪	`agents/chat#isReplayChunk`	2.4 KiB	8.7 KiB
⚪	`agents/chat#iterateWithStallWatchdog`	2.6 KiB	8.8 KiB
🟡	`agents/chat#KV_DELETE_MAX_KEYS`	2.3 KiB	8.2 KiB
⚪	`agents/chat#listActiveChatRecoveryIncidents`	2.4 KiB	8.4 KiB
⚪	`agents/chat#MAX_BOUND_PARAMS`	2.3 KiB	8.2 KiB
⚪	`agents/chat#MessageType`	2.4 KiB	9.0 KiB
🟡	`agents/chat#normalizeToolInput`	2.3 KiB	8.2 KiB
🟡	`agents/chat#parseProtocolMessage`	2.5 KiB	9.0 KiB
🟢	`agents/chat#partAwaitsClientInteraction`	2.4 KiB	8.6 KiB
⚪	`agents/chat#pausedExecutionUpdate`	2.4 KiB	8.4 KiB
🟡	`agents/chat#pendingChatTerminal`	2.3 KiB	8.3 KiB
⚪	`agents/chat#persistReconstructedOrphan`	3.1 KiB	11.0 KiB
🟢	`agents/chat#PreStreamTurns`	2.6 KiB	9.3 KiB
🟡	`agents/chat#readChatRecoveryProgress`	2.3 KiB	8.3 KiB
⚪	`agents/chat#reconcileMessages`	2.8 KiB	9.6 KiB
🟢	`agents/chat#reconcileOrphanPartial`	2.4 KiB	8.5 KiB
🟡	`agents/chat#recordChatTerminal`	2.4 KiB	8.3 KiB
🟡	`agents/chat#repairInterruptedToolParts`	2.6 KiB	9.1 KiB
🟡	`agents/chat#resolveChatRecoveryConfig`	2.6 KiB	8.9 KiB
⚪	`agents/chat#resolveToolMergeId`	2.4 KiB	8.5 KiB
🟡	`agents/chat#ResumableStream`	5.1 KiB	16.8 KiB
🟡	`agents/chat#ResumeHandshake`	3.0 KiB	10.4 KiB
⚪	`agents/chat#ROW_MAX_BYTES`	2.3 KiB	8.2 KiB
⚪	`agents/chat#runChatRecoveryExhaustion`	2.6 KiB	8.9 KiB
⚪	`agents/chat#sanitizeMessage`	2.5 KiB	8.8 KiB
⚪	`agents/chat#sendIfOpen`	2.4 KiB	8.4 KiB
🟢	`agents/chat#setChatRecovering`	2.5 KiB	8.5 KiB
⚪	`agents/chat#shouldCreditStreamProgress`	2.4 KiB	8.3 KiB
⚪	`agents/chat#STREAM_RESUME_NONE_REASONS`	2.3 KiB	8.2 KiB
⚪	`agents/chat#StreamAccumulator`	2.9 KiB	10.7 KiB
🟡	`agents/chat#StreamProgressCreditThrottle`	2.4 KiB	8.3 KiB
⚪	`agents/chat#SubmitConcurrencyController`	2.9 KiB	10.3 KiB
⚪	`agents/chat#sweepStaleChatRecoveryIncidents`	2.4 KiB	8.5 KiB
⚪	`agents/chat#TextSegmentJoiner`	2.7 KiB	9.2 KiB
⚪	`agents/chat#TIMED_OUT`	2.3 KiB	8.2 KiB
⚪	`agents/chat#toolApprovalUpdate`	2.4 KiB	8.5 KiB
⚪	`agents/chat#toolPartHasSettledResult`	2.3 KiB	8.4 KiB
⚪	`agents/chat#toolResultUpdate`	2.4 KiB	8.5 KiB
⚪	`agents/chat#truncateOlderMessages`	3.2 KiB	10.4 KiB
🟡	`agents/chat#TurnQueue`	2.6 KiB	9.2 KiB
⚪	`agents/chat#unwrapChatFiberSnapshot`	2.4 KiB	8.5 KiB
🟡	`agents/chat#wrapChatFiberSnapshot`	2.3 KiB	8.2 KiB
⚪	`agents/chat-sdk#ChatSdkStateAdapter`	261.0 KiB	1141.6 KiB
⚪	`agents/chat-sdk#ChatSdkStateAgent`	260.4 KiB	1139.1 KiB
⚪	`agents/chat-sdk#createChatSdkState`	261.0 KiB	1141.6 KiB
⚪	`agents/chat-sdk#defaultKeyShard`	258.8 KiB	1130.2 KiB
⚪	`agents/chat-sdk#defaultThreadShard`	258.7 KiB	1130.1 KiB
⚪	`agents/chat/react#detectToolsRequiringConfirmation`	3.3 KiB	8.3 KiB
⚪	`agents/chat/react#extractClientToolSchemas`	3.2 KiB	8.3 KiB
⚪	`agents/chat/react#getAgentMessages`	3.4 KiB	8.6 KiB
⚪	`agents/chat/react#getToolApproval`	3.1 KiB	8.0 KiB
⚪	`agents/chat/react#getToolCallId`	3.1 KiB	8.0 KiB
⚪	`agents/chat/react#getToolInput`	3.1 KiB	8.0 KiB
⚪	`agents/chat/react#getToolOutput`	3.1 KiB	8.0 KiB
⚪	`agents/chat/react#getToolPartState`	3.2 KiB	8.2 KiB
⚪	`agents/chat/react#useAgentChat`	132.9 KiB	609.7 KiB
⚪	`agents/chat/react#WebSocketChatTransport`	5.7 KiB	17.1 KiB
⚪	`agents/chat/transport#WebSocketChatTransport`	2.8 KiB	9.2 KiB
⚪	`agents/client#AgentClient`	5.7 KiB	16.6 KiB
⚪	`agents/client#AgentConnectionError`	582 B	993 B
⚪	`agents/client#agentFetch`	4.2 KiB	12.3 KiB
⚪	`agents/client#createStubProxy`	638 B	1.0 KiB
⚪	`agents/client#DEFAULT_CALL_TIMEOUT_MS`	473 B	770 B
⚪	`agents/client#isTerminalCloseEvent`	509 B	822 B
⚪	`agents/context#AgentContextProvider`	412 B	792 B
⚪	`agents/context#AgentSearchProvider`	640 B	1.3 KiB
⚪	`agents/context#ContextBlocks`	87.4 KiB	429.7 KiB
⚪	`agents/email#createAddressBasedEmailResolver`	193 B	227 B
⚪	`agents/email#createCatchAllEmailResolver`	110 B	97 B
⚪	`agents/email#createHeaderBasedEmailResolver`	334 B	492 B
⚪	`agents/email#createSecureReplyEmailResolver`	718 B	1.3 KiB
⚪	`agents/email#DEFAULT_MAX_AGE_SECONDS`	56 B	39 B
⚪	`agents/email#isAutoReplyEmail`	201 B	249 B
⚪	`agents/email#signAgentHeaders`	424 B	812 B
⚪	`agents/experimental/webmcp#registerWebMcp`	85.2 KiB	295.8 KiB
⚪	`agents/lifecycle#getCurrentAgent`	376 B	798 B
⚪	`agents/lifecycle#Lifecycle`	8.3 KiB	25.8 KiB
⚪	`agents/lifecycle#LifecycleCapability`	484 B	975 B
⚪	`agents/mcp#createLegacyMcpHandler`	375.9 KiB	1572.2 KiB
⚪	`agents/mcp#createMcpHandler`	388.2 KiB	1617.4 KiB
⚪	`agents/mcp#DurableObjectEventStore`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#ElicitRequestSchema`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#experimental_createMcpHandler`	376.1 KiB	1572.5 KiB
⚪	`agents/mcp#getMcpAuthContext`	342.5 KiB	1430.8 KiB
⚪	`agents/mcp#MCP_SERVER_ID_MAX_LENGTH`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#McpAgent`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#normalizeServerId`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#RPC_DO_PREFIX`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#RPCClientTransport`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#RPCServerTransport`	342.5 KiB	1430.7 KiB
⚪	`agents/mcp#SSEEdgeClientTransport`	342.6 KiB	1431.0 KiB
⚪	`agents/mcp#StreamableHTTPEdgeClientTransport`	342.6 KiB	1431.0 KiB
⚪	`agents/mcp#WorkerTransport`	345.8 KiB	1447.6 KiB
⚪	`agents/mcp/client#getNamespacedData`	62.9 KiB	240.0 KiB
⚪	`agents/mcp/client#MCP_SERVER_ID_MAX_LENGTH`	62.9 KiB	239.9 KiB
⚪	`agents/mcp/client#MCPClientManager`	158.6 KiB	702.6 KiB
⚪	`agents/mcp/client#normalizeServerId`	63.0 KiB	240.2 KiB
⚪	`agents/mcp/do-oauth-client-provider#DurableObjectOAuthClientProvider`	2.1 KiB	6.6 KiB
⚪	`agents/mcp/server#createMcpHandler`	80.5 KiB	307.2 KiB
⚪	`agents/mcp/server#getMcpAuthContext`	64.0 KiB	245.5 KiB
⚪	`agents/observability#channels`	259 B	549 B
⚪	`agents/observability#genericObservability`	470 B	1.2 KiB
⚪	`agents/observability#subscribe`	324 B	668 B
⚪	`agents/observability/ai#wrapAISDK`	8.8 KiB	30.5 KiB
⚪	`agents/react#_testUtils`	3.8 KiB	9.5 KiB
⚪	`agents/react#useAgent`	10.8 KiB	31.1 KiB
⚪	`agents/react#useAgentToolEvents`	5.6 KiB	16.8 KiB
⚪	`agents/routing#getAgentByName`	795 B	1.7 KiB
⚪	`agents/routing#routeAgentRequest`	1.6 KiB	3.6 KiB
⚪	`agents/routing#RoutedAgents`	2.4 KiB	6.2 KiB
⚪	`agents/schedule#getSchedulePrompt`	85.8 KiB	424.7 KiB
⚪	`agents/schedule#scheduleSchema`	85.3 KiB	423.6 KiB
⚪	`agents/schedule#unstable_getSchedulePrompt`	85.9 KiB	424.9 KiB
⚪	`agents/schedule#unstable_scheduleSchema`	85.3 KiB	423.6 KiB
⚪	`agents/schedules#Scheduler`	6.8 KiB	22.0 KiB
⚪	`agents/schedules/parser#getSchedulePrompt`	85.8 KiB	424.7 KiB
⚪	`agents/schedules/parser#scheduleSchema`	85.3 KiB	423.6 KiB
⚪	`agents/sessions#createCompactFunction`	1.7 KiB	4.0 KiB
⚪	`agents/sessions#Session`	2.0 KiB	6.0 KiB
⚪	`agents/sessions#Sessions`	9.0 KiB	31.8 KiB
⚪	`agents/skills#fromManifest`	309.8 KiB	1084.0 KiB
⚪	`agents/skills#parseSkillFrontmatter`	328.4 KiB	1146.2 KiB
⚪	`agents/skills#parseSkillMarkdown`	328.6 KiB	1146.5 KiB
⚪	`agents/skills#r2`	330.2 KiB	1150.4 KiB
⚪	`agents/skills#runner`	369.0 KiB	1297.8 KiB
⚪	`agents/skills#SkillRegistry`	416.4 KiB	1581.7 KiB
⚪	`agents/skills/compile#compileSkillScript`	15.4 KiB	43.4 KiB
⚪	`agents/skills/compile#isCompilableSkillScript`	15.4 KiB	43.3 KiB
⚪	`agents/streams#DEFAULT_MAX_CHUNK_BYTES`	94 B	96 B
🟡	`agents/streams#sseResponse`	857 B	1.6 KiB
⚪	`agents/streams#StreamClosedError`	175 B	212 B
⚪	`agents/streams#StreamNotFoundError`	215 B	276 B
🟡	`agents/streams#Streams`	4.2 KiB	13.8 KiB
⚪	`agents/streams#StreamSerializationError`	171 B	201 B
⚪	`agents/tasks#DuplicateTaskStepError`	328 B	463 B
⚪	`agents/tasks#MAX_SERIALIZED_BYTES`	190 B	232 B
⚪	`agents/tasks#MissingTaskDefinitionError`	358 B	536 B
⚪	`agents/tasks#NonRetryableError`	238 B	308 B
⚪	`agents/tasks#TaskReplayDivergedError`	341 B	483 B
⚪	`agents/tasks#Tasks`	8.9 KiB	31.8 KiB
⚪	`agents/tasks#TaskSerializationError`	258 B	339 B
⚪	`agents/types#MessageType`	211 B	365 B
⚪	`agents/vite#default`	353.8 KiB	1356.1 KiB
⚪	`agents/websockets#CALLABLES_RPC_QUERY`	12.4 KiB	43.3 KiB
⚪	`agents/websockets#CALLABLES_RPC_VALUE`	12.4 KiB	43.3 KiB
⚪	`agents/websockets#callablesFromDecorated`	12.7 KiB	44.3 KiB
⚪	`agents/websockets#callablesRpcUrl`	12.5 KiB	43.5 KiB
⚪	`agents/websockets#isCallablesRpcUpgrade`	12.4 KiB	43.4 KiB
⚪	`agents/websockets#WebSockets`	17.7 KiB	62.2 KiB
⚪	`agents/workflows#AgentWorkflow`	260.0 KiB	1134.8 KiB
⚪	`agents/workflows#WorkflowRejectedError`	258.7 KiB	1130.2 KiB
⚪	`agents/x402#normalizeNetwork`	14.7 KiB	61.1 KiB
⚪	`agents/x402#withX402`	23.0 KiB	89.2 KiB
⚪	`agents/x402#withX402Client`	104.1 KiB	346.5 KiB

Reported by agent-think[bot].

pkg-pr-new · 2026-09-09T11:19:13Z

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@2223

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@2223

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@2223

hono-agents

npm i https://pkg.pr.new/hono-agents@2223

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@2223

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@2223

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@2223

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@2223

commit: f7a442d

…retry a failed seed Review follow-ups on the derived progress marker. The retire is now a property of deletion rather than of the caller: the Streams sync aperture gains an onDelete hook that every delete path runs with the row and its cursor before removal, and the chat adapter retires chat rows there. A chat stream deleted through the capability's public delete() no longer lowers the marker. The hosts mirror the marker's durable part back to the pre-derivation KV key after each retire and each credit — one put per stream retired, none per chunk — so a build rolled back to the counter never reads a marker lower than an incident recorded under the derived one. A rejected legacy-counter read is no longer cached: the memoized seed promise clears itself on failure so the next evaluation retries instead of failing for the isolate's life.

…construction Both hosts construct the chat stream adapter inside a startup step that the runtime retries after a failed start, and each construction registered another deletion hook on the same Streams capability, so a stream deleted afterwards would have been retired once per construction. The aperture's onDelete now returns its unsubscribe, and the adapter replaces the hook of any adapter constructed earlier on the same capability.

…s rows Nothing calls it: the cutover's discard drops a completed stream's rows in the settle transaction, and reclaim() on the next start() catches the rest, so the per-stream discard has no caller in source, tests or docs.

…e-derive the legacy flag after a rolled-back cutover Think's cutover never passed a discard option, so an agent-tool child turn's stream rows were deleted with its message and a parent attaching after completion replayed nothing. The cutover now keeps the rows for a child turn, decided from the run map as ai-chat already does, and the child's next start() reclaims them. The settle's tail read can fold a v1 stream's rows inside the cutover transaction and, if they were the last, drop the legacy table and clear the in-memory flag. A throwing commit rolled the table back but not the flag, so the restored rows became invisible. The flag is now re-derived from the schema when the transaction leaves it changed.

…ry for transient reader errors Ports upstream cloudflare#2194, cloudflare#2190, cloudflare#2223, cloudflare#2360, cloudflare#2385 and cloudflare#2393 to AGUIChatAgent. - Recovery attempts are __cf_internal_chat_recovery Task runs dispatched to the model handoff; a root agent's chat turn is a Task run (facets stay on fibers). The origin-ids scope moves into the dispatch wrapper. - The progress marker is derived from the stream log; a settled TOOL_CALL_RESULT is flushed as it lands. - A platform-transient reader error routes into bounded recovery like a stall: onChatRecovery is consulted with the live turn's stash data, persist:false drops the partial, a blank new turn is retried, and a running attempt that reschedules leaves the incident to its successor. - A cancel during backoff cancels the queued recovery. - An auto-continuation that fails before streaming sends an error frame and reaches onChatResponse. The agui test worker follows the Tasks transport and the derived marker.

This comment was marked as resolved.

Sign in to view

mattzcarey added 3 commits September 9, 2026 13:03

mattzcarey merged commit dd8bf90 into main Sep 9, 2026
16 checks passed

mattzcarey deleted the perf/recovery-progress-from-streams branch September 9, 2026 13:52

mattzcarey mentioned this pull request Sep 12, 2026

fix(e2e): make the nightly e2e suites green #2256

Merged

ternarybits mentioned this pull request Sep 15, 2026

ChatRecoveryConfig.maxRecoveryWork docs retain old default and omit durable-segment unit #2282

Closed

threepointone mentioned this pull request Sep 23, 2026

docs(chat): document maxRecoveryWork's durable-segment unit and 10000 default (#2282) #2322

Merged

github-actions Bot mentioned this pull request Sep 23, 2026

Version Packages #2335

Merged

Navigation Menu

Repository navigation

perf(chat): derive the recovery progress marker from the stream log - #2223

perf(chat): derive the recovery progress marker from the stream log#2223
mattzcarey merged 5 commits into
mainfrom
perf/recovery-progress-from-streams

mattzcarey commented Sep 9, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Sep 9, 2026 •

edited

Loading

Uh oh!

agent-think Bot commented Sep 9, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

pkg-pr-new Bot commented Sep 9, 2026 •

edited

Loading

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mattzcarey commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Also in this PR

Cost

Upgrade

Tests

Uh oh!

changeset-bot Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

agent-think Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟡 agents import sizes

Uh oh!

This comment was marked as resolved.

Uh oh!

pkg-pr-new Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mattzcarey commented Sep 9, 2026 •

edited

Loading

changeset-bot Bot commented Sep 9, 2026 •

edited

Loading

agent-think Bot commented Sep 9, 2026 •

edited

Loading

pkg-pr-new Bot commented Sep 9, 2026 •

edited

Loading