Skip to content

Surface a live "recovering…" status to chat clients during durable recovery #1620

Closed

Description

@threepointone

Background

Follow-up to the deploy-churn recovery hardening (#1615, #1617, #1618, #1619).

When a chat turn is interrupted (deploy/eviction) and Think recovers it, there is
currently no client-facing "in progress" signal while recovery runs. The
client sees the partial transcript with no indication that the turn is being
recovered — it looks idle/stuck until recovery completes (or, now via #1619,
until a terminal error is replayed on connect).

#1619 fixed the terminal half (failed/exhausted turns are surfaced, and
replayed on reconnect — option "A", plus keeping benign skips silent, "C"). This
issue tracks the remaining progress half ("B"): show clients a
"recovering…" indicator so an interrupted-but-recovering turn reads as working
rather than frozen.

Why it's separate

Unlike #1619 (which reused the existing terminal MSG_CHAT_RESPONSE and needed
no client changes), a "recovering" indicator is a new client-facing
capability
and spans multiple layers:

  1. Protocol: a new chat message type (e.g. MSG_CHAT_RECOVERING) in
    agents/chat, broadcast on recovery detect/schedule and cleared on
    resolution.
  2. Hydration: the status must be durable and replayed on connect (like the
    terminal status in Fix silent/frozen interrupted turns: replay terminal status on reconnect #1619), or a client that connects mid-recovery still sees
    nothing — this is the load-bearing requirement for the deploy/reconnect-storm
    case.
  3. Debounce + reliable clear: recovery is detected many times under churn,
    so the signal must be idempotent (only on transition) and cleared on every
    terminal path (completed/failed/exhausted/skipped) or a spinner spins forever.
  4. Client rendering: useAgentChat (and downstream SPAs) must render the
    new status; needs browser validation.

Acceptance

  • A connected client shows a "recovering…" state for the duration of durable
    chat recovery, and a client that connects mid-recovery learns it on connect.
  • The indicator clears on every terminal outcome; benign skips do not produce a
    stuck indicator.
  • Backward-compatible: clients that don't understand the new signal ignore it.

Notes

Investigation + the A/B/C/D design options are written up in
examples/deploy-churn/INVESTIGATION.md (added in #1618).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions