Stale-turn detector: tell the parent when a worker stops streaming #62

Outcome

A worker turn that stops streaming is detected within seconds of its measured healthy limit, and the agent that started it is told. No working turn is ever flagged. Today nothing reads the stream clock from #56: a spawn_thread child can sit silent indefinitely and nobody notices.

Contribution to the Objective: agents run unattended; a dead worker must wake its parent, never wait for the human.

Elon record

Wanted result: exact-second stale detection at ~99.999% certainty, delivered to the parent (dispatcher or planner), for every running worker turn.

Evidence:

• feat(server): Prism stream clock for worker liveness, plus prism_cancel #56 is live on the installed build (checked 2026-09-28: /api/prism/liveness returns lastStreamAt, silenceMs, openTool, turn).
• Healthy Muse reasoning gaps top out around 18 s; Router's interim windows (model-router#135) are 90-300 s per driver and blind to reasoning in the T3 path.
• A 99 s macOS clamshell sleep was once counted as silence and killed a probably healthy turn (model-router#132).
• spawn_thread children have no watcher at all.

Cuts:

• Two detectors (one in Router, one in T3). One detector lives in T3, which already holds the clock and the per-turn stats; Router and the parent notification both consume it. This revises feat(server): Prism stream clock for worker liveness, plus prism_cancel #56's "no stall decisions in T3": the evidence is that spawn_thread children have no Router.
• Fixed timeouts as the rule.
• A separate watcher process or service.

Smallest surviving solution:

1. In the server, a detector that ticks every 1 s over running turns (in memory, alongside streamClock.ts).
2. Suspect when: no open tool, and silence > threshold for the turn's provider+model and phase (before first token vs mid-stream). Threshold = max healthy gap observed in recorded prism.stream-stats activities × margin (1.5), with a conservative default until at least 20 samples exist (mid-stream 90 s, first token 120 s). Never below the largest healthy gap seen.
3. Host sleep never counts as silence: detect wall-clock jumps against a monotonic clock (or power suspend events) and restart the silence window on wake.
4. Confirm: hold a further 25% of the threshold; any new event clears it. Also check that the provider session/process is still alive; a dead process or a provider error is stale immediately (no wait).
5. Expose on the existing liveness route: stale (boolean), staleSince (ISO or null), thresholdMs, thresholdSource (measured | default), reason. Contract change in packages/contracts/src/prism.ts.
6. For spawn_thread children (sub.<parent>.<suffix>), post one message into the parent thread when a turn becomes stale: child thread, model, last event time and kind, silence, threshold. One message per stale turn; no retry or cancel from T3.
7. Router jobs are handled by Router (model-router#132, which consumes stale); do not also message for Router-managed threads if Router marks them (skip when the thread belongs to a Prism job, determinable from existing Prism thread metadata; if not determinable cheaply, message anyway and note it).

Acceptance criteria

• Tests with fake streams: healthy long reasoning with 18 s gaps never flags; silence past threshold flags within threshold + confirm + 1 s; open tool never flags; simulated sleep never flags; dead provider flags immediately; thresholds switch from default to measured at 20 samples.
• Liveness route returns the new fields; contract test updated.
• Parent receives exactly one message for a stale child.
• Fork rules: new files first; any upstream edit allowlisted in scripts/fork-upstream-edits.txt and mapped in docs/fork-features.md; scripts/fork-check.sh --upstream upstream passes.

Non-goals

Cancelling or retrying from T3; a UI; changing the stats recorded by #56.

Blockers

None (#56 merged and installed).

Proof

Server tests for src/prism and src/mcp, fork-check.