session-flow/running-retro observer: mtime-idle end detection fires mid-session (default 900s), analyzes a partial transcript, never resumes

[kyle-sexton]

Summary

The session-flow running-retro detached observer declares a session ended after idle_seconds (default 900) of transcript quiet, runs its post-end analysis, and exits — permanently. In an attended session, 15 minutes of the operator reading and typing a long reply tripped it mid-session: the analysis ran against a partial transcript, and the remainder of the session was left unobserved.

Evidence

Status file from the affected run (v0.17.3):

{"cycles": 211, "lines_observed": 44, "idle_seconds": 900.1,
 "last_growth": "2026-07-26T04:01:39Z", "state": "ended-idle"}

Analysis fired at 04:16Z and the ledger checkpoint was written while the session was still active (the session continued for a substantial further stretch, unobserved).

Why this matters

• Attended/interactive sessions routinely have >15-minute human pauses; mtime-idle is a weak proxy for "session ended."
• The failure is silent and terminal: no re-arm, no notice to the session, analysis covers a prefix and presents itself as the session retro.

Suggested directions (any subset)

1. Growth re-check before analysis, and resume on renewed growth: on idle trip, snapshot offset; if the transcript grows during or after analysis, return to watching and supersede/append at the true end rather than exiting.
2. Raise the default for attended contexts (900s is tuned for abandoned sessions, not thinking humans) and surface the observer_idle_seconds config more prominently in the arm output.
3. Prefer a positive end signal where available — a SessionEnd hook event writing a tombstone the observer polls — with mtime-idle as fallback only.

---

Filed by an AI agent during an attended session (operator-directed). Related: #1468 (arm launcher NameError, same component).

[kyle-sexton]

This was generated by AI during triage.

Triage

State: raw → verified → briefed → human-gated

Scope note: this issue opened with priority: medium already set, not the repo's raw marker (priority: needs-triage) — the issue-triage-label workflow only applies the floor when an issue opens with no priority:* label. Per dogfood-filing.md the filer should have passed --priority needs-triage; that's a filing-contract miss, not a completed triage — the item carries no type, no status, no role label, no work-class, so it is treated as raw intake here.

Verified (source inspection of plugins/session-flow/skills/running-retro/scripts/observer.py, _tail_until_idle): confirmed by reading the code, not by reproduction. The loop computes idle_for = time.time() - _to_epoch(last_growth_iso) every poll cycle and on idle_for >= self.idle_secs (default --idle-seconds 900.0) sets state = "ended-idle" and unconditionally breaks out of the tail loop. run() then fires _run_analysis() once and returns — there is no code path that re-enters _tail_until_idle after an idle-triggered break, and no automatic re-arm is spawned. So: a >=900s mtime-idle gap of any origin (network stall, human reading/typing a long reply, etc.) is treated identically to session end, analysis runs over whatever was captured up to that point, and the process exits terminally. The docstring at the top of observer.py acknowledges the tradeoff explicitly ("It cannot distinguish a long single turn from end") but the code has no mitigation beyond "keep --idle-seconds above the longest expected single turn." This matches the reported behavior.

Classification: Bug (broken vs. intent — the ledger checkpoint is written and presented as this session's retro even though coverage silently stops at the idle trip; the reporter's evidence shows exactly this). priority: medium kept as filed — no blocking dependency or external deadline per the triage default rule, though this materially affects the observer's primary use case (attended sessions).

Why human-gated: the issue's own "Suggested directions (any subset)" is a real open design space, not a single well-scoped fix:

1. Growth re-check + resume after an idle trip (offset snapshot, re-arm on renewed growth, supersede/append at the true end) — the most correct fix but touches lifecycle/lock/redaction-boundary assumptions (_resume_offset, _cleanup_observations, lock lifetime vs. --max-seconds) that a scoped brief shouldn't decide unilaterally.
2. Raise the default --idle-seconds for attended contexts and surface observer_idle_seconds more prominently — cheap, but a guess at a new default with no stated basis, and doesn't fix the "never resumes" half.
3. Prefer a positive end signal (e.g. a SessionEnd-hook tombstone) with mtime-idle as fallback — the code's own docstring already gives the rationale against depending on SessionEnd alone ("a SessionEnd hook is not crash-safe"), which is design rationale a human should affirm or revisit rather than an agent silently overriding.

Decision needed: which of the three directions (or what combination/sequencing) should the fix take — a maintainer call, since it changes default behavior and touches the observer's crash-safety model, not a reversible/mechanical toggle.

Cross-reference: #1473's recommendation 3 ("disclose coverage truncation in the ledger itself") is downstream of this issue — whatever end-detection design is chosen here, the ledger should say when it covers a partial session. Not a shared decision carrier (independent root causes), just a note for whoever picks this up.

Related: #1468 (arm launcher NameError, same component, PR #1474 open).

[kyle-sexton]

Frontier disposition (drain session)

Ruling: Promote to agent-ready.

Small scoped bug with a bounded blast radius and enough specification for autonomous pickup. Removed needs-human, added agent-ready.

Re-open trigger: If implementation reveals an unresolved design fork or security policy decision.