fix(fleet): Runs screen loses step, tokens, cost and progress on long/resumed runs (bounded tail window misses agent_started)

[jrob5756]

Summary

On the Fleet Manager Runs screen, a long-running (and especially a resumed) workflow renders as a half-empty row: workflow name and total elapsed are present, but Step, Step-elapsed, Tokens, Cost and the progress bar are all blank — while the same run's detail screen (Enter) shows every one of those values correctly.

 ship        review    1h00    20m    24065K tok   ~$0.95   ████████░░░░  bg  874062
 implement   —         1h03    —      —            —        ░░░░░░░░░░░░  bg  868650

implement is a resumed run. Nothing is wrong with the run itself — it is executing normally, and the run record, event log and dashboard are all healthy.

Reproduction (live data)

$ uv run python -c "
from conductor.fleet.records import read_run_records
from conductor.fleet.summary import derive_run_summary
for r in read_run_records():
    s = derive_run_summary(r)
    print(r.workflow_name, '| status', s.status, '| step', s.current_step,
          '| tokens', s.total_tokens, '| cost', s.total_cost_usd,
          '| topology', None if s.topology is None else len(s.topology.agents))"

workflow  | status running | step review | tokens 24064553 | cost 6.93 | topology 23
implement | status running | step None   | tokens 0        | cost None | topology 10

The run record is fine — event_log_path is set and the file exists:

{"run_id": "3b0acfa3", "pid": 868650, "workflow_name": "implement",
 "event_log_path": "/tmp/conductor/conductor-implement-20260822-171741-3b0acfa3.events.jsonl",
 "port": 45919, "mode": "bg"}

derive_run_detail() on that same record returns full per-agent data, so the log is not the problem:

epic_selector        completed  tokens=6891441
coder                completed  tokens=274582904
epic_reviewer        running    tokens=136855117
...

Root cause

derive_run_summary reads a bounded 512 KB tail of the event log (src/conductor/fleet/summary.py:56, :936). Everything the blank columns need lives in structural events that occur at the start of a step (agent_started) or are spread across the whole run (agent_completed) — not in the trailing tool chatter that fills the window.

Measured across the three live logs, distance of the last root agent_started from EOF:

log	size	generations	last agent_started from EOF	in 512 KB window?
conductor-implement-…-3b0acfa3 (resumed)	9.14 MB	2	1459 KB	❌
conductor-ship-…-68ec8667	1.05 MB	1	479 KB	⚠️ by 33 KB
conductor-ship-…-32e4e0a9	1.65 MB	1	39 KB	✅

The tail window for the implement run contains only tool chatter:

tail events: 1259
[('agent_tool_start', 486), ('agent_tool_complete', 474),
 ('agent_turn_start', 272), ('agent_message', 27)]

No agent_started → current_step is None; no agent_completed → total_tokens == 0, total_cost_usd is None. That single missing value cascades into four visible columns (fleet/tui/screens/runs.py:1314-1321) plus the progress bar, which gates entirely on summary.current_step (runs.py:481, :497).

Resume is the amplifier, not the sole cause. resume_workflow_async deliberately re-opens the original log in append mode (src/conductor/cli/run.py:2770) so a multi-resume session produces one continuous file. That is the right behaviour for log continuity, but it means the tail window competes with every prior generation's history. The ship row above is 33 KB away from breaking on a first run; a resumed run is 3× past the boundary.

This exact failure mode was already found and fixed for topology only. _HEAD_BYTES (summary.py:317) exists precisely because "workflow_started is the first event a run writes, so on any log longer than the tail window it is the one event guaranteed to be outside it — which is why a long-running workflow's step list silently disappeared". The head fallback at summary.py:947 rescues topology / workflow_name / cwd / inputs and nothing else — which is exactly why the screenshot shows a correct workflow name next to blank step/usage columns. The same reasoning applies to agent_started / agent_completed and was never extended to them.

Secondary defect: a resumed run can be reported as failed / completed while it is running

An appended log contains the previous generation's terminal event. In the live log:

offset 4749124  agent_started    epic_reviewer
offset 4778177  workflow_failed  epic_reviewer   <-- generation 0 ends
offset 4778916  workflow_started                 <-- generation 1 (the resume) begins
offset 7649211  agent_started    epic_reviewer

_scan_events (summary.py:663, :667) latches status on workflow_completed / workflow_failed and never resets it. The only reset is for paused (summary.py:581); neither a subsequent workflow_started nor agent_started clears a terminal status. So whenever the tail window straddles a resume boundary, a healthy run is reported as dead:

$ uv run python -c "
from conductor.fleet.summary import _scan_events
print(_scan_events([
  {'type':'workflow_failed','timestamp':1.0,'data':{'agent_name':'a'}},
  {'type':'workflow_started','timestamp':2.0,'data':{'name':'wf','agents':[]}},
  {'type':'agent_started','timestamp':3.0,'data':{'agent_name':'b'}},
]).status)"
failed

This is strictly worse than the blank columns — it is a confident wrong answer, and it would make the run look stoppable/finished in the Runs list.

Tertiary defect: the head fallback reads the pre-resume generation

summary.py:947 takes the first workflow_started in the file. After a resume that is generation 0's. The resume path deliberately writes a fresh workflow_started built from the current workflow YAML (see the resume/checkpoint parity note in AGENTS.md) precisely because the file may have changed between generations — Fleet will never see it. Topology, cwd and inputs shown for a resumed run are the original run's. In the log above both generations happen to match, so it is currently invisible.

Adjacent: the full-log cap is already being exceeded

_DEFAULT_FULL_LOG_MAX_BYTES = 8 MB (summary.py:68) bounds the run-detail and step-detail screens, and truncates the log's trailing history. The implement log is 9.14 MB — the detail screens are already silently dropping the most recent ~1.1 MB. Because generations accumulate in one file, resumed runs reach this cap far sooner than single-generation runs.

fleet/history.py shows the same accumulation: the resumed log surfaces as a single History row with total_tokens=537,391,756 / total_cost_usd=$175.14 summed across both generations, with started_at from generation 0.

Suggested direction

Not prescriptive, but the pieces seem to be:

1. Make the tail window self-sufficient for step/usage, not just topology. Options: extend the tail backwards until the current step's opening event is found (bounded by a hard cap); or track a per-run incremental scan offset across poll ticks so the Runs screen only reads new bytes and keeps accumulated totals — this also removes the "window vs. log size" race entirely and is cheaper on the 2 s poll than today's fixed 512 KB re-read.
2. Reset status on workflow_started in _scan_events — a new generation opening invalidates the prior generation's terminal event. Cheap and independently correct.
3. Anchor the head read on the last workflow_started, not the first, so a resumed run's topology/cwd/inputs reflect the generation actually executing.
4. Re-check _DEFAULT_FULL_LOG_MAX_BYTES now that real logs exceed it, and decide whether History should report per-generation or aggregate totals for a multi-generation log.

Items 2 and 3 are small and self-contained; item 1 is the substantive one.

Environment

• Conductor main @ 3c07139
• Linux (WSL2), Python 3.12
• Reproduced against live ~/.conductor/runs/ records and /tmp/conductor/*.events.jsonl

[jrob5756]

Starting to take a look at this.