diff --git a/AGENTS.md b/AGENTS.md index 00f89648..20f5fbac 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -89,14 +89,15 @@ step-by-step checklist. - `app.py` - Main entry point, defines the Typer application, flat commands (including `guide`, rich_help_panel="Interact"), and the hidden `checkpoints`/`gate-respond` deprecation aliases - `guide.py` - `guide_impl(text, port, token)` behind `conductor guide` (issue #400): resolves the dashboard port via `pid.py::scan_pid_files()` when `--port` is omitted (read-only, matching `app.py::status`'s reasoning), POSTs `{"text": ...}` to `POST /api/guidance`, and maps 403/409/422/connect-error the same way `cli/gate.py::_gate_respond_impl` does. `guide` is a flat top-level command (not a `guide respond` sub-app) since there is exactly one verb - `checkpoint.py` - `checkpoint` group (`checkpoint list`) + shared `_list_checkpoints_impl` (modeled on `registry.py`) - - `gate.py` - `gate` group (`gate respond`) + shared `_gate_respond_impl` (modeled on `registry.py`) + - `gate.py` - `gate` group (`gate respond`) + shared `_gate_respond_impl` (modeled on `registry.py`). Also consumed directly by the Fleet Manager TUI's gate-resolve action (`conductor.fleet.tui.actions.resolve_gate`) — see the fleet bullet below. - `registry.py` - `registry` group (`list` / `add` / `remove` / `set-default` / `update` / `show`) - `plugin.py` - `plugin` group (`list` / `fetch`). Deliberately no `update`: a floating ref self-updates and a pinned one is meant not to. `fetch` existing as a separate verb is what keeps `conductor validate` off the network + - `fleet.py` - `fleet` group (`list`, `prune`). The **one deliberate deviation** from the `checkpoint`/`gate`/`registry` sub-app pattern: `fleet_app` sets `invoke_without_command=True` rather than `no_args_is_help=True`, because the bare `conductor fleet` (no subcommand) launches the interactive Textual TUI (Fleet Manager E7) — the TUI *is* the feature here, not a missing default. Do not "fix" this to match the other three sub-apps. The TUI is behind the optional `tui` extra (`pip install 'conductor-cli[tui]'`); a `TEXTUAL_AVAILABLE` flag (checked only in the bare-invocation callback, mirroring `providers/aca.py`'s `AZURE_IDENTITY_AVAILABLE`) prints an install hint and exits non-zero rather than raising `ImportError` when `textual` isn't installed. `fleet list`/`fleet prune` need no optional dependency. See `src/conductor/fleet/` below for the TUI itself. - `doctor.py` - `doctor` diagnostics rendering (thin presentation layer over `providers/diagnostics.py`) - `run.py` - Workflow execution command with verbose logging helpers - - `bg_runner.py` - Background process forking for `--web-bg` mode. Captures the detached child's stdout/stderr to `$TMPDIR/conductor/conductor---.bg.{stderr,stdout}.log` so silent crashes (uncaught Python exceptions, `faulthandler` dumps) leave a forensic trail — DEVNULL is **not** used for stdout/stderr. Passes `CONDUCTOR_RUN_ID`, `CONDUCTOR_BG_STDERR_LOG`, and `CONDUCTOR_BG_STDOUT_LOG` to the child via env so the child's `EventLogSubscriber` shares a run id with the bg log files and surfaces both paths in `workflow_started` system metadata. Returns a `BackgroundLaunch` dataclass (`url`, `stderr_log`, `stdout_log`, `run_id`, `workflow_started`). The launcher records that same `run_id` and both capture-log paths into the PID file (issue #404), rather than the launcher-invisible `run_id=""`/`log_file=""` defaults `write_pid_file` used to fall back to. `launch_background_resume` adopts the run id from the resolved checkpoint (`_checkpoint_run_id`, mirroring `cli/run.py`'s own checkpoint-resolution precedence) instead of minting a fresh one, so the PID file, `/api/info`, the events JSONL, and the capture-log filenames all agree on one id across a resume; a checkpoint with no usable id falls back to a fresh one. **Two-stage readiness contract** (issue #410): `_finalize_background_launch` no longer trusts a bare TCP connect as "started". Stage one, `_wait_for_server`, checks `proc.poll()` on every iteration of its socket-connect loop (keyword-only `proc` param, so the ~30 existing `patch(..., "_wait_for_server")` test sites are unaffected), so a child that dies before even binding the port (e.g. a `ConfigurationError` from a workflow that fails to load) is reported in well under a second instead of after the full 15s timeout. The PID file is written as soon as the port opens — deliberately *before* stage two, so a run that is initializing slowly is still visible to `conductor status`/`stop` throughout that wait. Stage two, `_wait_for_workflow_start`, polls `GET /api/info` (the same identity endpoint `conductor stop` already uses) for up to `CONDUCTOR_WEB_BG_START_TIMEOUT` seconds (default 30, `0` disables the probe) until the payload carries a `started_at` **key** (not truthiness — it can legitimately be `0`), proving the child's engine actually emitted `workflow_started` rather than just its HTTP server coming up. `StartProbe` enumerates the four outcomes (`STARTED` / `CHILD_EXITED` / `PORT_CONFLICT` / `TIMED_OUT`), mirroring the `Liveness`/`Identity` enum pattern in `cli/pid.py`/`cli/app.py`. A `CHILD_EXITED` with a non-zero code (or `PORT_CONFLICT`, when `/api/info` reports a foreign `pid`) removes the just-written PID file via `remove_pid_file_at` and raises `RuntimeError` with a bounded stderr-log tail (`_tail_log`); a clean exit-0 or a `TIMED_OUT` with the child still alive both return normally — the latter as `workflow_started=False`, which `cli/app.py` surfaces as a "still initializing" note rather than a failure. - - `pid.py` - PID file utilities for tracking/stopping background processes. The PID file records `run_id`, `stderr_log`, and `stdout_log` from the launch (see `bg_runner.py`); a PID file written before this field existed has `run_id` as an empty string and lacks the `stderr_log`/`stdout_log` keys entirely (they didn't exist yet) — both cases surface as JSON `null` via `conductor status --json`. - - `self_run.py` - Answers "is this PID-file entry the run I am executing inside?" for `conductor stop`'s self-exclusion (issue #399: an agent smoke-testing `stop` must not terminate its own workflow). Three signals, first match wins: (1) `CONDUCTOR_RUN_ID` matching the entry's `run_id`; (2) `CONDUCTOR_WEB_BG`/`CONDUCTOR_WEB_PORT` matching the entry's port, but *only* when the entry records no `run_id` (the pre-#411 compatibility path — an entry with a present-but-different id never falls back to this signal); (3) process ancestry (`/proc//status` `PPid:` walk + `os.getsid(0)`), POSIX-only — Windows relies on signals 1–2 alone. `partition_own_run` splits PID-file entries into `others`/`own`; `stop` targets `others` unless `--allow-self` is passed. + - `bg_runner.py` - Background process forking for `--web-bg` mode. Captures the detached child's stdout/stderr to `$TMPDIR/conductor/conductor---.bg.{stderr,stdout}.log` so silent crashes (uncaught Python exceptions, `faulthandler` dumps) leave a forensic trail — DEVNULL is **not** used for stdout/stderr. Passes `CONDUCTOR_RUN_ID`, `CONDUCTOR_BG_STDERR_LOG`, and `CONDUCTOR_BG_STDOUT_LOG` to the child via env so the child's `EventLogSubscriber` shares a run id with the bg log files and surfaces both paths in `workflow_started` system metadata. Returns a `BackgroundLaunch` dataclass (`url`, `stderr_log`, `stdout_log`, `run_id`, `workflow_started`, `still_running`). **The launch health gate is a run-record poll, not a PID write** (Fleet Manager D2): `_finalize_background_launch` waits for the dashboard to become reachable and then polls `conductor.fleet.records.read_run_record(run_id)` until the **child** has written its own record (matched on `pid`/`mode`/`port`) before returning success, terminating the child if it never does. The parent no longer writes a `.pid` file itself — `write_pid_file` (`cli/pid.py`) was removed as part of this change, since that was its only call site. Do not restore parent-side PID writing; a future bg-launch change belongs in this poll gate, not a reinstated `write_pid_file`. `launch_background_resume` adopts the run id from the resolved checkpoint (`_peek_resume_run_id`) instead of minting a fresh one, so the run record, `/api/info`, the events JSONL, and the capture-log filenames all agree on one id across a resume. **Readiness is three-staged.** Stage one, `_wait_for_server`, checks `proc.poll()` on every iteration of its socket-connect loop (keyword-only `proc` param), so a child that dies before binding the port is reported in well under a second instead of after the full 15s timeout. Stage one-and-a-half is the run-record poll above — a *stronger* signal than the `write_pid_file` it replaced, since the child only writes the record once it is executing. Stage two, `_wait_for_workflow_start`, polls `GET /api/info` for up to `CONDUCTOR_WEB_BG_START_TIMEOUT` seconds (default 30, `0` disables the probe) until the payload carries a `started_at` **key** (not truthiness — it can legitimately be `0`), proving the engine actually emitted `workflow_started` rather than just its HTTP server coming up (issue #410). `StartProbe` enumerates the four outcomes (`STARTED` / `CHILD_EXITED` / `PORT_CONFLICT` / `TIMED_OUT`), mirroring the `Liveness`/`Identity` enum pattern in `cli/pid.py`/`cli/app.py`. A `CHILD_EXITED` with a non-zero code (or `PORT_CONFLICT`, when `/api/info` reports a foreign `pid`) removes the child's run record via `_remove_dead_child_record` (identity-checked on `pid`, since a resumed launch can carry a checkpoint's original `run_id`) and raises `RuntimeError` with a bounded stderr-log tail (`_tail_log`); a clean exit-0 or a `TIMED_OUT` with the child still alive both return normally — the latter as `workflow_started=False`, which `cli/app.py` surfaces as a "still initializing" note rather than a failure. `still_running` is re-polled after the gate returns so a sub-second run is never advertised with a live dashboard URL. + - `pid.py` - **Legacy** PID file utilities, retained only so a still-running pre-upgrade background process (one that wrote its `.pid` file before Fleet Manager D2 removed `write_pid_file`) can still be discovered and cleaned up by `stop`/`fleet list`. Every current run path uses `conductor.fleet.records` (`run_id`-keyed JSON records) instead — see `fleet/records.py` below. + - `self_run.py` - Answers "is this run record the run I am executing inside?" for `conductor stop`'s self-exclusion (issue #399: an agent smoke-testing `stop` must not terminate its own workflow). Three signals, first match wins: (1) `CONDUCTOR_RUN_ID` **or** `CONDUCTOR_SELF_RUN_ID` matching the record's `run_id` — the former is set on a `--web-bg` child, the latter is exported by *every* run (`cli/run.py`) into its own environment so descendants inherit it, which is the only working signal for a foreground run off Linux (no port for signal 2, no `/proc` for signal 3); the two are separate names because `engine/event_log.py` reads `CONDUCTOR_RUN_ID` as "adopt this run id", which a nested `conductor run` must not do; (2) `CONDUCTOR_WEB_BG`/`CONDUCTOR_WEB_PORT` matching the record's port, but *only* when the record has no `run_id` (the pre-#411 compatibility path — a record with a present-but-different id never falls back to this signal); (3) process ancestry (`/proc//status` `PPid:` walk + `os.getsid(0)`), POSIX-only — Windows relies on signals 1–2 alone. `partition_own_run` splits run records into `others`/`own`; `stop` targets `others` unless `--allow-self` is passed. Since the Fleet Manager it operates on `RunRecord`s rather than PID-file dicts (`read_run_records()` already surfaces legacy `.pid` files in that shape), and `reasons` is keyed by PID rather than port because a foreground run has no port. - `update.py` - Update check and version comparison. Upgrades are delegated to the install script (`install.ps1`/`install.sh`); in-process self-upgrade was removed because on Windows the running Python interpreter sits inside the venv `uv tool install --force` is trying to recreate, which fails with "Access is denied". `conductor update` prints the OS-appropriate install-script one-liner; `conductor update --apply` spawns the installer detached (Windows: new console window; POSIX: `os.execvpe` replace) and exits the current process so file locks release. The startup hint is suppressed by `CONDUCTOR_NO_UPDATE_CHECK=1`, `--silent`, `--help`/`--version`, and the `update` subcommand itself. - **config/**: YAML loading and Pydantic schema validation @@ -159,6 +160,14 @@ step-by-step checklist. - **interrupt/**: Interactive workflow interruption (Esc/Ctrl+G to pause) - `listener.py` - Keyboard listener daemon thread for Esc/Ctrl+G detection +- **fleet/**: The Fleet Manager (`conductor stop` run-record scope, `conductor fleet`) — see `docs/fleet.md` for the user-facing guide. Fixes the bug where a plain `conductor run` (no `--web-bg`) was invisible to `conductor stop`/discovery: every run path now writes a `run_id`-keyed JSON record (not the legacy port-keyed `.pid` file `cli/pid.py` used to write) describing its mode/PID/workflow/port, so foreground, `--web-bg`, and `--web` runs are all discoverable the same way. + - `records.py` - `RunRecord` (nine fields, no more — a tenth `tty` field was considered and rejected as POSIX-only with `pid` already sufficient; `mode` is a `RunMode = Literal["fg", "fg-web", "bg"]` so the single write site is checked by `ty`, and an *unrecognised* mode read from disk normalises to `"bg"` rather than raising — a raise reaches `_read_and_prune` as `corrupt`, which deletes **without checking liveness**, so a newer Conductor's mode would make an older one delete a live run's record) + `write_run_record` / `read_run_records` / `read_run_record` / `remove_run_record` / `remove_run_record_for_current_process`. `read_run_records()` filters to processes that pass `cli.pid.is_process_alive` and tolerates legacy port-keyed `.pid` files (surfaced as `mode="bg"` records) alongside corrupt/partial JSON. + - `summary.py` - Derives a `RunSummary` (status/current-step/elapsed/tokens/cost/gate) from a **bounded tail** read of a run's JSONL event log (cheap enough to call once per row on the Runs screen's ~2s poll) plus a separate bounded **full**-log read (capped at 8 MiB from the start of the file) for the run-detail screen's per-agent history — an unusually large log's trailing history can fall outside that cap and be omitted, the same accepted trade-off the bounded tail read makes at the other end. Status is derived from explicit event markers (`gate_presented`/`gate_resolved`/`workflow_completed`/`workflow_failed`), never inferred from timing alone — the design's own measurement found that unreliable (228 false positives, 0 true positives). `gate_resolvable` (Fleet Manager D4) is computed once here: `True` whenever the record has a dashboard port (`fg-web`/`bg`), `False` for `mode == "fg"`. + - `history.py` - Enumerates every retained run directly from `$TMPDIR/conductor/*.events.jsonl` files (not run records, which are already gone by the time a run is history) for the History screen, regardless of outcome. Classifies each log by its terminal event (`workflow_completed`/`workflow_failed`); a log with neither is `"unknown"`, **never** `"running"` — the same non-inference constraint `summary.py` follows, sharpened here since there is no run record to fall back on at all. The returned list is bounded by `[fleet.retention].keep_last` and, independently, by a fixed 200-entry display cap (so a `keep_last` configured below 1 — "unbounded" for the pruning sweep — still cannot grow the History screen without limit). + - `launch.py` - Resolves a file path or registry reference (reusing `registry/resolver.py::resolve_ref` + `registry/cache.py::resolve_and_fetch`, the same pair `conductor show` uses) and calls `cli.bg_runner.launch_background()` directly for the New Run screen — never re-implements detached process spawning, per the design's explicit warning that doing so would make a TUI-launched run die with the TUI. + - `retention.py` - `prune_event_logs(keep_last, dry_run)` bounds `$TMPDIR/conductor/*.events.jsonl` (Fleet Manager D3), mirroring `CheckpointManager.rotate_periodic_checkpoints`'s `keep_last` vocabulary. Never deletes the `checkpoints/` subdirectory or an event log a live/resuming run still references; a retained/live log's `.bg.stderr.log`/`.bg.stdout.log` companions are kept or pruned alongside it. `maybe_prune_event_logs()` is the opportunistic-startup-sweep wrapper `cli/run.py` calls, gated by `[fleet.retention].enabled` and never raising. + - `tui/` - The Textual app (`app.py`, `Screen` push/pop stack) and its screens (`screens/runs.py` home; `screens/run_detail.py`; `screens/step_detail.py`; `screens/providers.py`; `screens/registries.py`; `screens/new_run.py`; `screens/history.py`; `screens/splash.py`), plus `actions.py` (shared stop/kill/gate-respond logic, reusing `cli/app.py::stop_records` and `cli/gate.py::_gate_respond_impl` rather than duplicating either), `theme.py` (the single status glyph/label/colour vocabulary — screens must not define local glyph maps), `dag.py` (step-status chips), `anim.py` (pure frame→glyph functions, disabled by `CONDUCTOR_FLEET_NO_ANIM`), `art.py`, `widgets.py`, and `notify.py` (terminal bell / OSC 9 notifications, debounced to fire once per status transition). Optional: imported only by `conductor fleet`'s bare invocation, gated behind the `tui` extra (`pip install 'conductor-cli[tui]'`, floor `textual>=8.0` — `widgets.py::BlockFooter` is written against 8.x `Footer` group rendering). + - **web/**: Real-time web dashboard for workflow visualization - `auth.py` - `OriginHostGuard` (issue #397): a pure-ASGI middleware (not Starlette's `BaseHTTPMiddleware` / `@app.middleware("http")`, neither of which ever sees WebSocket scopes — a decorator-style middleware would leave `/ws` completely unguarded) registered via `app.add_middleware(...)` on both `server.py` and `replay.py`. Enforces `Host`/`Origin` validation on every `http`/`websocket` scope (a present `Origin` must match; an *absent* one is allowed, since httpx/curl/`conductor gate respond` send none), then token auth + a JSON `Content-Type` on mutating HTTP routes, then token auth on the `/ws` handshake — closing it via `websocket.close` in reply to `websocket.connect`, before `accept()`, so a rejected socket can never send any message type (`gate_response`, `dialog_message`, `dialog_decline`, `iteration_limit_response` all included). `CONDUCTOR_WEB_ALLOW_ORIGINS` (comma-separated full origins) extends the allowlist for a dev server (e.g. Vite's `http://localhost:5173`) without disabling the check for anything else. A per-run token is minted automatically (`mint_token()`) so the protected configuration is the default; `resolve_expected_token(minted)` lets `CONDUCTOR_GATE_TOKEN` override it, preserving the pre-#397 escape hatch. `write_token_file`/`read_token_file`/`remove_token_file` persist that token at `~/.conductor/runs/dashboard-.token` (mode `0600` on POSIX, atomic temp-file + `os.replace`; on Windows the mode bits are not honoured, so the file is protected by the user-profile NTFS ACL instead, issue #425) so a separate CLI invocation (`conductor gate respond`, `conductor guide`, `conductor stop`'s graceful-kill rung) can discover it without a flag or env var; `resolve_cli_token(port, token)` is the shared `--token` > `CONDUCTOR_GATE_TOKEN` > token-file resolver all three call. `token_from_scope(scope)` reads `Authorization: Bearer` first, then the `token` query param — the latter exists only because a browser cannot set handshake headers, and is an acceptable exposure only because both dashboard apps run uvicorn with `log_level="warning"` (no access logs to leak the query string into). - `server.py` - FastAPI + uvicorn server with WebSocket broadcasting, late-joiner state replay, and `POST /api/stop` + `POST /api/kill` endpoints. `/api/stop` interrupts/pauses the current agent (a user can then Resume or Kill); if it arrives before the engine binds its interrupt event, it is latched via `_pending_stop` and drained by `set_interrupt_event` so the startup window takes the graceful pause path instead of a progress-losing hard stop. `/api/kill` hard-stops the run. Whenever a stop/kill actually *terminates* the run (cancels the engine task), it routes through `handle_dashboard_stop` so a best-effort checkpoint is written (or its absence explained) — see `handle_dashboard_stop` above (issue #245). `start()` mints and writes the run's token file once `_actual_port` resolves (works for both `--web` and `--web-bg`, since `cli/run.py` calls `start()`/`stop()` on both the run and resume paths and the `--web-bg` child goes through the same code); `stop()` removes it. `GET /` reads `static/index.html` and injects `