diff --git a/CHANGELOG.md b/CHANGELOG.md index 7aed5c47..7902871d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,9 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased](https://github.com/microsoft/conductor/compare/v0.1.10...HEAD) +## [Unreleased](https://github.com/microsoft/conductor/compare/v0.1.11...HEAD) + +## [0.1.11](https://github.com/microsoft/conductor/compare/v0.1.10...v0.1.11) - 2026-05-04 ### Added - `metadata` dict on workflow definitions, settable statically in YAML or @@ -23,6 +25,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Self-referential sub-workflows are now allowed; depth is bounded by the global `MAX_SUBWORKFLOW_DEPTH` plus an optional per-agent `max_depth` field on `AgentDef` ([#111](https://github.com/microsoft/conductor/pull/111)). +- `workflow.dir`, `workflow.file`, and `workflow.name` template variables are + now available in all agent contexts (regardless of context mode). Lets + registry-hosted workflows reference co-located scripts and assets without + depending on the caller's working directory + ([#121](https://github.com/microsoft/conductor/pull/121)). +- Script agent stdout that is valid JSON is auto-parsed and merged into + the agent's output dict alongside `stdout`, `stderr`, and `exit_code`, + enabling field-based `when:` route conditions instead of opaque exit-code + matching ([#122](https://github.com/microsoft/conductor/pull/122)). +- `conductor validate` now performs semantic validation in addition to + YAML schema checks, catching stale agent references, missing workflow + inputs, and undeclared explicit-mode dependencies before runtime in + `prompt`, `system_prompt`, `command`, `args`, `working_dir`, + `input_mapping`, parallel-group inputs, and workflow `output:` + templates ([#125](https://github.com/microsoft/conductor/pull/125)). +- Web dashboard: breadcrumb navigation, double-click dive-in to + sub-workflow graphs, isolated subworkflow contexts (no node-status + bleed across repeated runs), and reliable Stop button during + subworkflows ([#113](https://github.com/microsoft/conductor/pull/113), + follow-up fixes in [#146](https://github.com/microsoft/conductor/pull/146)). +- Dialog mode for agents: multi-turn conversational interactions + driven by a `dialog` gate with conditional transitions, full + Copilot and Claude provider support, and dedicated dashboard UI + (`DialogDetail`, `DialogEngagementPrompt`, `DialogOverlay`) + ([#130](https://github.com/microsoft/conductor/pull/130)). +- Markdown rendering and auto-linkification in human gate prompts. + Gate prompts render through Rich Markdown in the terminal and as + GitHub-Flavored Markdown in the dashboard. Bare file paths and URLs + in gate prompts are converted to clickable links; relative paths + open a sandboxed `FileViewer` modal served via a path-traversal-safe + `GET /api/files/{path}` endpoint + ([#131](https://github.com/microsoft/conductor/pull/131)). +- Workspace instructions support: `--workspace-instructions` and + `--instructions` CLI flags plus a YAML-level `instructions:` field on + the workflow. Auto-discovers `AGENTS.md`, `CLAUDE.md`, and + `.github/copilot-instructions.md` by walking from CWD to the git root, + prepends them to every agent's prompt, inherits into sub-workflows, + and persists in checkpoints + ([#141](https://github.com/microsoft/conductor/pull/141)). ### Changed - The dashboard's "context window remaining" bar now sources @@ -62,6 +103,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 and `install.sh` so `conductor` is available on PATH in new shells, CI agents, and IDE extensions after a fresh install ([#142](https://github.com/microsoft/conductor/pull/142)). +- In explicit context mode, `workflow.input` is now always available to + `script` and `type: workflow` agent templates regardless of the agent's + declared `input:` list. The explicit-mode contract still applies to LLM + agents (no undeclared inputs in prompts to control token cost) + ([#119](https://github.com/microsoft/conductor/pull/119)). +- Optional workflow inputs without an explicit `default:` now resolve to + type-appropriate zero values (`""`, `0`, `false`, `[]`, `{}`) instead of + Python `None`, so templates like + `{{ workflow.input.optional | default("fallback") }}` render the fallback + rather than the literal string `"None"` + ([#123](https://github.com/microsoft/conductor/pull/123)). +- Web dashboard: events without an engine-supplied `subworkflow_path` + stamp (e.g., `for_each_item_started` for a parent for_each over + `type: workflow` agents) now route strictly to the root context + instead of falling back to the user's currently-viewed path. This + fixes two related symptoms: dashboards opened during a run with + sub-workflows no longer auto-land inside an iteration, and a parent + for_each panel now displays every iteration rather than silently + dropping the middle ones into a sibling sub-workflow's context + ([#148](https://github.com/microsoft/conductor/pull/148)). ## [0.1.10](https://github.com/microsoft/conductor/compare/v0.1.9...v0.1.10) - 2026-04-30 diff --git a/README.md b/README.md index 97d430ae..8f089fc4 100644 --- a/README.md +++ b/README.md @@ -16,12 +16,15 @@ Conductor provides the patterns that work: evaluator-optimizer loops for iterati - **YAML-based workflows** - Define multi-agent workflows in readable YAML - **Multiple providers** - GitHub Copilot or Anthropic Claude with seamless switching - **Parallel execution** - Run agents concurrently (static groups or dynamic for-each) -- **Script steps** - Run shell commands and route on exit code without an AI agent +- **Sub-workflow composition** - Reusable sub-workflows with templated `input_mapping`, usable inside `for_each` groups for dynamic fan-out +- **Script steps** - Run shell commands and route on exit code or parsed JSON stdout +- **Dialog mode** - Agents can pause for multi-turn conversation when uncertain +- **Workspace instructions** - Auto-discover and inject `AGENTS.md` / `CLAUDE.md` / `.github/copilot-instructions.md` into every agent's prompt - **Conditional routing** - Route between agents based on output conditions -- **Human-in-the-loop** - Pause for human decisions with Rich terminal UI +- **Human-in-the-loop** - Pause for human decisions with Markdown-rendered prompts and clickable file links - **Safety limits** - Max iterations and timeout enforcement -- **[Web dashboard](#web-dashboard)** - Real-time workflow visualization with interactive DAG graph, live streaming, and in-browser human gates -- **Validation** - Validate workflows before execution +- **[Web dashboard](#web-dashboard)** - Real-time workflow visualization with interactive DAG graph, breadcrumb navigation into sub-workflows, live streaming, and in-browser human gates +- **Validation** - Catches stale template references, missing inputs, and undeclared dependencies before runtime ## Installation @@ -189,6 +192,9 @@ conductor run [OPTIONS] | Option | Description | |--------|-------------| | `-i, --input NAME=VALUE` | Workflow input (repeatable) | +| `-m, --metadata KEY=VALUE` | Workflow metadata (repeatable; surfaced in `workflow_started`) | +| `--workspace-instructions` | Auto-discover `AGENTS.md` / `CLAUDE.md` / `.github/copilot-instructions.md` and prepend to every agent prompt | +| `--instructions PATH` | Explicit instructions file (repeatable) | | `-p, --provider PROVIDER` | Override provider | | `--dry-run` | Preview execution plan | | `--skip-gates` | Auto-select at human gates | diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 238deee9..7d67aab4 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -23,6 +23,9 @@ conductor run [OPTIONS] |--------|-------|-------------| | `--input NAME=VALUE` | `-i` | Workflow input (repeatable) | | `--input.NAME=VALUE` | | Alternative input syntax | +| `--metadata KEY=VALUE` | `-m` | Workflow metadata (repeatable). Merged on top of YAML `metadata:` and surfaced in the `workflow_started` event. | +| `--workspace-instructions` | | Auto-discover convention files (`AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`) by walking from CWD up to the git root. Concatenated and prepended to every agent's prompt. | +| `--instructions PATH` | | Explicit path to an instructions file (repeatable). Combines with auto-discovered files when both flags are used. | | `--provider PROVIDER` | `-p` | Override provider (copilot, claude) | | `--dry-run` | | Show execution plan without running | | `--skip-gates` | | Auto-select first option at human gates | @@ -107,6 +110,19 @@ conductor run workflow.yaml --skip-gates conductor run workflow.yaml --silent --log-file auto --skip-gates --input question="Automated test" ``` +#### Metadata and Instructions + +```bash +# Inject runtime metadata (visible in the workflow_started event) +conductor run twig-sdlc.yaml --metadata work_item_id=1814 --metadata env=staging + +# Auto-discover and inject AGENTS.md / CLAUDE.md / copilot-instructions.md +conductor run workflow.yaml --workspace-instructions + +# Combine auto-discovery with an explicit extra file +conductor run workflow.yaml --workspace-instructions --instructions ./style-guide.md +``` + #### Complex Inputs ```bash @@ -160,7 +176,7 @@ conductor stop --all ## `conductor validate` -Validate a workflow file without executing it. Checks YAML syntax, schema compliance, and cross-references (agent names, routes, parallel groups). +Validate a workflow file without executing it. Checks YAML syntax, schema compliance, cross-references (agent names, routes, parallel groups), and Jinja2 template references throughout the workflow. ```bash conductor validate @@ -181,6 +197,7 @@ for f in examples/*.yaml; do conductor validate "$f"; done ### Validation Checks +**Errors** (validation fails): - YAML syntax errors - Schema compliance (required fields, types) - Agent name references in routes @@ -188,6 +205,12 @@ for f in examples/*.yaml; do conductor validate "$f"; done - For-each source references - Circular dependency detection - Input/output schema validation +- **Stale agent references in templates** — `{{ old_agent.output.field }}` where `old_agent` doesn't exist +- **Missing workflow input references** — `{{ workflow.input.x }}` where `x` isn't declared in `input:` +- Stale references checked across `prompt`, `system_prompt`, `command`, `args`, `working_dir`, `input_mapping`, parallel-group inputs, and workflow `output:` templates + +**Warnings** (validation passes with notes): +- **Undeclared dependencies in explicit mode** — agent prompt references `{{ a.output.val }}` but doesn't declare `a.output` in its `input:` list ## `conductor registry` diff --git a/docs/providers/claude.md b/docs/providers/claude.md index ef2a3fa1..43d443ff 100644 --- a/docs/providers/claude.md +++ b/docs/providers/claude.md @@ -97,7 +97,7 @@ echo '.env' >> .gitignore ## Model Selection -Claude offers multiple model tiers optimized for different use cases. All models support a 200K token context window. +Claude offers multiple model tiers optimized for different use cases. All current Claude models default to a 200K-token context window; the dashboard's "context remaining" bar sources this value from the Anthropic SDK at runtime, so it always reflects the actual cap your account has access to (rather than a hand-maintained number that can drift). Beta context modes such as Claude's 1M-token window are not enabled by default in conductor today. ### Available Models diff --git a/docs/providers/comparison.md b/docs/providers/comparison.md index 7742c80e..a75f5e0f 100644 --- a/docs/providers/comparison.md +++ b/docs/providers/comparison.md @@ -6,7 +6,7 @@ This guide helps you choose between GitHub Copilot and Anthropic Claude provider | Feature | Copilot | Claude | Winner | |---------|---------|--------|--------| -| **Context Window** | 8K-128K | 200K (all models) | Claude | +| **Context Window** | per-model (SDK-reported) | per-model (SDK-reported) | Tie | | **Pricing Model** | Subscription ($10-39/mo) | Pay-per-token | Depends | | **Setup** | GitHub auth | API key | Copilot (easier) | | **Model Selection** | GPT-5.2, o1 | Haiku, Sonnet, Opus | Tie | diff --git a/docs/workflow-syntax.md b/docs/workflow-syntax.md index f17c3f86..9b3ad87e 100644 --- a/docs/workflow-syntax.md +++ b/docs/workflow-syntax.md @@ -23,11 +23,20 @@ workflow: name: string # Required: Unique workflow identifier description: string # Optional: Human-readable description entry_point: string # Required: Name of first agent to execute - + + metadata: # Optional: free-form key/value metadata + tracker: ado # surfaced in the workflow_started event + project_url: https://... # CLI --metadata / -m can add or override + + instructions: # Optional: extra instruction files (paths) + - ./docs/conventions.md # prepended to every agent prompt + - ./AGENTS.md # also auto-discoverable via + # --workspace-instructions (see CLI ref) + limits: max_iterations: 10 # Default: 10, max: 500 timeout_seconds: 600 # Optional: Maximum wall-clock time (seconds) - + hooks: on_start: "{{ template }}" # Optional: Expression evaluated on start on_complete: "{{ template }}" # Optional: Expression evaluated on success @@ -36,6 +45,10 @@ workflow: context_mode: accumulate # accumulate | snapshot | minimal (default: accumulate) ``` +**Workflow metadata** is included verbatim in the `workflow_started` event and lets downstream consumers (dashboards, queue runners, observability tools) adapt without parsing the YAML. CLI `--metadata key=value` flags merge on top of YAML metadata (CLI wins on conflicts). + +**Instructions files** are loaded once and prepended to every agent's rendered prompt. They are inherited by sub-workflows and persisted in checkpoints so resume continues to use the same instructions. Use the YAML `instructions:` list for workflow-pinned context, or pass `--workspace-instructions` on the CLI to auto-discover `AGENTS.md`, `CLAUDE.md`, and `.github/copilot-instructions.md` by walking from CWD up to the git root. + ### Context Modes - **`accumulate`** (default): Agents see all previous agent outputs @@ -172,6 +185,25 @@ agents: | `stderr` | string | Captured standard error | | `exit_code` | integer | Process exit code (0 = success) | +**JSON stdout auto-parsing** — if `stdout` is valid JSON _and_ the parsed value is an object, its fields are merged into the agent's output dict alongside `stdout`/`stderr`/`exit_code`. This lets you route on parsed fields directly instead of opaque exit codes: + +```yaml +# Script writes to stdout: {"route": "planning", "issue_count": 3} +agents: + - name: detector + type: script + command: pwsh + args: ["-File", "{{ workflow.dir }}/scripts/detect.ps1"] + routes: + - to: planner + when: "route == 'planning'" # parsed field + - to: scaler + when: "issue_count > 100" # parsed field + - to: $end +``` + +JSON arrays and scalars are ignored (only objects merge). Non-JSON stdout is unchanged. Parsed fields shadow `stdout`/`stderr`/`exit_code` if a script outputs those as JSON keys. + Access in downstream agents: ```yaml @@ -207,6 +239,12 @@ agents: workflow: ./research-pipeline.yaml # Required: path to sub-workflow YAML input: # Optional: explicit input declarations - workflow.input.topic + input_mapping: # Optional: per-call inputs to the sub-workflow + topic: "{{ workflow.input.topic }}" + depth: "{{ research_planner.output.depth }}" + max_depth: 3 # Optional: per-agent recursion cap + # (additionally bounded by global + # MAX_SUBWORKFLOW_DEPTH = 10) output: # Optional: output schema for validation findings: type: string @@ -219,8 +257,9 @@ agents: - The `workflow` path is resolved relative to the parent workflow file - Sub-workflow inherits the parent's provider configuration - Sub-workflow output is stored in context and accessible via `{{ agent_name.output.field }}` -- Recursive composition is supported (sub-workflows can reference other sub-workflows) with a depth limit of 10 -- Circular references (a workflow referencing itself) are detected and rejected +- Recursive composition is supported (sub-workflows can reference other sub-workflows) with a global depth limit of `MAX_SUBWORKFLOW_DEPTH = 10` +- Self-referential sub-workflows (a workflow referencing itself) are allowed; depth is bounded by the global cap and the optional per-agent `max_depth` field +- `input_mapping` keys are sub-workflow input names; each value is a Jinja2 expression evaluated against the parent's context. When `input_mapping` is omitted, the parent's `workflow.input.*` is forwarded to the sub-workflow as before **Access sub-workflow output in downstream agents:** @@ -230,7 +269,24 @@ prompt: | {{ deep_research.output.findings }} ``` -**Restrictions** — workflow steps cannot have `prompt`, `model`, `provider`, `tools`, `system_prompt`, `command`, or `options`. Workflow steps also cannot be used inside `parallel` groups or `for_each` groups. +**Sub-workflows in `for_each` groups** — `type: workflow` agents can be used inside `for_each` groups to fan out one sub-workflow run per item in the source array. Each iteration receives its own `input_mapping` evaluated against the loop variable, and emits its own `subworkflow_started` / `subworkflow_completed` events: + +```yaml +parallel: + - name: plan_issues + for_each: + source: epic_planner.output.issues + as: issue + max_concurrent: 1 + agent: + type: workflow + workflow: ./plan-and-review.yaml + input_mapping: + work_item_id: "{{ issue.id }}" + title: "{{ issue.title }}" +``` + +**Restrictions** — workflow steps cannot have `prompt`, `model`, `provider`, `tools`, `system_prompt`, `command`, or `options`. ### Dialog Mode @@ -475,6 +531,41 @@ input: Access in agents: `{{ workflow.input.question }}` +**Optional inputs without an explicit `default`** resolve to type-appropriate zero values rather than `None`, so templates render cleanly: + +| Input `type` | Zero value | +|---|---| +| `string` | `""` | +| `number` | `0` | +| `boolean` | `false` | +| `array` | `[]` | +| `object` | `{}` | + +This means `{{ workflow.input.optional_msg | default("fallback") }}` correctly renders `"fallback"` when `optional_msg` is omitted, instead of the literal string `"None"`. + +### Workflow Metadata Variables + +In addition to `workflow.input.*`, every agent has access to: + +| Variable | Description | +|---|---| +| `workflow.name` | Workflow name from the YAML | +| `workflow.description` | Workflow description from the YAML | +| `workflow.dir` | Absolute path to the directory containing the workflow YAML | +| `workflow.file` | Absolute path to the workflow YAML file | + +These are available in **all** context modes (they're metadata, not inputs). `workflow.dir` is particularly useful for registry-hosted workflows that need to reference co-located scripts or assets without depending on the caller's working directory: + +```yaml +agents: + - name: detector + type: script + command: pwsh + args: + - "-File" + - "{{ workflow.dir }}/scripts/detect-state.ps1" +``` + ### Workflow Outputs Define the final workflow output: @@ -774,7 +865,7 @@ workflow: ### Available Hook Contexts **`on_start`**: -- `workflow.name`, `workflow.description` +- `workflow.name`, `workflow.description`, `workflow.dir`, `workflow.file` - `workflow.input.*` (all input values) **`on_complete`**: diff --git a/pyproject.toml b/pyproject.toml index 20880da7..2163ea3a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "conductor-cli" -version = "0.1.10" +version = "0.1.11" description = "A CLI tool for defining and running multi-agent workflows with the GitHub Copilot SDK" readme = "README.md" requires-python = ">=3.12" diff --git a/uv.lock b/uv.lock index b60456a0..7e785a3f 100644 --- a/uv.lock +++ b/uv.lock @@ -150,7 +150,7 @@ wheels = [ [[package]] name = "conductor-cli" -version = "0.1.10" +version = "0.1.11" source = { editable = "." } dependencies = [ { name = "anthropic" },