diff --git a/CHANGELOG.md b/CHANGELOG.md index ab525d1f..a8bbfac9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased](https://github.com/microsoft/conductor/compare/v0.1.19...HEAD) +### Added + +- **Hermes provider (experimental)** — optional third provider built on the + NousResearch [`hermes-agent`](https://github.com/NousResearch/hermes-agent) + library, which manages its own tool ecosystem (no MCP configuration). Install + separately with `pip install hermes-agent`; Conductor works without it. + Declared in the experimental provider tier with documented capability + carve-outs (no MCP servers, no per-agent workflow `tools:` allowlist, + structured output via prompt injection). Supports custom endpoints via + structured `runtime.provider` (`base_url` / `api_key`), `hermes_home` + profiles, `hermes_toolsets`, streaming + reasoning event callbacks, + cooperative interrupt, session-history resume, and `max_session_seconds`. + See [`docs/providers/hermes.md`](docs/providers/hermes.md). + ([#235](https://github.com/microsoft/conductor/pull/235)) + ## [0.1.19](https://github.com/microsoft/conductor/compare/v0.1.18...v0.1.19) - 2026-06-16 ### Added diff --git a/README.md b/README.md index cada0bb1..cbab7d98 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ Conductor makes multi-agent workflows — code review pipelines, research-then-s ## Features - **YAML-based workflows** - Define multi-agent workflows in readable YAML -- **Multiple providers** - GitHub Copilot, Anthropic Claude, or Claude Agent SDK with seamless switching +- **Multiple providers** - GitHub Copilot, Anthropic Claude, Claude Agent SDK, or NousResearch Hermes (experimental) with seamless switching - **Parallel execution** - Run agents concurrently (static groups or dynamic for-each) - **Sub-workflow composition** - Reusable sub-workflows with templated `input_mapping`, usable inside `for_each` groups for dynamic fan-out - **Script steps** - Run shell commands and route on exit code or parsed JSON stdout @@ -211,13 +211,14 @@ conductor stop Conductor supports multiple AI providers. Choose based on your needs: -| Feature | Copilot | Claude | Claude Agent SDK | -|---------|---------|--------|------------------| -| **Pricing** | Subscription ($10-39/mo) | Pay-per-token | Via Claude Code CLI | -| **Context Window** | 8K-128K tokens | 200K tokens | 200K tokens | -| **Tool Support (MCP)** | Yes | Planned | Yes (built-in) | -| **Streaming** | Yes | Planned | Yes | -| **Best For** | Heavy usage, tools | Large context, pay-per-use | Full Claude Code toolset | +| Feature | Copilot | Claude | Claude Agent SDK | Hermes | +|---------|---------|--------|------------------|--------| +| **Tier** | Stable | Stable | Experimental | Experimental | +| **Pricing** | Subscription ($10-39/mo) | Pay-per-token | Via Claude Code CLI | Pay-per-token (via hermes) | +| **Context Window** | 8K-128K tokens | 200K tokens | 200K tokens | Per-model | +| **Tool Support (MCP)** | Yes | Planned | Yes (built-in) | No (hermes internal tools) | +| **Streaming** | Yes | Planned | Yes | No | +| **Best For** | Heavy usage, tools | Large context, pay-per-use | Full Claude Code toolset | Multi-provider model access | ### Using Claude @@ -243,7 +244,18 @@ Requires the `claude` CLI to be installed and authenticated. Install the SDK: `u > **Note:** The `claude-agent-sdk` provider delegates tool and MCP server management to the `claude` CLI. Workflow-level `tools` and `runtime.mcp_servers` fields are ignored — configure these through your Claude Code settings instead. -**See also:** [Claude Documentation](docs/providers/claude.md) | [Provider Comparison](docs/providers/comparison.md) | [Migration Guide](docs/providers/migration.md) +### Using Hermes (Experimental) + +```yaml +workflow: + runtime: + provider: hermes + default_model: anthropic/claude-sonnet-4 +``` + +Install the library: `pip install hermes-agent` + +**See also:** [Claude Documentation](docs/providers/claude.md) | [Hermes Documentation](docs/providers/hermes.md) | [Provider Comparison](docs/providers/comparison.md) | [Migration Guide](docs/providers/migration.md) ### Using a Local / Custom LLM Endpoint (Ollama, vLLM, Azure OpenAI, ...) @@ -357,7 +369,8 @@ See the [`examples/`](./examples/) directory for complete workflows: | [Parallel Execution](./docs/parallel-execution.md) | Static parallel groups | | [Dynamic Parallel](./docs/dynamic-parallel.md) | For-each groups and array processing | | [Claude Provider](./docs/providers/claude.md) | Claude setup and configuration | -| [Provider Comparison](./docs/providers/comparison.md) | Copilot vs Claude decision guide | +| [Hermes Provider](./docs/providers/hermes.md) | Hermes setup and configuration | +| [Provider Comparison](./docs/providers/comparison.md) | Copilot vs Claude vs Hermes decision guide | ## Development diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 19653615..419d59b8 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -26,7 +26,7 @@ conductor run [OPTIONS] | `--metadata KEY=VALUE` | `-m` | Workflow metadata (repeatable). Merged on top of YAML `metadata:` and surfaced in the `workflow_started` event. | | `--workspace-instructions` | | Auto-discover convention files (`AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`, and `.github/instructions/**/*.instructions.md`) by walking from CWD up to the git root. Concatenated and prepended to every agent's prompt. See [Workspace Instructions](#workspace-instructions) below for details on the `.github/instructions/` directory convention. | | `--instructions PATH` | | Explicit path to an instructions file (repeatable). Combines with auto-discovered files when both flags are used. | -| `--provider PROVIDER` | `-p` | Override provider (copilot, claude, claude-agent-sdk) | +| `--provider PROVIDER` | `-p` | Override provider (copilot, claude, claude-agent-sdk, hermes) | | `--dry-run` | | Show execution plan without running | | `--skip-gates` | | Auto-select first option at human gates | | `--quiet` | `-q` | Minimal output (agent lifecycle and routing only) | diff --git a/docs/configuration.md b/docs/configuration.md index 67224341..45b3ff96 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -11,7 +11,7 @@ The `runtime` section of your workflow defines provider settings and global defa ```yaml workflow: runtime: - provider: copilot # or 'claude' or 'claude-agent-sdk' + provider: copilot # or 'claude', 'claude-agent-sdk', 'hermes' default_model: gpt-5.2 temperature: 0.7 max_tokens: 4096 @@ -77,6 +77,31 @@ workflow: **See**: [Claude Provider Documentation](providers/claude.md) +### Hermes Provider (Experimental) + +> **Experimental** — see [Experimental Providers](providers/experimental.md) for stability policy. + +Uses the NousResearch hermes-agent library for agent execution. Supports any OpenRouter-style model identifier. + +```yaml +workflow: + runtime: + provider: hermes + default_model: anthropic/claude-sonnet-4 + max_agent_iterations: 25 + max_tokens: 4096 + temperature: 0.7 +``` + +**Features**: +- Access to Anthropic, OpenAI, and OpenRouter models via one provider +- Built-in hermes tool ecosystem (no MCP config required) +- Custom endpoint routing via structured `provider:` config + +**Models**: `anthropic/claude-sonnet-4`, `openai/gpt-4o`, any OpenRouter `provider/model` string + +**See**: [Hermes Provider Documentation](providers/hermes.md) + ### Custom Provider Routing (Ollama / vLLM / Azure OpenAI) `runtime.provider` accepts either the bare string shorthand diff --git a/docs/providers/comparison.md b/docs/providers/comparison.md index b61e2a38..57a38f42 100644 --- a/docs/providers/comparison.md +++ b/docs/providers/comparison.md @@ -1,30 +1,34 @@ -# Provider Comparison: Copilot vs Claude vs Claude Agent SDK +# Provider Comparison: Copilot vs Claude vs Claude Agent SDK vs Hermes -This guide helps you choose between GitHub Copilot, Anthropic Claude, and Claude Agent SDK providers for your workflows. +This guide helps you choose between GitHub Copilot, Anthropic Claude, Claude Agent SDK, and NousResearch Hermes providers for your workflows. ## Quick Comparison -| Feature | Copilot | Claude | Claude Agent SDK | Winner | +| Feature | Copilot | Claude | Claude Agent SDK | Hermes | |---------|---------|--------|------------------|--------| -| **Tier** | Stable | Stable | Experimental ([#241](https://github.com/microsoft/conductor/issues/241)) | Copilot / Claude | -| **Context Window** | per-model (SDK-reported) | per-model (SDK-reported) | 200K | Tie | -| **Pricing Model** | Subscription ($10-39/mo) | Pay-per-token | Via Claude Code CLI | Depends | -| **Setup** | GitHub auth | API key | `claude` CLI auth | Copilot (easier) | -| **Model Selection** | GPT-5.2, o1 | Haiku, Sonnet, Opus | Haiku, Sonnet, Opus | Tie | -| **Streaming** | Yes | No (Phase 1) | Yes | Copilot / Claude Agent SDK | -| **Tool Support** | Yes (MCP, all types) | Yes (MCP, stdio only) | Yes (built-in, CLI-managed) | Copilot | -| **Reasoning / Extended Thinking** | Yes (`reasoning_effort` on session) | Yes (extended `thinking` budget) | Inherits from CLI config | Tie | -| **Speed** | Fast | Fast | Fast | Tie | -| **Output Quality** | Excellent | Excellent | Excellent | Tie | -| **Cost Predictability** | High (flat rate) | Variable (usage-based) | Variable | Copilot | -| **Multi-provider** | No | Yes (via Conductor) | No | Claude | -| **Agentic Loop** | SDK-managed | Manual (provider code) | SDK-managed (delegated to CLI) | Depends | - -> **About the experimental tier.** `claude-agent-sdk` declares specific -> capability carve-outs (no MCP, no per-agent tools allowlist, no -> reasoning_effort, no checkpoint resume). `conductor validate` catches -> workflows that depend on those features against this provider, and the -> CLI prints a one-time banner when the workflow runs. See +| **Tier** | Stable | Stable | Experimental | Experimental | +| **Context Window** | per-model (SDK-reported) | per-model (SDK-reported) | 200K | per-model | +| **Pricing Model** | Subscription ($10-39/mo) | Pay-per-token | Via Claude Code CLI | Pay-per-token (via hermes) | +| **Setup** | GitHub auth | API key | `claude` CLI auth | API key (model-provider's key) | +| **Model Selection** | GPT-5.2, o1 | Haiku, Sonnet, Opus | Haiku, Sonnet, Opus | Any OpenRouter-style model | +| **Streaming** | Yes | No (Phase 1) | Yes | Yes | +| **Tool Support** | Yes (MCP, all types) | Yes (MCP, stdio only) | Yes (built-in, CLI-managed) | Yes (hermes toolsets) | +| **MCP Servers** | Yes | Yes (stdio) | No | No | +| **Reasoning / Extended Thinking** | Yes (`reasoning_effort` on session) | Yes (extended `thinking` budget) | Inherits from CLI config | Yes (`reasoning_config`) | +| **Speed** | Fast | Fast | Fast | Depends on model | +| **Output Quality** | Excellent | Excellent | Excellent | Depends on model | +| **Cost Predictability** | High (flat rate) | Variable (usage-based) | Variable | Variable (usage-based) | +| **Multi-provider** | No | Yes (via Conductor) | No | Yes (native) | +| **Agentic Loop** | SDK-managed | Manual (provider code) | SDK-managed (delegated to CLI) | SDK-managed (delegated to hermes) | +| **Structured Output** | Prompt injection | Native | Prompt injection | Prompt injection | +| **Session Resume** | Yes | No | No | Yes | + +> **About the experimental tier.** `claude-agent-sdk` and `hermes` declare +> specific capability carve-outs (e.g. no MCP servers). `conductor validate` +> catches workflows that depend on those features against these providers, +> and the CLI prints a one-time banner when the workflow runs. See +> [docs/providers/experimental.md](./experimental.md) for the stability +> policy and promotion criteria. > [docs/providers/experimental.md](./experimental.md) for the stability > policy and promotion criteria. @@ -32,30 +36,19 @@ This guide helps you choose between GitHub Copilot, Anthropic Claude, and Claude ### ✅ Choose Copilot if: -1. **You have a GitHub Copilot subscription** - - Already paying $10-39/month - - No additional costs for API usage - - Predictable monthly billing - -2. **You need tool support (MCP)** - - Web search, code execution, file operations - - Real-time data access - - External API integrations +1. **You have a GitHub Copilot subscription** — Already paying $10-39/month, no additional API costs +2. **You need MCP tool support** — Web search, code execution, file operations, external API integrations +3. **You want streaming responses** — Real-time feedback, better UX for long-running workflows +4. **You prefer enterprise support** — GitHub Enterprise integration, SSO, access controls +5. **Heavy usage** — Flat rate beats pay-per-token at scale -3. **You want streaming responses** - - Real-time feedback as the model generates - - Better UX for long-running workflows - - Progress visibility +### When to Use Copilot -4. **You prefer enterprise support** - - GitHub Enterprise integration - - SSO and access controls - - Enterprise SLA and support - -5. **You need smaller context windows (cost optimization)** - - GPT-4: 8K context (cheaper) - - GPT-4 Turbo: 128K context (when needed) - - Pay only for subscription, not per token +1. **You have a GitHub Copilot subscription** — No additional costs, predictable billing +2. **You need MCP tool support** — Web search, code execution, file operations +3. **You want streaming responses** — Real-time feedback as the model generates +4. **Heavy usage** — Flat rate beats pay-per-token at scale +5. **Enterprise features** — SSO, GitHub Enterprise integration ### Example Copilot Workflow @@ -81,30 +74,11 @@ agents: ### ✅ Choose Claude if: -1. **You need a large context window** - - 200K tokens (all models) - - Process long documents, code, transcripts - - Multi-agent workflows with extensive context - -2. **You want fine-grained cost control** - - Pay only for what you use - - Scale to zero when idle - - Optimize costs with model selection (Haiku vs Opus) - -3. **You value output quality for reasoning tasks** - - Claude excels at analysis, synthesis, reasoning - - More verbose explanations - - Better at following complex instructions - -4. **You run low-volume or intermittent workflows** - - Pay-per-use cheaper than subscription - - No minimum monthly cost - - Scale up/down as needed - -5. **You want to avoid vendor lock-in** - - Anthropic API works with multiple tools - - Easier migration between platforms - - Future-proof for multi-provider strategies +1. **You need a large context window** — 200K tokens, all models; great for long documents and multi-agent workflows +2. **You want fine-grained cost control** — Pay only for what you use; scale to zero when idle +3. **You value reasoning quality** — Claude excels at analysis, synthesis, and following complex instructions +4. **Light/intermittent usage** — Pay-per-use is cheaper than a subscription at low volumes +5. **You need reliable structured output** — Native schema enforcement more robust than prompt injection ### Example Claude Workflow @@ -170,222 +144,45 @@ agents: prompt: "Research {{ topic }} using web search" ``` -## Cost Comparison - -### Scenario 1: Light Usage (10 hours/month) - -**Copilot**: -- Subscription: $10-39/month -- Total: **$10-39/month** - -**Claude**: -- ~100 requests/month -- ~1000 tokens/request input, ~2000 tokens/request output -- Sonnet: (0.1M × $3) + (0.2M × $15) = $0.30 + $3.00 = **$3.30/month** -- Haiku: (0.1M × $1) + (0.2M × $5) = $0.10 + $1.00 = **$1.10/month** - -**Winner**: Claude (3-35x cheaper) - -### Scenario 2: Medium Usage (40 hours/month) - -**Copilot**: -- Subscription: $10-39/month -- Total: **$10-39/month** - -**Claude**: -- ~500 requests/month -- ~2000 tokens/request input, ~4000 tokens/request output -- Sonnet: (1M × $3) + (2M × $15) = $3.00 + $30.00 = **$33/month** -- Haiku: (1M × $1) + (2M × $5) = $1.00 + $10.00 = **$11/month** - -**Winner**: Tie (depends on model choice and subscription tier) - -### Scenario 3: Heavy Usage (160+ hours/month) - -**Copilot**: -- Subscription: $10-39/month -- Total: **$10-39/month** (flat rate) - -**Claude**: -- ~2000 requests/month -- ~3000 tokens/request input, ~5000 tokens/request output -- Sonnet: (6M × $3) + (10M × $15) = $18 + $150 = **$168/month** -- Haiku: (6M × $1) + (10M × $5) = $6 + $50 = **$56/month** - -**Winner**: Copilot (3-17x cheaper) - -### Cost Optimization Tips - -**Copilot**: -- Use the subscription you already have -- Optimize prompts to reduce latency (not cost) -- No per-token optimization needed - -**Claude**: -- Use Haiku for simple tasks (3x cheaper than Sonnet) -- Limit `max_tokens` to reduce output costs -- Use `context: mode: explicit` to reduce input tokens - -## Feature Comparison - -### Context Window - -**Copilot**: -- GPT-5.2: 8K tokens -- GPT-5.2 Turbo: 128K tokens -- Model-dependent - -**Claude**: -- All models: 200K tokens -- Consistent across tiers -- Better for large documents - -**Winner**: Claude (200K vs 128K max) - -### Model Selection - -**Copilot**: -- `gpt-5.2` - Balanced performance -- `gpt-5.2-turbo` - Faster, larger context -- `gpt-5.2-mini` - Latest, optimized -- `o1-preview` - Advanced reasoning (limited availability) - -**Claude**: -- `claude-haiku-4.5` - Fast, cheap -- `claude-sonnet-4.5` - Balanced (default) -- `claude-opus-4.5` - Premium reasoning - -**Winner**: Tie (both offer good model tiers) - -### Structured Output - -**Copilot**: -- Native JSON mode -- Schema validation -- Reliable extraction - -**Claude**: -- Tool-based structured output -- JSON fallback parsing -- Highly reliable with tool approach - -**Winner**: Tie (both work well) - -### Streaming +## When to Use Hermes -**Copilot**: -- ✅ Real-time streaming -- Progressive response display -- Better UX for long responses +> **Experimental Provider** — Hermes is an experimental provider. See +> [Experimental Providers](./experimental.md) for stability policy and +> known limitations. -**Claude**: -- ❌ Not available in Phase 1 -- Planned for Phase 2+ -- Currently non-streaming only +### ✅ Choose Hermes if: -**Winner**: Copilot (until Claude Phase 2+) +1. **You want access to many model providers** — Anthropic, OpenAI, and any OpenRouter-supported model via a single provider +2. **You need hermes's built-in tool ecosystem** — Hermes manages its own tools internally; no MCP config required +3. **You're already using hermes-agent** — Integrate your existing hermes workflows with Conductor's orchestration +4. **Model flexibility matters most** — Switch between `anthropic/claude-sonnet-4` and `openai/gpt-4o` by changing a single field -### Tool Support (MCP) +### ✅ Avoid Hermes if: -**Copilot**: -- ✅ Full MCP support (stdio, http, sse) -- Web search, code execution, file ops -- Workflow-level and agent-level tools +- You need **MCP server support** — use Copilot or Claude instead +- You need **reliable structured output** — Hermes uses prompt injection; Claude/Copilot use native APIs +- You need **reasoning effort control** — the `reasoning.effort` field is a no-op for Hermes -**Claude**: -- ✅ MCP support for stdio servers -- Uses Conductor's built-in MCPManager -- HTTP/SSE servers not supported - -**Winner**: Copilot (broader transport support) - -See the [MCP Tools guide](../mcp-tools.md) for details. - -### Reasoning / Extended Thinking - -Both providers expose a unified [`reasoning.effort`](../configuration.md#reasoning-effort) -field (`low` | `medium` | `high` | `xhigh`) at workflow scope -(`runtime.default_reasoning_effort`) or per agent (`reasoning.effort`). -Conductor translates the value to each provider's native API: - -**Copilot**: -- Forwarded as `reasoning_effort` on `CopilotClient.create_session` -- Validated against the model's advertised `supported_reasoning_efforts` - -**Claude**: -- Translated to `messages.create(thinking={"type": "enabled", "budget_tokens": N})` -- Effort → budget: low=2048, medium=8192, high=16384, xhigh=32768 tokens -- Restricted to thinking-capable models (`claude-3-7-*`, `claude-opus-4*`, - `claude-sonnet-4*`, `claude-haiku-4*`) -- Auto-coerces `temperature=1.0` and bumps `max_tokens` to fit the budget - -Reasoning content from either provider surfaces as `agent_reasoning` events -in the dashboard, JSONL log, and `-vv` console output. - -**Winner**: Tie (both support it; pick the provider on other grounds) - -See [`examples/reasoning-effort.yaml`](../../examples/reasoning-effort.yaml). - -## Migration Path - -### From Copilot to Claude - -Minimal changes required: +### Example Hermes Workflow ```yaml -# Before (Copilot) workflow: + name: hermes-workflow runtime: - provider: copilot - default_model: gpt-5.2 + provider: hermes + default_model: anthropic/claude-sonnet-4 + max_agent_iterations: 25 -# After (Claude) -workflow: - runtime: - provider: claude - default_model: claude-sonnet-4.5 -``` - -See the [Migration Guide](migration.md) for detailed instructions. - -### From Claude to Copilot - -Also straightforward: - -```yaml -# Before (Claude) -workflow: - runtime: - provider: claude - default_model: claude-sonnet-4.5 - max_tokens: 4096 - -# After (Copilot) -workflow: - runtime: - provider: copilot - default_model: gpt-5.2 - # Remove Claude-specific fields +agents: + - name: researcher + prompt: "Research the following topic thoroughly: {{ workflow.input.topic }}" + output: + findings: + type: string + routes: + - to: $end ``` -## Decision Matrix - -Use this matrix to decide: - -| Your Situation | Recommended Provider | -|----------------|---------------------| -| Already have Copilot subscription | **Copilot** | -| Need tools (web search, code exec) | **Copilot** | -| Need streaming responses | **Copilot** | -| Heavy usage (>160 hrs/mo) | **Copilot** | -| Need 200K context window | **Claude** | -| Light usage (<10 hrs/mo) | **Claude** | -| Want pay-per-use pricing | **Claude** | -| Process long documents | **Claude** | -| Complex reasoning tasks | **Claude** (Opus) | -| Simple high-volume tasks | **Claude** (Haiku 4.5) | -| Already use `claude` CLI | **Claude Agent SDK** | -| Want streaming with Claude | **Claude Agent SDK** | ## Multi-Provider Strategy @@ -422,7 +219,7 @@ workflow: ## Summary **Choose Copilot** for: -- ✅ Tool support (MCP) +- ✅ MCP tool support (web search, code execution) - ✅ Streaming responses - ✅ Predictable costs (subscription) - ✅ Heavy usage @@ -433,6 +230,7 @@ workflow: - ✅ Pay-per-use pricing - ✅ Light/intermittent usage - ✅ Long document processing +- ✅ Reliable structured output - ✅ Cost optimization (Haiku) **Choose Claude Agent SDK** for: @@ -442,4 +240,10 @@ workflow: - ✅ Existing `claude` CLI users - ✅ No API key management -**Bottom line**: All three are excellent. Choose based on your usage patterns, budget, and feature requirements. Conductor makes it easy to switch between them or use all three strategically. +**Choose Hermes** for: +- ✅ Multi-provider model access (Anthropic, OpenAI, OpenRouter) +- ✅ Hermes's built-in tool ecosystem (no MCP config) +- ✅ Existing hermes-agent workflows +- ✅ Maximum model flexibility + +**Bottom line**: All four providers are excellent at what they do. Choose based on your usage patterns, budget, tool requirements, and model preferences. Conductor makes it easy to switch between them or use multiple strategically within a single multi-provider workflow. diff --git a/docs/providers/experimental.md b/docs/providers/experimental.md index 10dec2ed..c7fc64f9 100644 --- a/docs/providers/experimental.md +++ b/docs/providers/experimental.md @@ -98,6 +98,7 @@ adopting one does not inflate the install surface for others. | Provider | Upstream pin | Maintainer | Capability carve-outs | |---|---|---|---| | `claude-agent-sdk` | `claude-agent-sdk>=0.1.0` | `@lesandiz (best-effort)` | no `mcp_tools`, no `workflow_tools_passthrough`, no `reasoning_effort`, `prompt_injection` structured output, no `checkpoint_resume` | +| `hermes` | `hermes-agent` | `(community contribution)` | no `mcp_tools`, `prompt_injection` structured output | ## See also diff --git a/docs/providers/hermes.md b/docs/providers/hermes.md new file mode 100644 index 00000000..6e47069a --- /dev/null +++ b/docs/providers/hermes.md @@ -0,0 +1,286 @@ +# Hermes Provider Documentation + +> **Experimental Provider** — Hermes is an experimental provider with known +> capability carve-outs (no MCP servers, no per-agent tools allowlist, +> structured output via prompt injection only). `conductor validate` catches +> workflows that depend on unsupported features, and the CLI prints a one-time +> banner at runtime. See [Experimental Providers](./experimental.md) for the +> stability policy and promotion criteria. + +The Hermes provider enables Conductor workflows to use the [NousResearch hermes-agent](https://github.com/NousResearch/hermes-agent) library. Hermes is an agentic AI framework that manages its own tool ecosystem and supports models from multiple providers via OpenRouter-style model identifiers. + +## Table of Contents + +- [Quick Start](#quick-start) +- [Installation](#installation) +- [Model Format](#model-format) +- [Runtime Configuration](#runtime-configuration) +- [Toolset Control](#toolset-control) +- [Model Routing](#model-routing) +- [Structured Output](#structured-output) +- [Tool Use](#tool-use) +- [Limitations](#limitations) +- [Troubleshooting](#troubleshooting) + +## Quick Start + +### 1. Install the hermes-agent library + +```bash +pip install hermes-agent +``` + +### 2. Set up API credentials + +Hermes reads credentials from its own environment variables depending on the model provider you choose: + +```bash +# For Anthropic models (e.g. anthropic/claude-sonnet-4) +export ANTHROPIC_API_KEY=sk-ant-... + +# For OpenAI models (e.g. openai/gpt-4o) +export OPENAI_API_KEY=sk-... +``` + +### 3. Update your workflow + +```yaml +workflow: + name: my-workflow + runtime: + provider: hermes + default_model: anthropic/claude-sonnet-4 + +agents: + - name: assistant + prompt: | + Answer the following question: {{ workflow.input.question }} + output: + answer: + type: string + routes: + - to: $end +``` + +### 4. Run your workflow + +```bash +conductor run my-workflow.yaml --input question="What is Python?" +``` + +## Installation + +Hermes is an **optional dependency** — Conductor works without it. Install it only when you want to use the hermes provider: + +```bash +pip install hermes-agent +``` + +If the library is not installed and you try to use `provider: hermes`, Conductor raises a `ProviderError` with an install hint at startup. + +## Model Format + +Hermes uses OpenRouter-style model identifiers in the form `provider/model-name`: + +| Format | Example | +|--------|---------| +| `anthropic/model` | `anthropic/claude-sonnet-4` | +| `openai/model` | `openai/gpt-4o` | +| `openrouter/provider/model` | `openrouter/anthropic/claude-sonnet-4` | + +Set the default model for all agents via `runtime.default_model`, or override per-agent with `model:`: + +```yaml +workflow: + runtime: + provider: hermes + default_model: anthropic/claude-sonnet-4 + +agents: + - name: fast_task + model: openai/gpt-4o-mini # Override for this agent + prompt: "Classify: {{ text }}" +``` + +If `model` is omitted entirely (neither per-agent nor `default_model`), hermes uses its own configured default model. + +## Runtime Configuration + +| Parameter | Forwarded to AIAgent | Default | Description | +|-----------|----------------------|---------|-------------| +| `default_model` | `model=` | hermes default | Model in `provider/model` format | +| `max_agent_iterations` | `max_iterations=` | 90 | Maximum tool-calling iterations per agent | +| `max_tokens` | `max_tokens=` | hermes default | Maximum output tokens | +| `temperature` | `temperature=` | hermes default | Sampling temperature | +| `max_session_seconds` | asyncio timeout | no limit | Wall-clock deadline per agent execution | + +```yaml +workflow: + runtime: + provider: hermes + default_model: anthropic/claude-sonnet-4 + max_agent_iterations: 25 # Limit tool iterations (default: 90) + max_tokens: 4096 + temperature: 0.7 +``` + +### Per-Agent Overrides + +```yaml +agents: + - name: light_task + model: openai/gpt-4o-mini + max_agent_iterations: 5 # Fewer iterations for simple tasks + prompt: "Summarize: {{ text }}" +``` + +## Toolset Control + +By default, hermes loads its full set of built-in tools (approximately 33). Use the `hermes_toolsets` provider setting to restrict which toolsets are available across the workflow. + +> **Important**: The per-agent `tools:` field uses Conductor workflow tool names, which do not translate to Hermes toolset names. Setting a non-empty `tools:` list on an agent will raise a validation error. Use `tools: []` to disable all tools for a specific agent, or `hermes_toolsets` to restrict toolsets at the provider level. + +| Configuration | Effect | +|---------------|--------| +| _(no `hermes_toolsets`)_ | All hermes tools active (default) | +| `hermes_toolsets: [web, filesystem]` | Only named toolsets enabled | +| `hermes_toolsets: []` | No tools for any agent | +| Per-agent `tools: []` | No tools for that specific agent | + +```yaml +workflow: + runtime: + provider: + name: hermes + hermes_toolsets: [web, filesystem] # Restrict at provider level + +agents: + - name: judgment_only + tools: [] # Disables all hermes tools for this agent + prompt: "Based on this data, what do you conclude? {{ context }}" + + - name: web_researcher + prompt: "Research: {{ workflow.input.topic }}" + # Gets the provider-level toolsets (web, filesystem) +``` + +**Why this matters**: Loading all 33 tools inflates the system prompt by ~25k tokens per agent step. For workflows that only need specific toolsets, restricting via `hermes_toolsets` reduces input token costs significantly. + +## Model Routing + +To route requests through a custom endpoint (e.g. OpenRouter, a litellm gateway, or a corporate API proxy), use the structured `provider:` config: + +```yaml +workflow: + runtime: + provider: + name: hermes + base_url: "https://openrouter.ai/api/v1" + api_key: "${OPENROUTER_API_KEY}" + default_model: anthropic/claude-sonnet-4 +``` + +Both `base_url` and `api_key` are forwarded directly to `AIAgent`. The `api_key` value supports `${ENV_VAR}` interpolation in YAML so the literal secret never appears in event logs or checkpoints. + +## Structured Output + +Hermes does not have a native structured output API. When an agent declares an `output:` schema, Conductor automatically appends a JSON instruction to the prompt: + +``` +Respond ONLY with a valid JSON object. Do not include any explanation, +markdown, or text outside the JSON object. +``` + +The response is then parsed and validated against your schema as usual: + +```yaml +agents: + - name: analyzer + prompt: | + Analyze the following text and return your findings. + Text: {{ workflow.input.text }} + output: + sentiment: + type: string + description: "positive, negative, or neutral" + confidence: + type: number + description: "0.0 to 1.0" + routes: + - to: $end +``` + +**Note**: Because structured output relies on prompt engineering rather than a native API, reliability can vary. For workflows where schema compliance is critical, the `copilot` or `claude` providers offer more robust structured output. + +## Tool Use + +Hermes manages its own toolsets internally. Conductor's per-agent `tools:` field contains workflow tool names (resolved via `runtime.tools`) — these are a different vocabulary from Hermes toolset names and cannot be forwarded. Use `hermes_toolsets` in provider settings to control which Hermes toolsets are active — see [Toolset Control](#toolset-control) above. + +**Per-agent `tools: []`** is supported and disables all tools for that agent (judgment-only mode). + +**Isolation flags**: Conductor always passes `skip_context_files=True`, `skip_memory=True`, and `quiet_mode=True` to the hermes library. This prevents hermes from loading workspace files (`AGENTS.md`, etc.) or its own persistent memory — the conductor workflow YAML and rendered prompts are the sole source of context. + +**MCP servers**: The hermes provider does not support Conductor's `runtime.mcp_servers` configuration. Hermes has its own tool ecosystem separate from MCP. + +## Limitations + +| Limitation | Details | +|------------|---------| +| **No MCP server support** | `runtime.mcp_servers` is ignored; hermes uses its own tools | +| **No per-agent tools allowlist** | Per-agent `tools: [names]` is rejected; use `hermes_toolsets` in provider settings | +| **Structured output via prompt** | Less reliable than native schema enforcement (copilot/claude) | + +## Troubleshooting + +### Hermes library not installed + +**Error**: `ProviderError: Hermes provider requires the hermes-agent package` + +**Fix**: +```bash +pip install hermes-agent +``` + +### Model not found + +**Error**: `ProviderError: Hermes agent execution failed (model='anthropic/...'): ...` + +**Symptoms**: Hermes returns `failed: true` in the result dict. + +**Fix**: Verify the model identifier follows the `provider/model` format and that the corresponding API key is set: +```bash +# Verify key is set +echo $ANTHROPIC_API_KEY +echo $OPENAI_API_KEY +``` + +### Output schema validation failures + +**Error**: `ValidationError: missing required field 'answer'` + +**Cause**: The model returned text that wasn't valid JSON, or JSON that didn't match the schema. + +**Fix**: Make the prompt more explicit, or use the `claude` or `copilot` provider for strict schema compliance: +```yaml +agents: + - name: analyzer + prompt: | + Analyze the input and respond ONLY with a JSON object with these exact fields: + - sentiment: string (positive/negative/neutral) + - confidence: number (0.0-1.0) + + Input: {{ workflow.input.text }} +``` + +### Enable debug logging + +```bash +export CONDUCTOR_LOG_LEVEL=DEBUG +conductor run workflow.yaml +``` + +This logs: +- The resolved model and iteration limits per agent +- The full prompt sent to hermes (including the JSON instruction if applicable) +- Token counts from the hermes result +- The raw `final_response` before schema validation diff --git a/docs/workflow-syntax.md b/docs/workflow-syntax.md index 94f10dca..2ad8ae7c 100644 --- a/docs/workflow-syntax.md +++ b/docs/workflow-syntax.md @@ -45,7 +45,7 @@ workflow: context_mode: accumulate # accumulate | snapshot | minimal (default: accumulate) runtime: - provider: copilot # copilot | claude + provider: copilot # copilot | claude | hermes default_model: gpt-5.2 temperature: 0.7 max_tokens: 4096 diff --git a/examples/hermes-features.yaml b/examples/hermes-features.yaml new file mode 100644 index 00000000..a81d6752 --- /dev/null +++ b/examples/hermes-features.yaml @@ -0,0 +1,246 @@ +# Hermes Provider Feature Test +# +# Comprehensive functional test for the Hermes provider, exercising: +# - Structured output (prompt injection + JSON parse/validate) +# - runtime.default_reasoning_effort + per-agent reasoning.effort override +# - runtime.temperature and runtime.max_tokens +# - base_url / api_key via ProviderSettings env interpolation +# - tools: [] (disable toolsets per agent) +# - Parallel group with two Hermes agents +# - Nested input projection in explicit context mode +# - Set step → Hermes agent handoff +# - Conditional routing on structured output +# +# Requirements: +# - hermes-agent package installed (pip install hermes-agent) +# - API key for the configured model provider (e.g. OPENROUTER_API_KEY or +# the key expected by hermes for the default model) +# +# Usage: +# conductor run examples/hermes-features.yaml \ +# --input question="What is dependency injection in software engineering?" +# +# # With custom endpoint (set env vars before running): +# export HERMES_BASE_URL=https://openrouter.ai/api/v1 +# export HERMES_API_KEY=$OPENROUTER_API_KEY +# conductor run examples/hermes-features.yaml \ +# --input question="Explain the CAP theorem" + +workflow: + name: hermes-features + description: Comprehensive Hermes provider feature test + version: "1.0.0" + entry_point: set_config + + runtime: + provider: + name: hermes + # hermes_skip_memory: false # default: load profile memory + # hermes_skip_context_files: false # default: load SOUL.md / context + default_model: anthropic/claude-sonnet-4 + temperature: 0.7 + max_tokens: 2048 + default_reasoning_effort: medium + + input: + question: + type: string + required: true + description: A question to classify and research + + context: + mode: explicit + + limits: + max_iterations: 20 + timeout_seconds: 300 + +agents: + # --- Step 1: Set step derives a computed context value --- + - name: set_config + type: set + description: Derive a formatted topic string for downstream agents + values: + topic_formatted: "{{ workflow.input.question | upper }}" + timestamp: "2026-06-07" + routes: + - to: classifier + + # --- Step 2: Classifier with structured output + tools disabled --- + - name: classifier + description: Classify the question into a category (no tools allowed) + tools: [] + input: + - workflow.input.question + - set_config.output.topic_formatted + prompt: | + Classify the following question into exactly one category. + + Question: {{ workflow.input.question }} + (Formatted: {{ set_config.output.topic_formatted }}) + + Categories: + - "technical" — about software, engineering, algorithms, or systems + - "general" — about anything else + + Return your classification as JSON. + output: + category: + type: string + description: One of "technical" or "general" + confidence: + type: number + description: Confidence score from 0.0 to 1.0 + routes: + - to: parallel_researchers + when: "{{ output.category == 'technical' }}" + - to: simple_responder + + # --- Step 3a: Parallel research (technical path) --- + # Two Hermes agents run concurrently. One uses high reasoning effort. + - name: broad_researcher + description: Quick broad research pass + input: + - workflow.input.question + prompt: | + Provide a broad overview answer to this technical question: + {{ workflow.input.question }} + + Return JSON with your findings. + output: + summary: + type: string + description: Broad overview answer + key_concepts: + type: array + items: + type: string + description: List of key concepts mentioned + + - name: deep_researcher + description: Deep analysis pass with high reasoning effort + reasoning: + effort: high + input: + - workflow.input.question + prompt: | + Provide a deep technical analysis of this question: + {{ workflow.input.question }} + + Think carefully about edge cases, trade-offs, and nuances. + Return JSON with your findings structured with nested detail. + output: + findings: + type: object + description: Structured research findings + properties: + key_points: + type: array + items: + type: string + description: Key technical points + trade_offs: + type: string + description: Important trade-offs to consider + + # --- Step 4: Synthesizer using nested input projection --- + # Demonstrates nested projection: parallel_researchers.outputs.broad_researcher + # gives us the whole agent output, while deep_researcher.output.findings would + # give nested access on a regular agent. For parallel groups, we project at + # the agent level and access nested fields in the template. + - name: synthesizer + description: Synthesize parallel research outputs + input: + - workflow.input.question + - parallel_researchers.outputs.broad_researcher + - parallel_researchers.outputs.deep_researcher + prompt: | + Synthesize these research findings into a cohesive answer. + + Original question: {{ workflow.input.question }} + + Broad overview: + {{ parallel_researchers.outputs.broad_researcher.summary }} + + Key technical points: + {% for point in parallel_researchers.outputs.deep_researcher.findings.key_points %} + - {{ point }} + {% endfor %} + + Trade-offs: + {{ parallel_researchers.outputs.deep_researcher.findings.trade_offs }} + + Provide a comprehensive final answer as JSON. + output: + answer: + type: string + description: Final synthesized answer + depth: + type: string + description: Either "deep" or "surface" + routes: + - to: $end + + # --- Step 3b: Simple responder (non-technical path) --- + # Produces nested output to demonstrate nested input projection downstream. + - name: simple_responder + description: Handle non-technical questions with nested structured output + input: + - workflow.input.question + prompt: | + Answer this general question concisely: + {{ workflow.input.question }} + + Return JSON with a nested "result" object containing "text" and "tone" fields. + output: + result: + type: object + description: Nested answer structure + properties: + text: + type: string + description: The answer text + tone: + type: string + description: The tone of the answer (e.g. "informative", "casual") + routes: + - to: format_result + + # --- Step 4b: Demonstrates nested input projection from a regular agent --- + # Projects simple_responder.output.result.text (2 levels deep) into context. + - name: format_result + type: set + description: Extract nested field via explicit input projection + input: + - simple_responder.output.result.text + - simple_responder.output.result.tone + value: "{{ simple_responder.output.result.text }}" + routes: + - to: $end + +parallel: + - name: parallel_researchers + description: Run broad and deep research concurrently + agents: + - broad_researcher + - deep_researcher + failure_mode: continue_on_error + routes: + - to: synthesizer + +output: + question: "{{ workflow.input.question }}" + category: "{{ classifier.output.category }}" + confidence: "{{ classifier.output.confidence }}" + answer: | + {%- if synthesizer is defined -%} + {{ synthesizer.output.answer }} + {%- elif format_result is defined -%} + {{ format_result.output }} + {%- endif -%} + depth: | + {%- if synthesizer is defined -%} + {{ synthesizer.output.depth }} + {%- elif simple_responder is defined -%} + {{ simple_responder.output.result.tone }} + {%- endif -%} diff --git a/examples/hermes-simple.yaml b/examples/hermes-simple.yaml new file mode 100644 index 00000000..4d44a0e0 --- /dev/null +++ b/examples/hermes-simple.yaml @@ -0,0 +1,26 @@ +workflow: + name: hermes-simple-qa + description: Simple Q&A using the Hermes provider (NousResearch hermes-agent library) + entry_point: answerer + runtime: + provider: hermes + default_model: anthropic/claude-sonnet-4 + input: + question: + type: string + required: true + description: The question to answer + +agents: + - name: answerer + prompt: | + Answer the following question clearly and concisely: + {{ workflow.input.question }} + output: + answer: + type: string + routes: + - to: $end + +output: + answer: "{{ answerer.output.answer }}" diff --git a/examples/hermes-toolsets.yaml b/examples/hermes-toolsets.yaml new file mode 100644 index 00000000..4ea34e49 --- /dev/null +++ b/examples/hermes-toolsets.yaml @@ -0,0 +1,135 @@ +# Hermes Provider — Toolsets & For-Each Test +# +# Focused functional test for: +# - hermes_toolsets in ProviderSettings (restrict available toolsets globally) +# - for_each dynamic fan-out with Hermes agents +# - max_agent_iterations forwarding +# - max_session_seconds wall-clock timeout +# +# This workflow uses hermes_toolsets: [] to disable all tools globally, +# forcing the agents to respond from knowledge only (no tool use). +# +# Requirements: +# - hermes-agent package installed (pip install hermes-agent) +# - API key for the configured model provider +# +# Usage: +# conductor run examples/hermes-toolsets.yaml \ +# --input topic="Python design patterns" +# +# # With custom endpoint (set env vars before running): +# export HERMES_BASE_URL=https://openrouter.ai/api/v1 +# export HERMES_API_KEY=$OPENROUTER_API_KEY +# conductor run examples/hermes-toolsets.yaml \ +# --input topic="Rust ownership model" + +workflow: + name: hermes-toolsets + description: Test hermes_toolsets restriction and for_each dynamic fan-out + version: "1.0.0" + entry_point: generate_subtopics + + runtime: + provider: + name: hermes + hermes_toolsets: [] + default_model: anthropic/claude-sonnet-4 + max_tokens: 1024 + max_agent_iterations: 5 + max_session_seconds: 60 + + input: + topic: + type: string + required: true + description: A topic to break into subtopics and analyze + + limits: + max_iterations: 20 + timeout_seconds: 180 + +agents: + # Step 1: Hermes agent generates subtopics (no tools available due to hermes_toolsets: []) + - name: generate_subtopics + description: Break the topic into 3 subtopics for parallel analysis + prompt: | + Break the following topic into exactly 3 focused subtopics for deeper analysis. + + Topic: {{ workflow.input.topic }} + + Return JSON with an array of subtopics, each having a name and a guiding question. + output: + subtopics: + type: array + items: + type: object + properties: + name: + type: string + question: + type: string + description: Array of 3 subtopics with guiding questions + routes: + - to: analyze_subtopics + + # Step 2: For-each fans out over the subtopics array + - name: aggregator + description: Combine all subtopic analyses into a final summary + prompt: | + Combine these subtopic analyses into a cohesive summary. + + Topic: {{ workflow.input.topic }} + + Analyses: + {% for result in analyze_subtopics.outputs %} + ## {{ result.subtopic_name }} + {{ result.analysis }} + {% endfor %} + + Return JSON with a final summary and the total number of subtopics covered. + output: + summary: + type: string + description: Combined analysis summary + subtopics_covered: + type: number + description: Number of subtopics successfully analyzed + routes: + - to: $end + +for_each: + - name: analyze_subtopics + type: for_each + description: Analyze each subtopic independently + source: generate_subtopics.output.subtopics + as: subtopic + max_concurrent: 3 + failure_mode: continue_on_error + + agent: + name: subtopic_analyzer + prompt: | + Analyze this subtopic in depth. You have no tools available — rely on + your training knowledge only. + + Subtopic {{ _index + 1 }}: {{ subtopic.name }} + Guiding question: {{ subtopic.question }} + + Provide a focused analysis answering the guiding question. + Return JSON with your analysis. + output: + subtopic_name: + type: string + description: Name of the subtopic analyzed + analysis: + type: string + description: Focused analysis of the subtopic + + routes: + - to: aggregator + +output: + topic: "{{ workflow.input.topic }}" + subtopics_generated: "{{ generate_subtopics.output.subtopics | length }}" + summary: "{{ aggregator.output.summary }}" + subtopics_covered: "{{ aggregator.output.subtopics_covered }}" diff --git a/plugins/conductor/skills/conductor/SKILL.md b/plugins/conductor/skills/conductor/SKILL.md index db30c5ee..755d8836 100644 --- a/plugins/conductor/skills/conductor/SKILL.md +++ b/plugins/conductor/skills/conductor/SKILL.md @@ -5,7 +5,7 @@ description: Validate, run, and execute workflows; creating new workflows when e # Conductor -CLI tool for defining and running multi-agent workflows with the GitHub Copilot SDK, Anthropic Claude, or Claude Agent SDK. +CLI tool for defining and running multi-agent workflows with the GitHub Copilot SDK, Anthropic Claude, Claude Agent SDK, or Hermes (NousResearch, experimental). > **DO NOT create new workflow files unless the user explicitly asks you to create one.** Default to running, validating, or debugging existing workflows. If the user's request is ambiguous, assume they want to run or modify an existing workflow rather than create a new one. @@ -117,7 +117,7 @@ For runtime config, context modes, limits, and cost tracking, see [references/au | `limits` | Safety bounds (max_iterations up to 500, timeout_seconds) | | `timeout_seconds` (agent) | Hard wall-clock cancellation per agent (provider-backed agents only) | | `cost` | Token usage and cost tracking configuration | -| `runtime` | Provider (`copilot`, `claude`, `openai-agents`), model, temperature, max_tokens, reasoning effort, MCP servers | +| `runtime` | Provider (`copilot`, `claude`, `claude-agent-sdk`, `hermes`, `openai-agents`), model, temperature, max_tokens, reasoning effort, MCP servers | | `--web` | Real-time web dashboard with DAG graph, live streaming, in-browser human gates, sub-workflow dive-in, replay | | `checkpoint` | Auto-saved on failure; resume with `conductor resume` (run-flag parity: `--provider`, `--metadata`, `--web`, `--web-bg`, `--web-port`) | | `registry` | Named workflow sources (GitHub repo or local dir); refs accept `name@registry@version` and `workflow#ref` (tag/branch/SHA) | diff --git a/plugins/conductor/skills/conductor/references/authoring.md b/plugins/conductor/skills/conductor/references/authoring.md index a3a88ac0..48e7a3e4 100644 --- a/plugins/conductor/skills/conductor/references/authoring.md +++ b/plugins/conductor/skills/conductor/references/authoring.md @@ -12,7 +12,7 @@ workflow: entry_point: first_agent # Required: starting agent, parallel group, or for-each group runtime: - provider: copilot # copilot (default), claude, or openai-agents + provider: copilot # copilot (default), claude, claude-agent-sdk, hermes (experimental), or openai-agents default_model: gpt-5.2 # Default model for agents temperature: 0.7 # 0.0-1.0 (optional) max_tokens: 4096 # Max output tokens per response (optional) @@ -164,6 +164,7 @@ agents: - **Copilot**: forwarded as `reasoning_effort` on the session. Validated against the model's advertised `supported_reasoning_efforts`; raises `ValidationError` for unsupported combinations (skipped in mock-handler mode or when capability metadata is absent). - **Claude**: enables extended thinking via `thinking={"type": "enabled", "budget_tokens": N}` with mapping `low=2048`, `medium=8192`, `high=16384`, `xhigh=32768`. Auto-coerces `temperature` to `1.0` (logged at INFO) and bumps `max_tokens` to fit `budget + 4096` (capped at 64000, logged at INFO when clamped). Only valid on thinking-capable models (`claude-3-7-*`, `claude-opus-4*`, `claude-sonnet-4*`, `claude-haiku-4*`); raises `ValidationError` otherwise. +- **Hermes**: forwarded to the hermes-agent library via `reasoning_config={"effort": value}`. Support depends on the underlying model and hermes version. Both providers surface reasoning content via `agent_reasoning` events visible in the dashboard, JSONL logs, and the console at `-vv`. Not allowed on `script`, `human_gate`, `workflow`, or `wait` agent types. @@ -805,8 +806,9 @@ agents: - name: agent input: - workflow.input.question - - other_agent.output.result # Required - - optional_agent.output? # Optional (? suffix) + - other_agent.output.result # Required field + - other_agent.output.nested.field # Nested projection (deep path) + - optional_agent.output? # Optional (? suffix) ``` ## Multi-Provider Workflows @@ -825,6 +827,11 @@ agents: model: claude-haiku-4.5 prompt: "Classify: {{ workflow.input.text }}" + - name: tool_using_agent + provider: hermes # Uses Hermes (NousResearch agent SDK) + model: anthropic/claude-sonnet-4 + prompt: "Use tools to research: {{ workflow.input.topic }}" + - name: deep_analyzer # Uses default copilot provider model: gpt-5.2 diff --git a/plugins/conductor/skills/conductor/references/execution.md b/plugins/conductor/skills/conductor/references/execution.md index 9a9e35ca..e5a8bd77 100644 --- a/plugins/conductor/skills/conductor/references/execution.md +++ b/plugins/conductor/skills/conductor/references/execution.md @@ -17,7 +17,7 @@ conductor run [OPTIONS] | `--input`, `-i NAME=VALUE` | Workflow input (repeatable) | | `--input.NAME=VALUE` | Alternative input syntax | | `--metadata`, `-m KEY=VALUE` | Workflow metadata, merged on top of YAML `metadata:` (repeatable; values stay strings) | -| `--provider`, `-p PROVIDER` | Override provider (`copilot`, `claude`, `openai-agents`) | +| `--provider`, `-p PROVIDER` | Override provider (`copilot`, `claude`, `claude-agent-sdk`, `hermes`, `openai-agents`) | | `--dry-run` | Show execution plan only | | `--skip-gates` | Auto-select first option at human gates | | `--web` | Start real-time web dashboard | @@ -221,7 +221,7 @@ Performs both schema and **semantic** checks: - Parallel group agent references - For-each `source` format and reserved names - Stale agent references and undeclared explicit-mode dependencies in `prompt`, `system_prompt`, `command`, `args`, `working_dir`, `input_mapping`, parallel-group inputs, and workflow `output:` templates -- Warning when an agent defines `system_prompt` but no `prompt:` (portability hazard since the Claude provider drops `system_prompt`) +- Warning when an agent defines `system_prompt` but no `prompt:` (unusual — system prompts are only meaningful alongside a user prompt) The success summary table includes Parallel Groups and For-each Groups counts. @@ -556,6 +556,7 @@ If the workflow file has changed since the checkpoint was saved, a warning is di ```bash conductor run workflow.yaml -p claude # Use Claude for all agents conductor run workflow.yaml -p copilot # Use Copilot (default) +conductor run workflow.yaml -p hermes # Use Hermes (NousResearch agent SDK) conductor run workflow.yaml -p openai-agents # Use OpenAI Agents SDK ``` diff --git a/plugins/conductor/skills/conductor/references/yaml-schema.md b/plugins/conductor/skills/conductor/references/yaml-schema.md index a31f183f..e7354770 100644 --- a/plugins/conductor/skills/conductor/references/yaml-schema.md +++ b/plugins/conductor/skills/conductor/references/yaml-schema.md @@ -27,11 +27,11 @@ workflow: # Runtime configuration runtime: - provider: string | object # "copilot" (default), "claude", "claude-agent-sdk", or "openai-agents" + provider: string | object # "copilot" (default), "claude", "claude-agent-sdk", "hermes", or "openai-agents" # — or a ProviderSettings object (see below) default_model: string # Default model for all agents - temperature: float # 0.0-1.0, controls randomness (optional, copilot/claude only) - max_tokens: integer # Max OUTPUT tokens per response, 1-200000 (optional, copilot/claude only) + temperature: float # 0.0-1.0, controls randomness (optional, copilot/claude/hermes) + max_tokens: integer # Max OUTPUT tokens per response, 1-200000 (optional, copilot/claude/hermes) timeout: float # Per-request timeout in seconds (optional, default: 600, copilot/claude only) max_agent_iterations: integer # Max tool-use roundtrips per agent (1-500, optional) max_session_seconds: float # Wall-clock timeout per agent session in seconds (optional) @@ -107,11 +107,12 @@ agents: type: string # "agent" (default), "human_gate", "script", "workflow", "wait", or "terminate" description: string # What this agent does model: string # Override default_model - provider: string # Per-agent provider override ("copilot", "claude", or "claude-agent-sdk") + provider: string # Per-agent provider override ("copilot", "claude", "claude-agent-sdk", or "hermes") # Input specification (for explicit context mode) input: - string # Reference paths, e.g., "workflow.input.question" + # Nested projection: "agent.output.field.subfield" # Use "?" suffix for optional: "other_agent.output?" # Prompt templates @@ -138,7 +139,7 @@ agents: # Agent-level tools tools: # null = all workflow tools, [] = none, [list] = subset - - string + - string # Hermes: non-empty lists rejected (use hermes_toolsets in ProviderSettings) # Agent-level limits (override workflow runtime defaults) max_agent_iterations: integer # Max tool-use roundtrips for this agent (1-500, optional) @@ -212,8 +213,9 @@ agents: - **Copilot**: forwards `reasoning_effort` to the session. Validated against the model's advertised `supported_reasoning_efforts` (when available); raises `ValidationError` for unsupported combinations. - **Claude**: enables extended thinking via `thinking={"type":"enabled","budget_tokens":N}` with mapping low=2048, medium=8192, high=16384, xhigh=32768. Auto-coerces `temperature=1.0` (Anthropic API requirement) and bumps `max_tokens` to fit `budget+4096` (capped at 64000). Only valid on thinking-capable models (Claude 3.7+, Opus/Sonnet/Haiku 4.x); raises `ValidationError` otherwise. +- **Hermes**: forwarded to hermes-agent via `reasoning_config={"effort": value}`. Support depends on the underlying model and hermes version. -Both providers continue to surface reasoning content via `agent_reasoning` events visible in the dashboard, JSONL logs, and console at `-vv`. +All three providers surface reasoning content via `agent_reasoning` events visible in the dashboard, JSONL logs, and console at `-vv`. Forbidden on agent types: `script`, `human_gate`, `workflow`, `wait`. @@ -696,15 +698,17 @@ endpoints (Ollama, vLLM, LM Studio, Azure OpenAI, etc.). ```yaml runtime: provider: - name: string # "copilot" (default), "claude", "openai-agents" + name: string # "copilot" (default), "claude", "hermes", "openai-agents" type: string # "openai" | "azure" | "anthropic" (Copilot-only) wire_api: string # "completions" | "responses" (Copilot-only) - base_url: string # Endpoint base URL - api_key: string # SecretStr; redacted in dumps. Prefer ${OPENAI_API_KEY}. - bearer_token: string # SecretStr; takes precedence over api_key. + base_url: string # Endpoint base URL (copilot + hermes) + api_key: string # SecretStr; redacted in dumps. (copilot + hermes) + bearer_token: string # SecretStr; takes precedence over api_key. (Copilot-only) headers: {string: string} # Extra HTTP headers (Copilot-only) azure: # Azure-specific options (requires type: azure) api_version: string # e.g. "2024-10-21" + hermes_home: string # Path to Hermes home directory/profile (Hermes-only) + hermes_toolsets: [string] # Hermes toolset names to enable (Hermes-only; null=all, []=none) ``` ### Local OpenAI-compatible endpoint (Ollama) @@ -754,7 +758,7 @@ OpenAI credentials to arbitrary `base_url`); use the The schema rejects these misconfigurations at config-load time: -- `name != "copilot"` with any non-`name` field set +- `name == "claude"` (or other non-copilot/hermes) with any non-`name` field set (`base_url`/`api_key` are supported for `copilot` and `hermes`; structured config for other providers is not yet implemented) - `type: azure` without `azure: { api_version: ... }` (or vice versa) - Anchorless fields: `wire_api`, `type`, `headers`, `azure` alone without `base_url` / `api_key` / `bearer_token` diff --git a/src/conductor/config/schema.py b/src/conductor/config/schema.py index 627f391e..aa88c8b9 100644 --- a/src/conductor/config/schema.py +++ b/src/conductor/config/schema.py @@ -574,7 +574,7 @@ class AgentDef(BaseModel): ) = None """Agent type. Defaults to 'agent' if not specified.""" - provider: Literal["copilot", "claude", "claude-agent-sdk"] | None = None + provider: Literal["copilot", "claude", "claude-agent-sdk", "hermes"] | None = None """Provider override for this agent. If None (default), the agent uses the workflow.runtime.provider. @@ -583,6 +583,7 @@ class AgentDef(BaseModel): Example: provider: claude # Use Claude for this agent + provider: hermes # Use Hermes Agent for this agent """ model: str | None = None @@ -1488,7 +1489,7 @@ class ProviderSettings(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) - name: Literal["copilot", "openai-agents", "claude", "claude-agent-sdk"] = "copilot" + name: Literal["copilot", "openai-agents", "claude", "claude-agent-sdk", "hermes"] = "copilot" """SDK provider to use for agent execution.""" type: Literal["openai", "azure", "anthropic"] | None = None @@ -1518,6 +1519,23 @@ class ProviderSettings(BaseModel): """Bearer token. Takes precedence over ``api_key`` when both are set. Copilot-only.""" + auth_token: SecretStr | None = None + """Bearer token for OAuth / gateway authentication. Claude-only. + + Sent as ``Authorization: Bearer `` by the Anthropic SDK instead + of the usual ``x-api-key`` header. Use for Databricks AI Gateway, + LiteLLM proxies, or any endpoint that expects a bearer token. + + Falls back to ``ANTHROPIC_AUTH_TOKEN`` env var when not set in YAML. + + Example:: + + provider: + name: claude + base_url: https://my-gateway.example.com/api/v1 + auth_token: ${DATABRICKS_TOKEN} + """ + headers: dict[str, str] | None = None """Extra HTTP headers to send with every request. Copilot-only.""" @@ -1525,6 +1543,43 @@ class ProviderSettings(BaseModel): """Azure-specific options (e.g. ``api_version``). Requires ``type: azure``. Copilot-only.""" + hermes_home: str | None = None + """Path to a Hermes home directory (profile). Hermes-only. + + When set, the Hermes provider loads its config (soul, memory, toolsets) + from this path instead of the default ``~/.hermes``. Supports + ``${ENV_VAR}`` interpolation. + + Example: + hermes_home: ~/.hermes-research + """ + + hermes_toolsets: list[str] | None = None + """Hermes toolset names to enable for all agents. Hermes-only. + + When set, restricts which Hermes toolsets are available during agent + execution. ``None`` (default) = Hermes uses all available toolsets. + Empty list = no tools at all. + + Example: + hermes_toolsets: [filesystem, web] + """ + + hermes_skip_memory: bool | None = None + """Skip loading Hermes memory files during agent initialization. Hermes-only. + + ``None`` (default) = the hermes-agent library default applies (memory is loaded). + Set to ``True`` to explicitly disable memory for stateless workflows. + """ + + hermes_skip_context_files: bool | None = None + """Skip loading Hermes context/soul files during agent initialization. Hermes-only. + + ``None`` (default) = the hermes-agent library default applies (context files + including SOUL.md are loaded, preserving the agent's persona). + Set to ``True`` to explicitly disable context file loading. + """ + @model_validator(mode="after") def _check_field_compatibility(self) -> ProviderSettings: copilot_only_fields = { @@ -1534,6 +1589,9 @@ def _check_field_compatibility(self) -> ProviderSettings: "headers": self.headers, "azure": self.azure, } + claude_only_fields = { + "auth_token": self.auth_token, + } if self.name != "copilot": extras = sorted(k for k, v in copilot_only_fields.items() if v is not None) if extras: @@ -1541,11 +1599,29 @@ def _check_field_compatibility(self) -> ProviderSettings: f"Provider fields {extras} are only supported when name='copilot'. " "Structured provider config for other providers is not yet implemented." ) - if self.base_url is not None or self.api_key is not None: - raise ValueError( - f"Structured provider config (base_url/api_key) for name='{self.name}' " - "is not yet implemented; use environment variables for the underlying SDK." - ) + if self.name not in ("copilot", "claude", "hermes") and ( + self.base_url is not None or self.api_key is not None + ): + raise ValueError( + f"Structured provider config (base_url/api_key) for name='{self.name}' " + "is not yet implemented; use environment variables for the underlying SDK." + ) + if self.name != "claude": + extras = sorted(k for k, v in claude_only_fields.items() if v is not None) + if extras: + raise ValueError(f"Provider fields {extras} are only supported when name='claude'.") + + if self.hermes_home is not None and self.name != "hermes": + raise ValueError("'hermes_home' is only supported when name='hermes'.") + + if self.hermes_toolsets is not None and self.name != "hermes": + raise ValueError("'hermes_toolsets' is only supported when name='hermes'.") + + if self.hermes_skip_memory is not None and self.name != "hermes": + raise ValueError("'hermes_skip_memory' is only supported when name='hermes'.") + + if self.hermes_skip_context_files is not None and self.name != "hermes": + raise ValueError("'hermes_skip_context_files' is only supported when name='hermes'.") if self.azure is not None and self.type != "azure": raise ValueError("'azure' options require type='azure'") @@ -1558,7 +1634,11 @@ def _check_field_compatibility(self) -> ProviderSettings: raise ValueError( "'headers' must contain at least one entry; remove the key to omit headers" ) - for secret_field, value in (("api_key", self.api_key), ("bearer_token", self.bearer_token)): + for secret_field, value in ( + ("api_key", self.api_key), + ("bearer_token", self.bearer_token), + ("auth_token", self.auth_token), + ): if value is not None and value.get_secret_value() == "": raise ValueError( f"'{secret_field}' is empty; remove the key or supply a value " @@ -1607,6 +1687,7 @@ def has_custom_routing(self) -> bool: self.base_url, self.api_key, self.bearer_token, + self.auth_token, self.headers, self.azure, ) diff --git a/src/conductor/providers/capabilities.py b/src/conductor/providers/capabilities.py index 2b578341..5c329417 100644 --- a/src/conductor/providers/capabilities.py +++ b/src/conductor/providers/capabilities.py @@ -204,6 +204,7 @@ def declared_limitations(self) -> list[str]: "copilot": "conductor.providers.copilot:CopilotProvider", "claude": "conductor.providers.claude:ClaudeProvider", "claude-agent-sdk": "conductor.providers.claude_agent_sdk:ClaudeAgentSdkProvider", + "hermes": "conductor.providers.hermes:HermesProvider", } # Provider names that appear in the schema / factory but are not yet diff --git a/src/conductor/providers/claude.py b/src/conductor/providers/claude.py index 7b36622c..65689aae 100644 --- a/src/conductor/providers/claude.py +++ b/src/conductor/providers/claude.py @@ -163,6 +163,8 @@ class ClaudeProvider(AgentProvider): def __init__( self, api_key: str | None = None, + auth_token: str | None = None, + base_url: str | None = None, model: str | None = None, temperature: float | None = None, max_tokens: int | None = None, @@ -177,6 +179,14 @@ def __init__( Args: api_key: Anthropic API key. If None, uses ANTHROPIC_API_KEY env var. + auth_token: Bearer token for OAuth / gateway authentication. Sent as + ``Authorization: Bearer `` instead of ``x-api-key``. + If None, falls back to ANTHROPIC_AUTH_TOKEN env var (SDK-native). + Use this for Databricks AI Gateway, LiteLLM, or any proxy that + expects a bearer token rather than a raw API key. + base_url: Custom API endpoint (e.g. Databricks gateway URL). + If None, falls back to ANTHROPIC_BASE_URL env var then the + default Anthropic API endpoint. model: Default model to use. Defaults to "claude-3-5-sonnet-latest". This default is chosen for stability and to avoid dated model deprecation risk. The "-latest" suffix ensures compatibility @@ -209,6 +219,8 @@ def __init__( self._client: AsyncAnthropic | None = None self._api_key = api_key + self._auth_token = auth_token + self._base_url = base_url self._default_model = model or "claude-3-5-sonnet-latest" # Validate and store temperature (enforce schema bounds at instantiation) @@ -256,10 +268,14 @@ def _initialize_client(self) -> None: if not ANTHROPIC_SDK_AVAILABLE or AsyncAnthropic is None: return - self._client = AsyncAnthropic( - api_key=self._api_key, - timeout=self._timeout, - ) + client_kwargs: dict[str, Any] = {"timeout": self._timeout} + if self._api_key is not None: + client_kwargs["api_key"] = self._api_key + if self._auth_token is not None: + client_kwargs["auth_token"] = self._auth_token + if self._base_url is not None: + client_kwargs["base_url"] = self._base_url + self._client = AsyncAnthropic(**client_kwargs) # Log SDK version if anthropic is not None: diff --git a/src/conductor/providers/factory.py b/src/conductor/providers/factory.py index b36c34c3..2738bef9 100644 --- a/src/conductor/providers/factory.py +++ b/src/conductor/providers/factory.py @@ -17,14 +17,18 @@ ) from conductor.providers.context_tier import ContextTier from conductor.providers.copilot import CopilotProvider, IdleRecoveryConfig +from conductor.providers.hermes import HERMES_SDK_AVAILABLE, HermesProvider from conductor.providers.reasoning import ReasoningEffort if TYPE_CHECKING: from conductor.config.schema import ProviderSettings +ProviderType = Literal["copilot", "openai-agents", "claude", "claude-agent-sdk", "hermes"] + + async def create_provider( - provider_type: Literal["copilot", "openai-agents", "claude", "claude-agent-sdk"] = "copilot", + provider_type: ProviderType = "copilot", validate: bool = True, mcp_servers: dict[str, Any] | None = None, default_model: str | None = None, @@ -107,6 +111,12 @@ async def create_provider( "Claude provider requires anthropic SDK", suggestion="Install with: uv add 'anthropic>=0.77.0,<1.0.0'", ) + claude_auth_token: str | None = None + claude_base_url: str | None = None + if provider_settings is not None and provider_settings.name == "claude": + if provider_settings.auth_token is not None: + claude_auth_token = provider_settings.auth_token.get_secret_value() + claude_base_url = provider_settings.base_url provider = ClaudeProvider( model=default_model, temperature=temperature, @@ -116,6 +126,42 @@ async def create_provider( max_agent_iterations=max_agent_iterations, max_session_seconds=max_session_seconds, default_reasoning_effort=default_reasoning_effort, + auth_token=claude_auth_token, + base_url=claude_base_url, + ) + case "hermes": + if not HERMES_SDK_AVAILABLE: + raise ProviderError( + "Hermes provider requires the hermes-agent package", + suggestion="Install with: pip install hermes-agent", + ) + hermes_base_url: str | None = None + hermes_api_key: str | None = None + hermes_home: str | None = None + hermes_toolsets: list[str] | None = None + hermes_skip_memory: bool | None = None + hermes_skip_context_files: bool | None = None + if provider_settings is not None and provider_settings.name == "hermes": + hermes_base_url = provider_settings.base_url + if provider_settings.api_key is not None: + hermes_api_key = provider_settings.api_key.get_secret_value() + hermes_home = provider_settings.hermes_home + hermes_toolsets = provider_settings.hermes_toolsets + hermes_skip_memory = provider_settings.hermes_skip_memory + hermes_skip_context_files = provider_settings.hermes_skip_context_files + provider = HermesProvider( + model=default_model, + max_tokens=max_tokens, + temperature=temperature, + base_url=hermes_base_url, + api_key=hermes_api_key, + hermes_home=hermes_home, + hermes_toolsets=hermes_toolsets, + skip_memory=hermes_skip_memory, + skip_context_files=hermes_skip_context_files, + max_agent_iterations=max_agent_iterations, + max_session_seconds=max_session_seconds, + default_reasoning_effort=default_reasoning_effort, ) case "claude-agent-sdk": if not CLAUDE_AGENT_SDK_AVAILABLE: @@ -162,7 +208,9 @@ async def create_provider( case _: raise ProviderError( f"Unknown provider: {provider_type}", - suggestion="Valid providers are: copilot, openai-agents, claude, claude-agent-sdk", + suggestion=( + "Valid providers are: copilot, openai-agents, claude, claude-agent-sdk, hermes" + ), ) if validate and not await provider.validate_connection(): diff --git a/src/conductor/providers/hermes.py b/src/conductor/providers/hermes.py new file mode 100644 index 00000000..74b37758 --- /dev/null +++ b/src/conductor/providers/hermes.py @@ -0,0 +1,656 @@ +"""Hermes Agent provider implementation. + +This module provides the HermesProvider class for executing agents +using the hermes-agent Python library (NousResearch/hermes-agent). + +The library is an optional dependency — install with: + pip install hermes-agent + +Error Handling Strategy: +- ValidationError: Invalid inputs or output schema violations. +- ProviderError: Library failures, API errors, or unexpected result states. +""" + +from __future__ import annotations + +import asyncio +import contextlib +import json +import logging +import tempfile +from pathlib import Path +from typing import TYPE_CHECKING, Any + +from conductor.exceptions import ProviderError, ValidationError +from conductor.executor.output import parse_json_output, validate_output +from conductor.providers.base import AgentOutput, AgentProvider, EventCallback +from conductor.providers.capabilities import ProviderCapabilities +from conductor.providers.reasoning import ReasoningEffort, resolve_reasoning_effort + +if TYPE_CHECKING: + from conductor.config.schema import AgentDef + +# The hermes-agent package ships its public API under the top-level module +# name "run_agent" (not "hermes_agent"). Catch only ModuleNotFoundError so +# that real dependency failures inside the package (e.g. missing openai) +# propagate instead of producing a misleading "install hermes-agent" hint. +try: + from run_agent import AIAgent # ty: ignore[unresolved-import] + + HERMES_SDK_AVAILABLE = True +except ModuleNotFoundError as _e: + if _e.name is not None and _e.name.split(".")[0] == "run_agent": + HERMES_SDK_AVAILABLE = False + AIAgent: Any = None + else: + raise + +logger = logging.getLogger(__name__) + +# Maximum number of recovery attempts when JSON parsing fails. +_MAX_PARSE_RECOVERY_ATTEMPTS = 3 + +# Maximum schema nesting depth for prompt schema generation. +_MAX_SCHEMA_DEPTH = 10 + + +class HermesProvider(AgentProvider): + """Hermes Agent SDK provider. + + Translates Conductor agent definitions into hermes-agent library calls and + normalizes responses into AgentOutput format. + + Requires the hermes-agent package: + pip install hermes-agent + + Example: + >>> provider = HermesProvider() + >>> await provider.validate_connection() + True + >>> await provider.close() + """ + + CAPABILITIES = ProviderCapabilities( + tier="experimental", + mcp_tools=False, + workflow_tools_passthrough=False, + streaming_events=True, + agent_reasoning_events=True, + reasoning_effort=("low", "medium", "high", "xhigh"), + structured_output="prompt_injection", + interrupt=False, + max_session_seconds=True, + checkpoint_resume=True, + usage_tracking=True, + concurrent_safe=True, + upstream_pin="hermes-agent", + maintainer="(community contribution)", + ) + + def __init__( + self, + model: str | None = None, + max_tokens: int | None = None, + temperature: float | None = None, + base_url: str | None = None, + api_key: str | None = None, + hermes_home: str | None = None, + hermes_toolsets: list[str] | None = None, + skip_memory: bool | None = None, + skip_context_files: bool | None = None, + max_agent_iterations: int | None = None, + max_session_seconds: float | None = None, + default_reasoning_effort: ReasoningEffort | None = None, + ) -> None: + """Initialize the Hermes provider. + + Args: + model: Default model in hermes/OpenRouter format, e.g. + ``"anthropic/claude-sonnet-4"`` or ``"openai/gpt-4o"``. + If None, uses whatever model hermes has configured. + max_tokens: Maximum output tokens forwarded to ``AIAgent``. + temperature: Sampling temperature forwarded to ``AIAgent``. + base_url: Override endpoint base URL, e.g. OpenRouter. + Forwarded to ``AIAgent`` when set. + api_key: API key for the endpoint. Forwarded to ``AIAgent`` + when set. Use ``${ENV_VAR}`` interpolation in YAML so the + literal value never appears in event logs. + hermes_home: Path to a Hermes home directory (profile). When + set, hermes loads config/soul/memory from this path instead + of ``~/.hermes``. Thread-safe via ``ContextVar`` override. + hermes_toolsets: Hermes toolset names to enable (e.g. + ``["filesystem", "web"]``). None = hermes defaults (all + available toolsets); empty list = no tools. + skip_memory: Whether to skip loading Hermes memory files. + None (default) = hermes library default (False, memory is + loaded). Set True to disable memory for stateless workflows. + skip_context_files: Whether to skip loading context/soul files. + None (default) = hermes library default (False, SOUL.md and + context files are loaded, preserving persona). Set True to + disable context file loading. + max_agent_iterations: Maximum tool-calling iterations per agent + execution. Maps to hermes ``max_iterations``. Defaults to + 90 (hermes default) when None. + max_session_seconds: Maximum wall-clock duration for agent sessions. + Not directly supported by the hermes library — used only to + impose an ``asyncio.wait_for`` timeout around each call. + default_reasoning_effort: Workflow-wide default reasoning effort. + Forwarded to hermes via ``reasoning_config``. + """ + if not HERMES_SDK_AVAILABLE: + raise ProviderError( + "Hermes provider requires the hermes-agent package", + suggestion="Install with: pip install hermes-agent", + ) + + self._default_model = model + self._default_max_tokens = max_tokens + self._default_temperature = temperature + self._base_url = base_url + self._api_key = api_key + self._hermes_home = hermes_home + self._hermes_toolsets = hermes_toolsets + self._skip_memory = skip_memory + self._skip_context_files = skip_context_files + self._default_max_agent_iterations = max_agent_iterations + self._default_max_session_seconds = max_session_seconds + self._default_reasoning_effort = default_reasoning_effort + + # Session state for checkpoint resume. Maps agent name → path to a + # JSON file containing the conversation history from the last run. + self._session_ids: dict[str, str] = {} + self._resume_session_ids: dict[str, str] = {} + self._session_dir = Path(tempfile.gettempdir()) / "conductor" / "hermes-sessions" + + async def execute( + self, + agent: AgentDef, + context: dict[str, Any], + rendered_prompt: str, + tools: list[str] | None = None, + interrupt_signal: asyncio.Event | None = None, + event_callback: EventCallback | None = None, + ) -> AgentOutput: + """Execute an agent via the hermes-agent library. + + Args: + agent: Agent definition from workflow config. + context: Accumulated workflow context (not passed to hermes directly; + context is already rendered into ``rendered_prompt``). + rendered_prompt: Jinja2-rendered user prompt. + tools: Hermes toolset names to enable (e.g. ["filesystem", "web"]). + None = hermes defaults; empty list = no tools. + interrupt_signal: Optional event that, when set, signals a + mid-agent interrupt request. Monitored during execution; + when fired the current library call is cancelled and a + ProviderError is raised. + event_callback: Optional callback for streaming events upstream. + Emits ``agent_turn_start`` (before call) and ``agent_message`` + (after response) events. + + Returns: + Normalized AgentOutput with structured content. + + Raises: + ProviderError: If the hermes library call fails or returns an + error state. + ValidationError: If output doesn't match the declared schema. + """ + # Resolve per-agent overrides + resolved_model = agent.model or self._default_model + resolved_max_iter = ( + agent.max_agent_iterations + if agent.max_agent_iterations is not None + else self._default_max_agent_iterations + ) + resolved_timeout = ( + agent.max_session_seconds + if agent.max_session_seconds is not None + else self._default_max_session_seconds + ) + + # Append schema instruction when agent declares a structured output schema + prompt = rendered_prompt + schema_for_prompt: dict[str, Any] | None = None + if agent.output: + schema_for_prompt = _build_prompt_schema(agent.output) + schema_desc = json.dumps(schema_for_prompt, indent=2) + prompt += ( + f"\n\n**IMPORTANT: You MUST respond with a JSON object matching this schema:**\n" + f"```json\n{schema_desc}\n```\n" + f"Return ONLY the JSON object, no other text." + ) + + _fire(event_callback, "agent_turn_start", {"turn": "awaiting_model"}) + + # Build AIAgent kwargs — omit model when not set to use hermes default + agent_kwargs: dict[str, Any] = { + "quiet_mode": True, + } + if self._skip_context_files is not None: + agent_kwargs["skip_context_files"] = self._skip_context_files + if self._skip_memory is not None: + agent_kwargs["skip_memory"] = self._skip_memory + if resolved_model: + agent_kwargs["model"] = resolved_model + if resolved_max_iter is not None: + agent_kwargs["max_iterations"] = resolved_max_iter + if self._default_max_tokens is not None: + agent_kwargs["max_tokens"] = self._default_max_tokens + if self._default_temperature is not None: + # AIAgent doesn't accept temperature directly; route through + # request_overrides which is applied at the transport layer. + agent_kwargs.setdefault("request_overrides", {})["temperature"] = ( + self._default_temperature + ) + if self._base_url: + agent_kwargs["base_url"] = self._base_url + if self._api_key: + agent_kwargs["api_key"] = self._api_key + + # Resolve reasoning effort (per-agent override → workflow default) + effort = resolve_reasoning_effort(agent, self._default_reasoning_effort) + if effort is not None: + agent_kwargs["reasoning_config"] = {"effort": effort} + + # Resolve enabled_toolsets. Conductor's per-agent tools: field + # contains workflow tool names (not Hermes toolset names), so a + # non-empty list cannot be forwarded. Provider-level hermes_toolsets + # gives authors the knob to restrict which Hermes toolsets are active. + if tools: + raise ProviderError( + f"Agent '{agent.name}' declares tools={tools!r}, but " + "the Hermes provider does not support per-agent workflow tool " + "allowlists (workflow tool names do not translate to Hermes " + "toolset names).", + suggestion=( + "Remove the 'tools:' field to use Hermes default toolsets, " + "set 'tools: []' to disable all tools, or configure " + "'hermes_toolsets' in the provider settings to restrict " + "which Hermes toolsets are available." + ), + ) + elif tools is not None: + # Explicit empty list = no tools + agent_kwargs["enabled_toolsets"] = [] + elif self._hermes_toolsets is not None: + # Provider-level toolset restriction + agent_kwargs["enabled_toolsets"] = self._hermes_toolsets + + # Wire streaming callbacks so events fire incrementally from the + # executor thread. The _fire helper is thread-safe (swallows errors). + if event_callback is not None: + agent_kwargs["stream_delta_callback"] = lambda text: _fire( + event_callback, "agent_message", {"content": text} + ) + agent_kwargs["reasoning_callback"] = lambda text: _fire( + event_callback, "agent_reasoning", {"content": text} + ) + + # Load conversation history from a prior checkpoint if available + conversation_history: list[dict[str, Any]] | None = None + resume_path = self._resume_session_ids.get(agent.name) + if resume_path: + try: + conversation_history = json.loads(Path(resume_path).read_text()) + logger.info( + "Resuming agent '%s' with %d prior messages", + agent.name, + len(conversation_history), + ) + except (OSError, json.JSONDecodeError) as e: + logger.warning( + "Could not load conversation history for '%s' from %s: %s", + agent.name, + resume_path, + e, + ) + + loop = asyncio.get_running_loop() + + # Keep a reference to the AIAgent so we can call interrupt() from + # the async side — hermes interrupt() is thread-safe by design. + hermes_agent_ref: list[Any] = [] + + def _run_sync() -> dict[str, Any]: + # Apply hermes_home profile override (thread-safe ContextVar) + _home_token = None + if self._hermes_home: + import hermes_constants # ty: ignore[unresolved-import] + + expanded_home = str(Path(self._hermes_home).expanduser()) + _home_token = hermes_constants.set_hermes_home_override(expanded_home) + try: + hermes_agent = AIAgent(**agent_kwargs) # ty: ignore[call-non-callable] + hermes_agent_ref.append(hermes_agent) + return hermes_agent.run_conversation( + prompt, + system_message=agent.system_prompt or None, + conversation_history=conversation_history, + ) + finally: + if _home_token is not None: + import hermes_constants + + hermes_constants.reset_hermes_home_override(_home_token) + + # Wrap the blocking call and optionally race against interrupt / timeout + call_task = loop.run_in_executor(None, _run_sync) + + awaitables: list[Any] = [call_task] + interrupt_task = None + if interrupt_signal is not None: + interrupt_task = asyncio.ensure_future(_wait_for_event(interrupt_signal)) + awaitables.append(interrupt_task) + + try: + if resolved_timeout is not None: + done, pending = await asyncio.wait( + awaitables, + timeout=resolved_timeout, + return_when=asyncio.FIRST_COMPLETED, + ) + else: + done, pending = await asyncio.wait( + awaitables, + return_when=asyncio.FIRST_COMPLETED, + ) + finally: + # Clean up the interrupt watcher regardless of outcome + if interrupt_task is not None and not interrupt_task.done(): + interrupt_task.cancel() + + if call_task not in done: + # Either timeout or interrupt fired first. Signal hermes to stop + # cooperatively — it checks _interrupt_requested between iterations. + if hermes_agent_ref: + hermes_agent_ref[0].interrupt() + call_task.cancel() + if interrupt_task is not None and interrupt_task in done: + raise ProviderError( + f"Agent '{agent.name}' was interrupted by user request", + is_retryable=False, + ) + raise ProviderError( + f"Agent '{agent.name}' exceeded maximum session duration " + f"of {resolved_timeout:.0f}s", + is_retryable=True, + ) + + try: + result = call_task.result() + except asyncio.CancelledError: + raise + except Exception as e: + raise ProviderError( + f"Hermes agent execution failed (model='{resolved_model}'): {e}", + suggestion="Check the hermes-agent logs and verify base_url/api_key.", + ) from e + + # Surface library-level failures as ProviderError + if result.get("failed"): + error_msg = result.get("error") or "hermes agent run failed" + raise ProviderError( + f"Hermes agent execution failed (model='{resolved_model}'): {error_msg}", + ) + + final_response: str | None = result.get("final_response") + if final_response is None: + partial_error = result.get("error") or "no final response returned" + raise ProviderError( + f"Hermes agent returned no final response: {partial_error}", + ) + + # If no output schema, wrap as plain text and we're done + if not agent.output: + content: dict[str, Any] = {"text": final_response} + else: + # Try to parse as JSON with recovery loop (mirrors Copilot pattern) + content = await self._parse_with_recovery( + final_response, + result.get("messages", []), + schema_for_prompt, # type: ignore[arg-type] + agent_kwargs, + agent, + conversation_history, + loop, + ) + + # Populate token counts from the result dict when available. + # Use explicit None-check (not `or`) so legitimate zero is preserved. + input_tokens: int | None = result.get("input_tokens") + if input_tokens is None: + input_tokens = result.get("prompt_tokens") + output_tokens: int | None = result.get("output_tokens") + if output_tokens is None: + output_tokens = result.get("completion_tokens") + tokens_used: int | None = result.get("total_tokens") + if tokens_used is None and input_tokens is not None and output_tokens is not None: + tokens_used = input_tokens + output_tokens + + # Use the actual model reported by hermes (may differ from requested) + actual_model = result.get("model") or resolved_model + + # Persist conversation history for checkpoint resume + messages = result.get("messages") + if messages: + self._save_session(agent.name, messages) + + return AgentOutput( + content=content, + raw_response={ + "final_response": final_response, + "messages": result.get("messages", []), + "api_calls": result.get("api_calls"), + "completed": result.get("completed"), + "partial": result.get("partial", False), + "model": result.get("model"), + "provider": result.get("provider"), + }, + tokens_used=tokens_used, + input_tokens=input_tokens, + output_tokens=output_tokens, + model=actual_model, + partial=bool(result.get("partial", False)), + ) + + async def validate_connection(self) -> bool: + """Confirms the hermes-agent SDK is importable. + + The import is performed at module load and reflected in the + module-level ``HERMES_SDK_AVAILABLE`` constant; this method + simply returns that value. Does NOT verify the configured + ``base_url``/``api_key`` are reachable — credential and endpoint + failures surface only at first agent execution. + """ + return HERMES_SDK_AVAILABLE + + async def close(self) -> None: + """No-op — the hermes provider is stateless (no persistent sessions).""" + + # ------------------------------------------------------------------ + # Structured output: parse with recovery (mirrors Copilot pattern) + # ------------------------------------------------------------------ + + async def _parse_with_recovery( + self, + response: str, + messages: list[dict[str, Any]], + schema: dict[str, Any], + agent_kwargs: dict[str, Any], + agent: AgentDef, + conversation_history: list[dict[str, Any]] | None, + loop: asyncio.AbstractEventLoop, + ) -> dict[str, Any]: + """Parse response as JSON, retrying via conversation if parsing fails.""" + last_error: str | None = None + + for attempt in range(_MAX_PARSE_RECOVERY_ATTEMPTS + 1): + try: + content = parse_json_output(response) + validate_output(content, agent.output) # type: ignore[arg-type] + return content + except (json.JSONDecodeError, ValueError, ValidationError) as e: + last_error = str(e) + if attempt >= _MAX_PARSE_RECOVERY_ATTEMPTS: + break + + logger.info( + "Agent '%s' parse recovery attempt %d/%d: %s", + agent.name, + attempt + 1, + _MAX_PARSE_RECOVERY_ATTEMPTS, + last_error, + ) + + # Build recovery prompt and re-run with conversation history + recovery_prompt = _build_recovery_prompt(last_error, response, schema) + # Use the messages from the failed run as history + history = messages if messages else conversation_history + + recovery_kwargs = { + k: v + for k, v in agent_kwargs.items() + if k not in ("stream_delta_callback", "reasoning_callback") + } + + def _run_recovery( + prompt: str = recovery_prompt, + sys_msg: str | None = agent.system_prompt or None, + hist: list[dict[str, Any]] | None = history, + kwargs: dict[str, Any] = recovery_kwargs, + ) -> dict[str, Any]: + _home_token = None + if self._hermes_home: + import hermes_constants # ty: ignore[unresolved-import] + + expanded_home = str(Path(self._hermes_home).expanduser()) + _home_token = hermes_constants.set_hermes_home_override(expanded_home) + try: + hermes_agent = AIAgent(**kwargs) # ty: ignore[call-non-callable] + return hermes_agent.run_conversation( + prompt, + system_message=sys_msg, + conversation_history=hist, + ) + finally: + if _home_token is not None: + import hermes_constants + + hermes_constants.reset_hermes_home_override(_home_token) + + try: + recovery_result = await loop.run_in_executor(None, _run_recovery) + response = recovery_result.get("final_response") or "" + messages = recovery_result.get("messages", []) + except (json.JSONDecodeError, ValueError, ValidationError): + raise + except Exception as e: + raise ProviderError( + f"Hermes recovery call failed for agent '{agent.name}': {e}", + suggestion="Check hermes-agent logs and verify base_url/api_key.", + ) from e + + expected_fields = list(agent.output.keys()) if agent.output else [] + raise ProviderError( + f"Failed to parse structured output after {_MAX_PARSE_RECOVERY_ATTEMPTS} " + f"recovery attempts: {last_error}", + suggestion=( + f"Agent was expected to return JSON with fields: {expected_fields}. " + "Consider simplifying the output schema or making the prompt more explicit." + ), + ) + + # ------------------------------------------------------------------ + # Session state for checkpoint resume + # ------------------------------------------------------------------ + + def _save_session(self, agent_name: str, messages: list[dict[str, Any]]) -> None: + """Persist conversation history to a temp file for checkpoint resume.""" + self._session_dir.mkdir(parents=True, exist_ok=True) + session_file = self._session_dir / f"{agent_name}.json" + try: + session_file.write_text(json.dumps(messages, ensure_ascii=False)) + self._session_ids[agent_name] = str(session_file) + except OSError as e: + logger.warning("Failed to save hermes session for '%s': %s", agent_name, e) + + def get_session_ids(self) -> dict[str, str]: + """Return mapping of agent names to session file paths.""" + return self._session_ids.copy() + + def set_resume_session_ids(self, ids: dict[str, str]) -> None: + """Set session file paths for resuming conversations on next execution.""" + self._resume_session_ids = dict(ids) + + def cleanup_sessions(self) -> None: + """Remove all saved session files.""" + for path_str in self._session_ids.values(): + with contextlib.suppress(OSError): + Path(path_str).unlink(missing_ok=True) + self._session_ids.clear() + if self._session_dir.exists(): + with contextlib.suppress(OSError): + self._session_dir.rmdir() + + +def _fire(callback: EventCallback | None, event: str, data: dict[str, Any]) -> None: + """Call event_callback safely, swallowing any exception.""" + if callback is None: + return + try: + callback(event, data) + except Exception: + logger.warning("Error in event_callback for %s", event, exc_info=True) + + +async def _wait_for_event(event: asyncio.Event) -> None: + """Coroutine that resolves when the given asyncio.Event is set.""" + await event.wait() + + +def _build_prompt_schema(schema: dict[str, Any], depth: int = 0) -> dict[str, Any]: + """Build a prompt-facing schema description from OutputField definitions.""" + if depth > _MAX_SCHEMA_DEPTH: + raise ValidationError( + f"Schema nesting depth exceeds maximum of {_MAX_SCHEMA_DEPTH} levels", + suggestion="Simplify your output schema to reduce nesting depth", + ) + result: dict[str, Any] = {} + for field_name, field_def in schema.items(): + field_schema: dict[str, Any] = {"type": field_def.type} + if field_def.description: + field_schema["description"] = field_def.description + else: + field_schema["description"] = f"The {field_name} field" + if field_def.type == "object" and field_def.properties: + field_schema["properties"] = _build_prompt_schema(field_def.properties, depth + 1) + field_schema["required"] = list(field_def.properties.keys()) + if field_def.type == "array" and field_def.items: + item_schema: dict[str, Any] = {"type": field_def.items.type} + if field_def.items.description: + item_schema["description"] = field_def.items.description + if field_def.items.type == "object" and field_def.items.properties: + item_schema["properties"] = _build_prompt_schema( + field_def.items.properties, depth + 1 + ) + field_schema["items"] = item_schema + result[field_name] = field_schema + return result + + +def _build_recovery_prompt(parse_error: str, original_response: str, schema: dict[str, Any]) -> str: + """Build a prompt to recover from JSON parse failures.""" + truncated = original_response[:500] + if len(original_response) > 500: + truncated += "..." + schema_desc = json.dumps(schema, indent=2) + return ( + f"Your previous response could not be parsed as valid JSON.\n\n" + f"**Parse Error:** {parse_error}\n\n" + f"**Your response started with:**\n```\n{truncated}\n```\n\n" + f"**Expected JSON schema:**\n```json\n{schema_desc}\n```\n\n" + f"Please respond with ONLY a valid JSON object matching the schema above. " + f"Do NOT include markdown code blocks, explanatory text, or anything other " + f"than the raw JSON object." + ) diff --git a/src/conductor/providers/registry.py b/src/conductor/providers/registry.py index 4b390276..ad622a91 100644 --- a/src/conductor/providers/registry.py +++ b/src/conductor/providers/registry.py @@ -15,7 +15,7 @@ from conductor.config.schema import AgentDef, WorkflowConfig -ProviderType = Literal["copilot", "openai-agents", "claude", "claude-agent-sdk"] +ProviderType = Literal["copilot", "openai-agents", "claude", "claude-agent-sdk", "hermes"] class ProviderRegistry: diff --git a/tests/test_config/test_backward_compatibility.py b/tests/test_config/test_backward_compatibility.py index 56ec5b60..21849f2d 100644 --- a/tests/test_config/test_backward_compatibility.py +++ b/tests/test_config/test_backward_compatibility.py @@ -43,9 +43,11 @@ def get_copilot_example_files() -> list[Path]: copilot_examples = [] for example in all_examples: - # Skip Claude-specific examples + # Skip provider-specific examples if "claude" in example.name.lower(): continue + if "hermes" in example.name.lower(): + continue copilot_examples.append(example) diff --git a/tests/test_config/test_provider_settings.py b/tests/test_config/test_provider_settings.py index a05095bb..5afc3473 100644 --- a/tests/test_config/test_provider_settings.py +++ b/tests/test_config/test_provider_settings.py @@ -74,7 +74,29 @@ def test_non_copilot_with_copilot_only_field_rejected(self) -> None: def test_non_copilot_with_base_url_rejected(self) -> None: with pytest.raises(ValidationError, match="not yet implemented"): - ProviderSettings(name="claude", base_url="http://anthropic-proxy/v1") + ProviderSettings(name="openai-agents", base_url="http://some-proxy/v1") + + def test_claude_with_base_url_accepted(self) -> None: + s = ProviderSettings(name="claude", base_url="https://my-gateway.example.com/api/v1") + assert s.base_url == "https://my-gateway.example.com/api/v1" + + def test_claude_with_auth_token_accepted(self) -> None: + s = ProviderSettings(name="claude", auth_token="dapi-abc123") + assert s.auth_token is not None + assert s.auth_token.get_secret_value() == "dapi-abc123" + + def test_claude_with_base_url_and_auth_token_accepted(self) -> None: + s = ProviderSettings( + name="claude", + base_url="https://my-gateway.example.com/api/v1", + auth_token="dapi-abc123", + ) + assert s.base_url == "https://my-gateway.example.com/api/v1" + assert s.auth_token.get_secret_value() == "dapi-abc123" + + def test_auth_token_on_non_claude_rejected(self) -> None: + with pytest.raises(ValidationError, match="only supported when name='claude'"): + ProviderSettings(name="copilot", auth_token="some-token", base_url="http://x/v1") def test_azure_options_require_azure_type(self) -> None: with pytest.raises(ValidationError, match="require type='azure'"): @@ -179,6 +201,51 @@ def test_secrets_redacted_in_dump(self) -> None: assert dumped["provider"]["api_key"] == "**********" +class TestHermesProviderSettings: + """Hermes provider accepts ``base_url`` and ``api_key`` in structured config.""" + + def test_hermes_with_base_url_accepted(self) -> None: + s = ProviderSettings(name="hermes", base_url="https://openrouter.ai/api/v1") + assert s.base_url == "https://openrouter.ai/api/v1" + assert s.has_custom_routing() + + def test_hermes_with_api_key_accepted(self) -> None: + s = ProviderSettings( + name="hermes", base_url="https://openrouter.ai/api/v1", api_key="sk-or-test" + ) + assert isinstance(s.api_key, SecretStr) + assert s.api_key.get_secret_value() == "sk-or-test" + + def test_hermes_with_base_url_and_api_key_accepted(self) -> None: + s = ProviderSettings( + name="hermes", + base_url="https://openrouter.ai/api/v1", + api_key="sk-or-test", + ) + assert s.has_custom_routing() + + def test_hermes_skip_memory_accepted(self) -> None: + s = ProviderSettings(name="hermes", hermes_skip_memory=True) + assert s.hermes_skip_memory is True + + def test_hermes_skip_context_files_accepted(self) -> None: + s = ProviderSettings(name="hermes", hermes_skip_context_files=False) + assert s.hermes_skip_context_files is False + + def test_hermes_skip_memory_rejected_for_non_hermes(self) -> None: + with pytest.raises(ValidationError, match="hermes_skip_memory"): + ProviderSettings(name="copilot", hermes_skip_memory=True) + + def test_hermes_skip_context_files_rejected_for_non_hermes(self) -> None: + with pytest.raises(ValidationError, match="hermes_skip_context_files"): + ProviderSettings(name="copilot", hermes_skip_context_files=True) + + def test_unsupported_provider_with_base_url_still_rejected(self) -> None: + # claude/copilot/hermes support base_url; other providers must still reject it. + with pytest.raises(ValidationError, match="not yet implemented"): + ProviderSettings(name="claude-agent-sdk", base_url="http://proxy/v1") + + class TestHasCustomRouting: """``has_custom_routing()`` gates env-var fallback activation.""" diff --git a/tests/test_providers/test_claude_parameter_passing.py b/tests/test_providers/test_claude_parameter_passing.py index 8b17aae6..78c2b737 100644 --- a/tests/test_providers/test_claude_parameter_passing.py +++ b/tests/test_providers/test_claude_parameter_passing.py @@ -48,6 +48,8 @@ async def test_common_parameters_passed_from_factory(self, mock_claude_class: Mo max_agent_iterations=None, max_session_seconds=None, default_reasoning_effort=None, + auth_token=None, + base_url=None, ) @patch("conductor.providers.claude.ANTHROPIC_SDK_AVAILABLE", True) diff --git a/tests/test_providers/test_factory.py b/tests/test_providers/test_factory.py index c078d676..f4ad6914 100644 --- a/tests/test_providers/test_factory.py +++ b/tests/test_providers/test_factory.py @@ -4,7 +4,9 @@ from unittest.mock import MagicMock, patch import pytest +from pydantic import SecretStr +from conductor.config.schema import ProviderSettings from conductor.exceptions import ProviderError from conductor.providers.copilot import CopilotProvider from conductor.providers.factory import create_provider @@ -297,3 +299,103 @@ async def test_factory_accepts_empty_mcp_servers(self) -> None: ) assert isinstance(provider, ClaudeAgentSdkProvider) await provider.close() + + +class TestHermesFactory: + """Tests for the hermes factory branch.""" + + @patch("conductor.providers.factory.HERMES_SDK_AVAILABLE", False) + @pytest.mark.asyncio + async def test_factory_raises_when_sdk_not_available(self) -> None: + """Test that hermes provider raises ProviderError when SDK not available.""" + with pytest.raises(ProviderError, match="hermes-agent package"): + await create_provider("hermes", validate=False) + + @patch("conductor.providers.factory.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @pytest.mark.asyncio + async def test_factory_creates_hermes_provider(self) -> None: + """Test that hermes provider can be created successfully.""" + from conductor.providers.hermes import HermesProvider + + provider = await create_provider("hermes", validate=False) + assert isinstance(provider, HermesProvider) + await provider.close() + + @patch("conductor.providers.factory.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @pytest.mark.asyncio + async def test_factory_passes_all_config_to_hermes(self) -> None: + """Test that factory forwards all runtime config to HermesProvider.""" + from conductor.providers.hermes import HermesProvider + + provider = await create_provider( + "hermes", + validate=False, + default_model="anthropic/claude-sonnet-4", + max_tokens=4096, + temperature=0.7, + max_agent_iterations=25, + max_session_seconds=120.0, + default_reasoning_effort="high", + ) + assert isinstance(provider, HermesProvider) + assert provider._default_model == "anthropic/claude-sonnet-4" + assert provider._default_max_tokens == 4096 + assert provider._default_temperature == 0.7 + assert provider._default_max_agent_iterations == 25 + assert provider._default_max_session_seconds == 120.0 + assert provider._default_reasoning_effort == "high" + await provider.close() + + @patch("conductor.providers.factory.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @pytest.mark.asyncio + async def test_factory_extracts_provider_settings_for_hermes(self) -> None: + """Test that ProviderSettings with name='hermes' extracts base_url and api_key.""" + from conductor.providers.hermes import HermesProvider + + settings = ProviderSettings( + name="hermes", + base_url="http://localhost:8080", + api_key=SecretStr("sk-test-key"), + hermes_home="/tmp/hermes-test", + hermes_toolsets=["filesystem", "web"], + hermes_skip_memory=True, + hermes_skip_context_files=False, + ) + provider = await create_provider( + "hermes", + validate=False, + provider_settings=settings, + ) + assert isinstance(provider, HermesProvider) + assert provider._base_url == "http://localhost:8080" + assert provider._api_key == "sk-test-key" + assert provider._hermes_home == "/tmp/hermes-test" + assert provider._hermes_toolsets == ["filesystem", "web"] + assert provider._skip_memory is True + assert provider._skip_context_files is False + await provider.close() + + @patch("conductor.providers.factory.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @pytest.mark.asyncio + async def test_factory_ignores_non_hermes_provider_settings(self) -> None: + """Test that ProviderSettings with name != 'hermes' leaves base_url/api_key as None.""" + from conductor.providers.hermes import HermesProvider + + settings = ProviderSettings(name="copilot") + provider = await create_provider( + "hermes", + validate=False, + provider_settings=settings, + ) + assert isinstance(provider, HermesProvider) + assert provider._base_url is None + assert provider._api_key is None + assert provider._hermes_home is None + assert provider._hermes_toolsets is None + assert provider._skip_memory is None + assert provider._skip_context_files is None + await provider.close() diff --git a/tests/test_providers/test_hermes.py b/tests/test_providers/test_hermes.py new file mode 100644 index 00000000..7daf7ff8 --- /dev/null +++ b/tests/test_providers/test_hermes.py @@ -0,0 +1,752 @@ +"""Unit tests for the HermesProvider implementation.""" + +from __future__ import annotations + +import asyncio +from typing import Any +from unittest.mock import MagicMock, Mock, patch + +import pytest + +from conductor.config.schema import AgentDef, OutputField +from conductor.exceptions import ProviderError +from conductor.providers.hermes import HermesProvider + + +def _make_agent( + name: str = "test_agent", + model: str | None = None, + output: dict[str, OutputField] | None = None, + max_agent_iterations: int | None = None, + max_session_seconds: float | None = None, + tools: list[str] | None = None, + system_prompt: str | None = None, +) -> AgentDef: + return AgentDef( + name=name, + model=model, + output=output, + max_agent_iterations=max_agent_iterations, + max_session_seconds=max_session_seconds, + tools=tools, + system_prompt=system_prompt, + ) + + +def _make_result( + final_response: str = "hello", + completed: bool = True, + failed: bool = False, + partial: bool = False, + error: str | None = None, + model: str | None = "anthropic/claude-sonnet-4", + input_tokens: int | None = 10, + output_tokens: int | None = 20, + total_tokens: int | None = 30, +) -> dict[str, Any]: + return { + "final_response": final_response, + "completed": completed, + "failed": failed, + "partial": partial, + "error": error, + "messages": [], + "api_calls": 1, + "model": model, + "provider": "anthropic", + "input_tokens": input_tokens, + "output_tokens": output_tokens, + "total_tokens": total_tokens, + } + + +class TestHermesProviderInit: + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", False) + def test_raises_when_sdk_not_installed(self) -> None: + with pytest.raises(ProviderError, match="hermes-agent"): + HermesProvider() + + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.AIAgent", MagicMock()) + def test_init_defaults(self) -> None: + p = HermesProvider() + assert p._default_model is None + assert p._default_max_agent_iterations is None + assert p._default_max_session_seconds is None + + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.AIAgent", MagicMock()) + def test_init_custom(self) -> None: + p = HermesProvider(model="openai/gpt-4o", max_agent_iterations=25) + assert p._default_model == "openai/gpt-4o" + assert p._default_max_agent_iterations == 25 + + +class TestHermesValidateConnection: + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", False) + def test_returns_false_when_sdk_missing(self) -> None: + p = object.__new__(HermesProvider) + result = asyncio.run(p.validate_connection()) + assert result is False + + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.AIAgent", MagicMock()) + def test_returns_true_when_sdk_available(self) -> None: + p = object.__new__(HermesProvider) + result = asyncio.run(p.validate_connection()) + assert result is True + + +class TestHermesExecute: + @pytest.fixture() + def provider(self) -> HermesProvider: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + return HermesProvider(model="anthropic/claude-sonnet-4", max_agent_iterations=10) + + def _run(self, coro: Any) -> Any: + return asyncio.run(coro) + + def test_plain_text_no_schema(self, provider: HermesProvider) -> None: + agent = _make_agent() + result_dict = _make_result(final_response="world") + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = result_dict + mock_cls.return_value = mock_instance + + output = self._run(provider.execute(agent, {}, "say hello")) + + assert output.content == {"text": "world"} + assert output.raw_response["final_response"] == "world" + + def test_json_schema_appends_instruction(self, provider: HermesProvider) -> None: + schema = {"answer": OutputField(type="string")} + agent = _make_agent(output=schema) + result_dict = _make_result(final_response='{"answer": "pong"}') + + captured_prompts: list[str] = [] + + def fake_run_conv(prompt: str, **kwargs: Any) -> dict[str, Any]: + captured_prompts.append(prompt) + return result_dict + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.side_effect = fake_run_conv + mock_cls.return_value = mock_instance + + output = self._run(provider.execute(agent, {}, "answer this")) + + assert "MUST respond with a JSON object matching this schema" in captured_prompts[0] + assert output.content == {"answer": "pong"} + + def test_json_schema_validation_error(self, provider: HermesProvider) -> None: + schema = {"answer": OutputField(type="string")} + agent = _make_agent(output=schema) + # Missing required field 'answer' — recovery loop exhausted + result_dict = _make_result(final_response='{"wrong": "field"}') + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = result_dict + mock_cls.return_value = mock_instance + + with pytest.raises(ProviderError, match="Failed to parse structured output"): + self._run(provider.execute(agent, {}, "answer this")) + + def test_passes_model_to_aiagent(self, provider: HermesProvider) -> None: + agent = _make_agent(model="openai/gpt-4o") + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + self._run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["model"] == "openai/gpt-4o" + + def test_uses_provider_default_model_when_agent_has_none( + self, provider: HermesProvider + ) -> None: + agent = _make_agent(model=None) + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + self._run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["model"] == "anthropic/claude-sonnet-4" + + def test_omits_model_when_neither_set(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider_no_model = HermesProvider() + + agent = _make_agent(model=None) + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + self._run(provider_no_model.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert "model" not in kwargs + + def test_passes_max_iterations(self, provider: HermesProvider) -> None: + agent = _make_agent(max_agent_iterations=42) + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + self._run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["max_iterations"] == 42 + + def test_quiet_mode_always_set(self, provider: HermesProvider) -> None: + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + self._run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["quiet_mode"] is True + + def test_skip_flags_omitted_by_default(self, provider: HermesProvider) -> None: + """When skip_memory/skip_context_files are not configured, they are omitted + from agent_kwargs so the hermes-agent library defaults apply (False).""" + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + self._run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert "skip_context_files" not in kwargs + assert "skip_memory" not in kwargs + + def test_skip_memory_true_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(skip_memory=True) + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["skip_memory"] is True + + def test_skip_context_files_true_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(skip_context_files=True) + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["skip_context_files"] is True + + def test_skip_flags_false_forwarded(self) -> None: + """Explicit False is forwarded (though it matches library default).""" + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(skip_memory=False, skip_context_files=False) + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["skip_memory"] is False + assert kwargs["skip_context_files"] is False + + def test_token_counts_populated(self, provider: HermesProvider) -> None: + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result( + input_tokens=100, output_tokens=50, total_tokens=150 + ) + mock_cls.return_value = mock_instance + + output = self._run(provider.execute(agent, {}, "hello")) + + assert output.input_tokens == 100 + assert output.output_tokens == 50 + assert output.tokens_used == 150 + + def test_session_metadata_in_raw_response(self, provider: HermesProvider) -> None: + agent = _make_agent() + result_dict = _make_result(final_response="hi", model="openai/gpt-4o") + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = result_dict + mock_cls.return_value = mock_instance + + output = self._run(provider.execute(agent, {}, "hello")) + + assert output.raw_response["model"] == "openai/gpt-4o" + assert "messages" in output.raw_response + assert "api_calls" in output.raw_response + + def test_raises_provider_error_on_failed_result(self, provider: HermesProvider) -> None: + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result( + failed=True, final_response=None, error="quota exhausted" + ) + mock_cls.return_value = mock_instance + + with pytest.raises(ProviderError, match="quota exhausted"): + self._run(provider.execute(agent, {}, "hello")) + + def test_raises_provider_error_on_none_final_response(self, provider: HermesProvider) -> None: + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result( + final_response=None, completed=False, error="truncated" + ) + mock_cls.return_value = mock_instance + + with pytest.raises(ProviderError, match="no final response"): + self._run(provider.execute(agent, {}, "hello")) + + def test_raises_provider_error_on_sdk_exception(self, provider: HermesProvider) -> None: + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.side_effect = RuntimeError("network error") + mock_cls.return_value = mock_instance + + with pytest.raises(ProviderError, match="network error"): + self._run(provider.execute(agent, {}, "hello")) + + def test_event_callback_fires(self, provider: HermesProvider) -> None: + agent = _make_agent() + events: list[tuple[str, dict]] = [] + + def cb(event: str, data: dict) -> None: + events.append((event, data)) + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result(final_response="hi") + mock_cls.return_value = mock_instance + + self._run(provider.execute(agent, {}, "hello", event_callback=cb)) + + # agent_turn_start fires before the executor call + event_types = [e[0] for e in events] + assert "agent_turn_start" in event_types + turn_start = next(d for t, d in events if t == "agent_turn_start") + assert turn_start == {"turn": "awaiting_model"} + + # Streaming callbacks are wired into AIAgent constructor + _, kwargs = mock_cls.call_args + assert "stream_delta_callback" in kwargs + assert "reasoning_callback" in kwargs + + def test_streaming_callback_emits_events(self, provider: HermesProvider) -> None: + """Verify that stream_delta_callback and reasoning_callback emit events.""" + agent = _make_agent() + events: list[tuple[str, dict]] = [] + + def cb(event: str, data: dict) -> None: + events.append((event, data)) + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result(final_response="hi") + mock_cls.return_value = mock_instance + + self._run(provider.execute(agent, {}, "hello", event_callback=cb)) + + # Simulate what hermes does: invoke the callbacks + _, kwargs = mock_cls.call_args + kwargs["stream_delta_callback"]("hello ") + kwargs["stream_delta_callback"]("world") + kwargs["reasoning_callback"]("thinking...") + + msg_events = [(t, d) for t, d in events if t == "agent_message"] + assert len(msg_events) == 2 + assert msg_events[0][1] == {"content": "hello "} + assert msg_events[1][1] == {"content": "world"} + + reason_events = [(t, d) for t, d in events if t == "agent_reasoning"] + assert len(reason_events) == 1 + assert reason_events[0][1] == {"content": "thinking..."} + + def test_interrupt_signal_raises_provider_error(self, provider: HermesProvider) -> None: + agent = _make_agent() + + async def run_with_pre_set_interrupt() -> None: + interrupt = asyncio.Event() + interrupt.set() # already set before execute — wins the race immediately + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + import time + + mock_instance = Mock() + mock_instance.run_conversation.side_effect = lambda *a, **kw: time.sleep(5) + mock_cls.return_value = mock_instance + + with pytest.raises(ProviderError, match="interrupted"): + await provider.execute(agent, {}, "hello", interrupt_signal=interrupt) + + asyncio.run(run_with_pre_set_interrupt()) + + +class TestHermesSystemPrompt: + def test_system_prompt_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider() + + agent = _make_agent(system_prompt="You are a helpful assistant.") + captured: list[dict] = [] + + def fake_run_conv(prompt: str, **kwargs: Any) -> dict[str, Any]: + captured.append(kwargs) + return _make_result() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.side_effect = fake_run_conv + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + assert captured[0].get("system_message") == "You are a helpful assistant." + + def test_system_prompt_none_when_not_set(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider() + + agent = _make_agent(system_prompt=None) + captured: list[dict] = [] + + def fake_run_conv(prompt: str, **kwargs: Any) -> dict[str, Any]: + captured.append(kwargs) + return _make_result() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.side_effect = fake_run_conv + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + assert captured[0].get("system_message") is None + + +class TestHermesToolsMapping: + def test_tools_none_uses_hermes_defaults(self) -> None: + """tools=None (omitted) does not set enabled_toolsets — hermes uses its defaults.""" + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider() + + agent = _make_agent(tools=None) + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert "enabled_toolsets" not in kwargs + + def test_tools_empty_disables_all(self) -> None: + """tools=[] explicitly disables all hermes toolsets.""" + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider() + + agent = _make_agent(tools=[]) + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello", tools=[])) + + _, kwargs = mock_cls.call_args + assert kwargs["enabled_toolsets"] == [] + + def test_tools_nonempty_raises_provider_error(self) -> None: + """Non-empty tools: list raises ProviderError (vocabulary mismatch).""" + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider() + + agent = _make_agent(tools=["web_search", "read_file"]) + + with pytest.raises(ProviderError, match="does not support per-agent workflow tool"): + asyncio.run(provider.execute(agent, {}, "hello", tools=["web_search", "read_file"])) + + def test_hermes_toolsets_forwarded_as_enabled_toolsets(self) -> None: + """Provider-level hermes_toolsets is forwarded when tools=None.""" + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(hermes_toolsets=["filesystem", "web"]) + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["enabled_toolsets"] == ["filesystem", "web"] + + def test_hermes_toolsets_empty_disables_all(self) -> None: + """Provider-level hermes_toolsets=[] disables all toolsets.""" + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(hermes_toolsets=[]) + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["enabled_toolsets"] == [] + + +class TestHermesProviderParams: + def test_max_tokens_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(max_tokens=1024) + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["max_tokens"] == 1024 + + def test_temperature_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(temperature=0.5) + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["request_overrides"] == {"temperature": 0.5} + + def test_base_url_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(base_url="https://openrouter.ai/api/v1") + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["base_url"] == "https://openrouter.ai/api/v1" + + def test_api_key_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(api_key="sk-test-key") + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "sk-test-key" + + def test_error_message_includes_model(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(model="anthropic/claude-sonnet-4") + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result( + failed=True, final_response=None, error="quota exhausted" + ) + mock_cls.return_value = mock_instance + + with pytest.raises(ProviderError, match="anthropic/claude-sonnet-4"): + asyncio.run(provider.execute(agent, {}, "hello")) + + def test_missing_params_not_forwarded(self) -> None: + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider() + + agent = _make_agent() + + with patch("conductor.providers.hermes.AIAgent") as mock_cls: + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + _, kwargs = mock_cls.call_args + assert "max_tokens" not in kwargs + assert "temperature" not in kwargs + assert "base_url" not in kwargs + assert "api_key" not in kwargs + assert "skip_memory" not in kwargs + assert "skip_context_files" not in kwargs + + +class TestHermesHome: + def test_tilde_expanded_before_sdk_call(self) -> None: + """hermes_home with ~ is expanded to an absolute path.""" + import sys + + mock_hermes_constants = MagicMock() + mock_hermes_constants.set_hermes_home_override.return_value = "token" + + with ( + patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True), + patch("conductor.providers.hermes.AIAgent"), + ): + provider = HermesProvider(hermes_home="~/.hermes/profiles/chloe") + + agent = _make_agent() + + with ( + patch("conductor.providers.hermes.AIAgent") as mock_cls, + patch.dict(sys.modules, {"hermes_constants": mock_hermes_constants}), + ): + mock_instance = Mock() + mock_instance.run_conversation.return_value = _make_result() + mock_cls.return_value = mock_instance + + asyncio.run(provider.execute(agent, {}, "hello")) + + called_path = mock_hermes_constants.set_hermes_home_override.call_args[0][0] + assert "~" not in called_path + assert called_path.endswith(".hermes/profiles/chloe") + + +class TestHermesClose: + @patch("conductor.providers.hermes.HERMES_SDK_AVAILABLE", True) + @patch("conductor.providers.hermes.AIAgent", MagicMock()) + def test_close_is_noop(self) -> None: + p = HermesProvider() + asyncio.run(p.close()) # should not raise