[mrthankyou]

Closes #28.

Summary

• Add an opt-in Claude agent backend, driven by the Claude Code CLI
• Construct project agents per task instead of at registry import
• Qualify the generated agents import so a sibling file cannot shadow it
• Keep --agent from carrying the previous agent's model onto the new one
• Report a stale agent registry instead of failing later at launch

Validation

• Runtime test suite — 59 passed, 1 skipped, 0 failed
• Config test suite — 16 passed
• CLI test suite — 6 passed
• Full workspace build and typecheck — clean
• Generated adapters typechecked against smithers-orchestrator 0.27.0 and driven end-to-end under bun, since templates are emitted as text and the workspace build does not cover them

What this does

ultrafuzz init now also generates a ClaudeAgent backed by the Claude Code CLI (claude --print). The default profile stays on CodexAgent; Claude is opt-in via the new claude profile, selected per node with model_profiles = ["claude"] or for a single run with --agent ClaudeAgent.

Both auth modes drive the claude CLI — api-key only changes how that CLI authenticates, so the CLI is required either way. This is not a direct Anthropic API client.

Bugs found while reviewing the change

Each was reachable only once a second agent existed, and each has a regression test that fails without its fix:

1. --agent ClaudeAgent set the agent but kept the default profile's model, so Claude launched with Codex's gpt-5.5. Switching agents without naming a model now drops the previous agent's model and reasoning; --model still pins it. The override logic was duplicated verbatim in validate.ts and plan-run.ts and now lives once in @ultrafuzz/config.
2. Importing the agent registry constructed every agent, and constructing reads auth — so a Claude-only project still failed with OPENAI_API_KEY is not set. Agents are now constructed per task.
3. init preserves project-owned files, so an existing project got claude.ts on disk with nothing exporting it. init now warns and names the missing agent rather than deferring the failure to launch.

🤖 Generated with Claude Code

Greptile Summary

This PR adds an opt-in ClaudeAgent backend driven by the Claude Code CLI, and proactively fixes three bugs exposed by having two agents: model/reasoning carryover when switching agents via --agent, agent auth being eagerly read at registry import time, and stale agent registry detection after init.

• New Claude backend: ultrafuzz init now generates .smithers/agents/claude.ts (via claude.tsx template) alongside a shared TOML helper (toml.tsx) used by both agent adapters. The default profile stays on CodexAgent; Claude is opt-in via model_profiles = ["claude"] or --agent ClaudeAgent.
• Centralized override logic: applyDefaultProfileOverrides is extracted into @ultrafuzz/config and replaces duplicated inline code in both validate.ts and plan-run.ts; switching agents without --model now clears the previous agent's model and reasoning before applying the new agent.
• Lazy agent construction: The generated agents/index.ts no longer exports a top-level CodexAgent instance; both agents are now exposed solely as factory functions in agentFactories, so importing the registry cannot trigger auth reads for backends the project doesn't use.

Confidence Score: 5/5

Safe to merge; the three proactively fixed bugs each have a targeted regression test, the shared TOML decoder correctly handles all TOML escape sequences, and lazy agent construction prevents cross-backend credential reads.

All changes are well-tested, the new ClaudeAgent template mirrors the existing CodexAgent pattern, and the centralised override logic is correct for the primary --agent-only case. The one edge case where reasoning is not cleared when both --agent and --model are supplied together was already flagged in a prior review thread.

packages/config/src/model-profiles.ts — the clearing condition in applyDefaultProfileOverrides has a known gap noted in a previous review thread.

Important Files Changed

Filename	Overview
packages/config/src/model-profiles.ts	Adds applyDefaultProfileOverrides; correctly clears model/reasoning on agent switch when no model is pinned, but reasoning is not cleared when both --agent and --model are supplied together (already flagged in previous thread)
packages/runtime/src/templates/smithers/agents/toml.tsx	New shared TOML basic-string decoder; correctly handles all TOML escapes including \uXXXX and \UXXXXXXXX, validates Unicode scalar range, and replaces the former JSON.parse approach
packages/runtime/src/templates/smithers/agents/claude.tsx	New ClaudeAgent template; reads auth from ultrafuzz.toml, uses shared toml helper, hardcodes bypassPermissions (documented), and defers construction to call time (no top-level instance)
packages/runtime/src/templates/smithers/agents/codex.tsx	Removes top-level CodexAgent instance and inline TOML helpers, now importing from the shared ./toml module; eliminates eager auth read at import time
packages/runtime/src/templates/smithers/agents/index.tsx	Updated registry template exposes both agents as factory references only; no agent is constructed at import time, and agentFactories includes both ClaudeAgent and CodexAgent
packages/runtime/src/init.ts	Generates toml.ts and both agent adapters during init; adds staleAgentRegistryDiagnostics to warn when a preserved registry is missing a newly generated agent factory entry
packages/runtime/src/validate.ts	Replaces duplicated applyAgentOverrides inline logic with a call to the shared applyDefaultProfileOverrides from @ultrafuzz/config
packages/runtime/src/plan-run.ts	Replaces duplicated applyWorkflowRunOverrides inline logic with applyDefaultProfileOverrides; no functional change beyond the agent-switch clearing behavior
packages/runtime/src/templates/smithers/workflows/workflow.tsx	Changes agents import to explicit ../agents/index.ts path to prevent a sibling agents.ts scaffolded by Smithers from shadowing the agents/ directory under bun
packages/runtime/test/runtime.test.ts	Adds regression tests for all three bugs fixed (model carryover, eager auth read, stale registry), and validates ClaudeAgent template structure and TOML helper sharing
docs/config.md	Documents the Claude agent backend, auth modes, reasoning-effort limitation, and the fixed bypassPermissions permission mode
ultrafuzz.toml	Adds [models.claude] opt-in profile and [agents.ClaudeAgent] section with subscription auth; well-commented with references to the bypassPermissions behavior

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[ultrafuzz run --agent ClaudeAgent] --> B[applyDefaultProfileOverrides]
    B --> C{agent changed AND no --model?}
    C -- yes --> D[delete profile.model and profile.reasoning]
    C -- no --> E[keep existing model/reasoning]
    D --> F[profile.agent = ClaudeAgent]
    E --> F
    F --> G[task compiled with agentRef = ClaudeAgent, modelName = null]
    H[import agents/index.ts] --> I{agent factory lookup}
    I -- ClaudeAgent --> J[createClaudeAgent called per task]
    I -- CodexAgent --> K[createCodexAgent called per task]
    J --> L[readClaudeAuthConfig from ultrafuzz.toml via toml.ts]
    K --> M[readCodexAuthConfig from ultrafuzz.toml via toml.ts]
    L --> N[new SmithersClaudeCodeAgent permissionMode bypassPermissions]
    M --> O[new SmithersCodexAgent]
    P[ultrafuzz init] --> Q[write toml.ts]
    P --> R[write claude.ts]
    P --> S[write codex.ts]
    P --> T[staleAgentRegistryDiagnostics]
    T --> U{registry exists AND agent file exists AND NOT in agentFactories?}
    U -- yes --> V[warn INIT_AGENT_REGISTRY_STALE]
    U -- no --> W[silent]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[ultrafuzz run --agent ClaudeAgent] --> B[applyDefaultProfileOverrides]
    B --> C{agent changed AND no --model?}
    C -- yes --> D[delete profile.model and profile.reasoning]
    C -- no --> E[keep existing model/reasoning]
    D --> F[profile.agent = ClaudeAgent]
    E --> F
    F --> G[task compiled with agentRef = ClaudeAgent, modelName = null]
    H[import agents/index.ts] --> I{agent factory lookup}
    I -- ClaudeAgent --> J[createClaudeAgent called per task]
    I -- CodexAgent --> K[createCodexAgent called per task]
    J --> L[readClaudeAuthConfig from ultrafuzz.toml via toml.ts]
    K --> M[readCodexAuthConfig from ultrafuzz.toml via toml.ts]
    L --> N[new SmithersClaudeCodeAgent permissionMode bypassPermissions]
    M --> O[new SmithersCodexAgent]
    P[ultrafuzz init] --> Q[write toml.ts]
    P --> R[write claude.ts]
    P --> S[write codex.ts]
    P --> T[staleAgentRegistryDiagnostics]
    T --> U{registry exists AND agent file exists AND NOT in agentFactories?}
    U -- yes --> V[warn INIT_AGENT_REGISTRY_STALE]
    U -- no --> W[silent]

Loading

_{Reviews (3): Last reviewed commit: "Document that ClaudeAgent bypasses Claud..." | Re-trigger Greptile}

[greptile-apps]

bypassPermissions is hardcoded with no user-visible documentation

permissionMode: "bypassPermissions" is always set unconditionally, meaning the Claude Code CLI runs with unrestricted access to the filesystem and shell for every task. The template comment above ClaudeTaskOptions notes that reasoningEffort is silently ignored, but there is no parallel note explaining that permission mode is fixed and not user-configurable. A user inspecting only the ultrafuzz.toml [agents.ClaudeAgent] section (which exposes only auth, api_key_env, and config_dir) has no indication that all permission checks are bypassed.

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/src/templates/smithers/agents/claude.tsx
Line: 14

Comment:
**`bypassPermissions` is hardcoded with no user-visible documentation**

`permissionMode: "bypassPermissions"` is always set unconditionally, meaning the Claude Code CLI runs with unrestricted access to the filesystem and shell for every task. The template comment above `ClaudeTaskOptions` notes that `reasoningEffort` is silently ignored, but there is no parallel note explaining that permission mode is fixed and not user-configurable. A user inspecting only the `ultrafuzz.toml` [agents.ClaudeAgent] section (which exposes only `auth`, `api_key_env`, and `config_dir`) has no indication that all permission checks are bypassed.

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

Stale reasoning leaks when both --agent and --model are provided together

The clearing block is only triggered when overrides.model === undefined. When a user runs --agent ClaudeAgent --model claude-sonnet-5, the condition evaluates to false (because overrides.model !== undefined), so profile.reasoning is not deleted. The resulting profile keeps the old agent's reasoning value (e.g. "xhigh" from the Codex default), which flows through to task metadata as reasoningEffort: "xhigh" on a ClaudeAgent task — contradicting the invariant the PR establishes. The test for the pinned case (--agent ClaudeAgent --model claude-sonnet-5) does not include reasoningEffort in its type assertion, so this case is currently untested. The fix is to drop the && overrides.model === undefined guard so that reasoning is always cleared on any agent switch, regardless of whether a model is also supplied.

Suggested change

	if (overrides.agent !== undefined && overrides.agent !== profile.agent && overrides.model === undefined) {
	delete profile.model;
	delete profile.reasoning;
	}
	// A profile's model and reasoning are chosen for its agent, so switching the
	// agent must not hand one backend's model or reasoning to another.
	if (overrides.agent !== undefined && overrides.agent !== profile.agent) {
	delete profile.model;
	delete profile.reasoning;
	}

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/config/src/model-profiles.ts
Line: 107-110

Comment:
**Stale `reasoning` leaks when both `--agent` and `--model` are provided together**

The clearing block is only triggered when `overrides.model === undefined`. When a user runs `--agent ClaudeAgent --model claude-sonnet-5`, the condition evaluates to false (because `overrides.model !== undefined`), so `profile.reasoning` is not deleted. The resulting profile keeps the old agent's reasoning value (e.g. `"xhigh"` from the Codex default), which flows through to task metadata as `reasoningEffort: "xhigh"` on a ClaudeAgent task — contradicting the invariant the PR establishes. The test for the pinned case (`--agent ClaudeAgent --model claude-sonnet-5`) does not include `reasoningEffort` in its type assertion, so this case is currently untested. The fix is to drop the `&& overrides.model === undefined` guard so that `reasoning` is always cleared on any agent switch, regardless of whether a model is also supplied.

```suggestion
  // A profile's model and reasoning are chosen for its agent, so switching the
  // agent must not hand one backend's model or reasoning to another.
  if (overrides.agent !== undefined && overrides.agent !== profile.agent) {
    delete profile.model;
    delete profile.reasoning;
  }
```

How can I resolve this? If you propose a fix, please make it concise.