Repository navigation
Add opt-in Claude agent backend (#28) #58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
a5ae114
562747c
716483c
a0809d9
54b50f6
0964c65
ae3e35e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| import { readFileSync } from "node:fs"; | ||
| import path from "node:path"; | ||
| import { ClaudeCodeAgent as SmithersClaudeCodeAgent } from "smithers-orchestrator"; | ||
| import { readStringTable, stringField } from "./toml"; | ||
|
|
||
| type ClaudeAuthConfig = { auth?: string; api_key_env?: string; config_dir?: string }; | ||
| type ClaudeAuthOptions = { apiKey?: string; configDir?: string }; | ||
| // reasoningEffort is part of the shared agent factory contract but has no | ||
| // ClaudeCodeAgent equivalent, so it is accepted and ignored. | ||
| export type ClaudeTaskOptions = { model?: string; reasoningEffort?: string }; | ||
|
|
||
| export function createClaudeAgent(options: ClaudeTaskOptions = {}): SmithersClaudeCodeAgent { | ||
| return new SmithersClaudeCodeAgent({ | ||
| ...(options.model === undefined ? {} : { model: options.model }), | ||
| // Every task runs with Claude Code's permission checks off: agents work | ||
| // unattended in a throwaway worktree, so there is nobody to answer a | ||
| // prompt. This mirrors permissions.trust_model = "skip-permissions" in | ||
| // ultrafuzz.toml, which is the only trust model ultrafuzz accepts, and is | ||
| // deliberately not configurable per agent -- edit this generated file if a | ||
| // project needs otherwise. | ||
| permissionMode: "bypassPermissions", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Prompt To Fix With AIThis is a comment left during a code review.
Path: packages/runtime/src/templates/smithers/agents/claude.tsx
Line: 14
Comment:
**`bypassPermissions` is hardcoded with no user-visible documentation**
`permissionMode: "bypassPermissions"` is always set unconditionally, meaning the Claude Code CLI runs with unrestricted access to the filesystem and shell for every task. The template comment above `ClaudeTaskOptions` notes that `reasoningEffort` is silently ignored, but there is no parallel note explaining that permission mode is fixed and not user-configurable. A user inspecting only the `ultrafuzz.toml` [agents.ClaudeAgent] section (which exposes only `auth`, `api_key_env`, and `config_dir`) has no indication that all permission checks are bypassed.
How can I resolve this? If you propose a fix, please make it concise. |
||
| ...claudeAuthOptions() | ||
| }); | ||
| } | ||
|
|
||
| function claudeAuthOptions(): ClaudeAuthOptions { | ||
| const config = readClaudeAuthConfig(); | ||
| const auth = config.auth ?? "subscription"; | ||
| if (auth === "api-key") { | ||
| return { apiKey: requiredEnv(config.api_key_env ?? "ANTHROPIC_API_KEY") }; | ||
| } | ||
| if (auth === "subscription") { | ||
| // ClaudeCodeAgent clears ANTHROPIC_API_KEY itself so the logged-in | ||
| // Claude subscription (`claude -p`) is used; we only forward an | ||
| // isolated config directory when one is configured. | ||
| return config.config_dir === undefined ? {} : { configDir: resolveConfigDir(config.config_dir) }; | ||
| } | ||
| throw new Error(`unsupported ClaudeAgent auth mode in ultrafuzz.toml: ${auth}`); | ||
| } | ||
|
|
||
| function readClaudeAuthConfig(): ClaudeAuthConfig { | ||
| const configPath = path.join(process.cwd(), "ultrafuzz.toml"); | ||
| const claude = readStringTable(readFileSync(configPath, "utf8"), "agents.ClaudeAgent"); | ||
| return { | ||
| auth: stringField(claude, "auth"), | ||
| api_key_env: stringField(claude, "api_key_env"), | ||
| config_dir: stringField(claude, "config_dir") | ||
| }; | ||
| } | ||
|
|
||
| function requiredEnv(name: string): string { | ||
| const value = process.env[name]; | ||
| if (value === undefined || value.trim() === "") { | ||
| throw new Error(`agents.ClaudeAgent auth is api-key, but ${name} is not set`); | ||
| } | ||
| return value; | ||
| } | ||
|
|
||
| function resolveConfigDir(value: string): string { | ||
| if (value.trim() === "") { | ||
| throw new Error("agents.ClaudeAgent.config_dir cannot be empty"); | ||
| } | ||
| return path.isAbsolute(value) ? value : path.resolve(process.cwd(), value); | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,10 @@ | ||
| import { createClaudeAgent } from "./claude"; | ||
| import { createCodexAgent } from "./codex"; | ||
|
|
||
| export { CodexAgent, createCodexAgent } from "./codex"; | ||
| export const agentFactories = { CodexAgent: createCodexAgent }; | ||
| export { createClaudeAgent } from "./claude"; | ||
| export { createCodexAgent } from "./codex"; | ||
|
|
||
| // Agents are constructed per task from the selected model profile, never at | ||
| // import time: an agent's auth is only read when that agent is actually used, | ||
| // so a project running one backend does not need the other's credentials. | ||
| export const agentFactories = { ClaudeAgent: createClaudeAgent, CodexAgent: createCodexAgent }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reasoningleaks when both--agentand--modelare provided togetherThe clearing block is only triggered when
overrides.model === undefined. When a user runs--agent ClaudeAgent --model claude-sonnet-5, the condition evaluates to false (becauseoverrides.model !== undefined), soprofile.reasoningis not deleted. The resulting profile keeps the old agent's reasoning value (e.g."xhigh"from the Codex default), which flows through to task metadata asreasoningEffort: "xhigh"on a ClaudeAgent task — contradicting the invariant the PR establishes. The test for the pinned case (--agent ClaudeAgent --model claude-sonnet-5) does not includereasoningEffortin its type assertion, so this case is currently untested. The fix is to drop the&& overrides.model === undefinedguard so thatreasoningis always cleared on any agent switch, regardless of whether a model is also supplied.Prompt To Fix With AI