Skip to content

Worktree-isolated sessions refuse claude plugin eval (guard reads 'eval' as shell eval) #5696

Open

Description

@kyle-sexton

Summary

In a worktree-isolated Claude Code session, the Bash tool refuses any command that contains eval as a word, including claude plugin eval and claude plugin eval --help. The guard reads the subcommand as the shell builtin eval. Plugin skills that run plugin evals (/evals:plugin-eval) therefore cannot run from an isolated worktree session.

The guard is built into Claude Code: its message text is in the 2.1.285 binary, not in any plugin in this repo. Per our rule against filing on Anthropic's repositories, this issue tracks the limitation here.

Repro

  1. Start a session isolated in a git worktree (Claude Code 2.1.285, Linux/WSL2).
  2. Have the model run claude plugin eval --help.
  3. Observed: refused with "this command runs a string through eval, which can't be verified to stay inside the worktree."
  4. Expected: claude plugin eval is a CLI subcommand, not shell eval, so it should run.

claude --version and claude plugin --help pass. Another message from the same guard: "uses a subshell in a command inside a construct too complex to verify".

Workaround

Run the command in a separate terminal outside Claude Code. The ! prefix does not work: the user's own ! command in the same session is refused with the identical message (verified 2026-10-01). Do not bypass the guard with wrapper scripts.

Follow-ups

  • /evals:plugin-eval should name this limitation and give the user the exact command to run in a separate terminal when it runs in a worktree-isolated session.
  • The refusal message ends "a worktree-isolated session's git operations must target its own worktree", which is misleading because this command is not a git operation.
  • Recheck on each Claude Code release: run claude plugin eval --help from an isolated worktree session, and close this issue once it passes.

Found while running an approved observation run during the eval-design/hillclimbing digest interview (branch feat/eval-design-hillclimbing-digest).

🤖 Generated with Claude Code

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 1, 2026
  2. kyle-sexton commented on Oct 4, 2026

    @kyle-sexton
    MemberAuthor

    This was generated by AI during triage.

    Triage: human-gated, upstream watch

    Routing reason: the plugin-side follow-up is done, and the remaining closure criterion is upstream: claude plugin eval --help has to pass from a worktree-isolated session, which only a Claude Code release can change. Re-running it needs an isolated session, so this item stays parked rather than agent-ready.

    What triage established

    Next human action: from a worktree-isolated session on 2.1.289, run claude plugin eval --help. If it passes, close this issue and refresh the plugin-eval verification row. If it is still refused, refresh the row's version and as-of date and leave this open.

    Agent Brief

    Type: Task
    Summary: Recheck whether the built-in worktree guard still refuses claude plugin eval on each Claude Code release that touches worktree isolation or plugin eval.

    Current behavior:
    On 2.1.285 the built-in guard refuses any command containing the word eval in a worktree-isolated session, including claude plugin eval --help and the user's own ! command.

    Desired behavior:
    After each triggering release, the outcome of claude plugin eval --help from an isolated session is recorded in the plugin-eval verification record. The issue closes once the command passes.

    Key interfaces:

    • The plugin-eval skill's worktree-isolation fact row and its recheck trigger.

    Acceptance criteria:

    • claude plugin eval --help run from a worktree-isolated session on the current release, with the result recorded.
    • The verification row's version and as-of date match that run.

    Out of scope:

    • Wrapper scripts or any other bypass of the guard.
    • Filing on Anthropic's repositories.
  3. added
    priority: lowNice-to-have, cosmetic, or speculative; opportunistic.
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 4, 2026
  4. kyle-sexton commented on Oct 13, 2026

    @kyle-sexton
    MemberAuthor

    Owner ratified this admission in an attended session on 2026-10-13. Relabelled work-class: mechanical, because the step is a one-line SKILL.md edit, which read-only does not allow.

  5. added
    work-class: mechanicalDeterministic, trivially reversible maintenance: dependency bumps, lint/format, sync.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    and removed
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    work-class: read-onlyAudits, research, reports. No repository mutation; tracker and queue writes only.
    on Oct 13, 2026
  6. self-assigned this
    on Oct 14, 2026
  7. kyle-sexton commented on Oct 14, 2026

    @kyle-sexton
    MemberAuthor
    No description provided.
  8. removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Oct 14, 2026
  9. added theissue type on Oct 15, 2026
  10. removed their assignment
    on Oct 19, 2026
  11. kyle-sexton commented on Oct 19, 2026

    @kyle-sexton
    MemberAuthor

    work-item-lease reclaimed: lease expired (renewed_at 2026-10-14T00:51:21Z) with no activity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: lowNice-to-have, cosmetic, or speculative; opportunistic.work-class: mechanicalDeterministic, trivially reversible maintenance: dependency bumps, lint/format, sync.

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions