Skip to content

test(source-control): put the eval gh fixture on PATH without relying on zsh's .zshenv #7176

Closed

Description

@kyle-sexton

Context

Follow-up to #7164 (#6997). The owner accepted zsh-only coverage for the offline gh fixture so that PR could merge; this issue tracks the shell-independent fix.

Problem

plugins/source-control/evals/fixtures/gh-forge/install.sh puts the stand-in gh on PATH only by appending an export to the run's ~/.zshenv. On a host whose SHELL is bash (a default CI runner, for example), the stand-in is never on PATH: the run falls through to the real gh, which has no login inside the run and no network under the sandbox, so the ready cases stop at the PR lookup again.

Evidence from #7164 (Claude Code 2.1.296 probes):

  • A case cannot set PATH: case.yaml env keys are limited to EVAL_*.
  • The scaffold's HOME is the run's temporary home, and the eval run does not source ~/.zshrc, ~/.bashrc or ~/.profile: functions and variables defined there were missing from the run's Bash.
  • Only ~/.zshenv took effect, because zsh reads it on every zsh -c. That is undocumented runner behavior.
  • No run was made on a bash-SHELL host; the fall-through there is inferred from the above.

Ask

The clean fix needs a capability in claude plugin eval itself, for example:

  1. a case-level PATH prefix (a case.yaml key naming a directory under the scaffolded workspace to prepend), or
  2. a scaffold-written env file the runner applies to the run's shell.

This repository cannot add either; record the request upstream through the owner's normal channel, then switch gh-forge/install.sh to the new mechanism and drop the .zshenv append. An operator-level dispatcher gh on PATH (option 3 in #7164) works on any shell but adds a convention to every invocation; it stays the fallback if upstream declines.

Acceptance

  • The pull-request ready cases reach gh pr ready through the stand-in on a bash-SHELL host, shown by a kept-trace run.
  • install.sh no longer writes ~/.zshenv.

Related: #6997, #7164

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 11, 2026
  2. github-actions commented on Oct 11, 2026

    @github-actions
    Contributor

    Thanks for the report. Automated triage handed this issue to a maintainer.

    Automated triage. A maintainer reviews it.

  3. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Oct 11, 2026
  4. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Oct 12, 2026
  5. added theissue type on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.

    Type

    Fields

    No fields configured for Task.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions