CI hardening: provider API keys exposed on pull_request (evals.yml) + third-party actions pinned to mutable tags #1948

Summary

Two CI/CD supply-chain hardening gaps, found during a security review of v1.57.7.0.

1. Provider API keys are available to pull_request-triggered runs

.github/workflows/evals.yml is triggered on: pull_request (line 3) and runs the test matrix on PR-authored code with real provider keys in env:

ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}   # :142
OPENAI_API_KEY:    ${{ secrets.OPENAI_API_KEY }}       # :143
GEMINI_API_KEY:    ${{ secrets.GEMINI_API_KEY }}       # :144

GitHub does not pass secrets to fork PRs unless the repo opts in / a maintainer approves the run — so this is safe only if that setting is locked down. If "send secrets to workflows from fork pull requests" is enabled (or for any branch PR from a write-access contributor), a PR that edits a matrix test/build file executes arbitrary code with all three keys in the environment → key exfiltration. (evals-periodic.yml uses the same keys but is schedule-triggered, so it isn't fork-reachable.)

Fix: confirm fork-PR secret passing is disabled and rely on maintainer "approve and run"; better, split the secret-bearing eval runs into a gated stage (label-triggered, merge-queue, or workflow_dispatch) so the fork-facing PR check runs without provider keys.

2. Third-party actions are pinned to mutable tags, not commit SHAs

Every action reference uses a moving tag rather than a pinned SHA, e.g.:

• docker/login-action@v3, docker/build-push-action@v6 — evals.yml:30,49; evals-periodic.yml:30,49; ci-image.yml
• oven-sh/setup-bun@v1 / @v2 — windows-free-tests.yml, windows-setup-e2e.yml, skill-docs.yml, version-gate.yml
• actions/checkout@v4, actions/upload-artifact@v4, actions/download-artifact@v4 — throughout

If any tag is force-moved (compromised maintainer / hijacked org), the next run executes attacker code — including in the secret-bearing evals job (#1 above) and the pull_request_target pr-title-sync job (which holds a write token). docker/build-push-action runs in jobs with packages: write and pushes ghcr.io/<repo>/ci, so a compromise there poisons the CI base image consumed by later eval runs.

Fix: pin actions to full 40-char commit SHAs (with a # vX.Y.Z trailing comment) and let Dependabot/Renovate bump them; pin bun-version to an explicit version rather than latest.

Related and already filed: #1706 (avoid piping the remote Bun installer straight into the CI shell). The GitHub pr-title-sync.yml workflow, by contrast, is correctly hardened — untrusted PR fields go through env: and it checks out the base repo only.