Skip to content

feat(dev): run harnesses locally in agentcore dev - #2525

Closed
tejaskash wants to merge 1 commit into
refactorfrom
feat/harness-local-dev
Closed

tejaskash wants to merge 1 commit into
refactorfrom
feat/harness-local-dev

Conversation

@tejaskash

Copy link
Copy Markdown
Contributor

What

  • agentcore dev runs a project's harnesses locally in the public harness image, one container per harness, behind an endpoint on port 8090. --harness <name> selects one.
  • agentcore invoke --harness <name> --local sends a turn to that endpoint and prints the same transcript as the deployed path.
  • The Inspector harness chat routes to the local harness.
  • API key models (apiKeyArn) work through a dev workload identity, agentcore-dev-<project>-<target>, created on first use and deleted on target teardown.
  • Dev containers (harness and container runtimes) now get AWS credentials the CLI resolves on the host, through a mounted credential_process file the CLI refreshes. The host ~/.aws mount and AWS_PROFILE forwarding are removed.

Why

Harness changes to harness.yaml or system-prompt.md needed a deploy to test. The credentials change fixes dev containers for profiles that use credential_process (for example ada), which failed because the host program does not exist in the container.

How tested

  • bun run typecheck, bun run lint, bun run format:check, bun test (4099 pass).
  • bun run test:e2e scoped to e2eTest/project/harness.test.ts add tests and the new local invocation test (deploy cases skipped).
  • Built CLI on macOS arm64 with Docker, ada profile, no exported keys: harness turns over two sessions, Inspector SSE chat, a container runtime invoke, credential file refresh inside the container, and cleanup on shutdown.
  • API key path with a fake OpenAI key: identity created, token sent per turn, OpenAI returned 401.
  • Not verified: Linux hosts where the container user cannot read the 0600 credentials file (rootless Podman, host uid other than 1000).

@github-actions github-actions Bot added the size/xl PR size: XL label Oct 5, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added agentcore-harness-reviewing AgentCore Harness review in progress claude-security-reviewing Claude Code /security-review in progress labels Oct 5, 2026

@agentcore-devx-automation agentcore-devx-automation Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AgentCore Harness Review

Verdict: Looks good

Thorough review of the source changes — nothing serious to block merging. The feature is cleanly decomposed (core/dev/harness/{hosts,endpoint,image,request}.ts), tests favor real temp directories and real Bun.serve fakes over mocks, and the loopback HTTP endpoint has good defense in depth (loopback-only Host, Origin rejection, required X-Agentcore-Local header).

A few things I checked and am comfortable with:

  • containerCredentials credential_process flow (src/core/dev/container.ts) — the 0600 + bind-mounted credential_process file is a clean replacement for the ~/.aws mount, and the Linux uid-mismatch limitation is already called out in the PR description. The atomic writeFile → rename for the credentials JSON and the longLived/forceRefresh logic look correct.
  • HarnessDevHosts lifecycle (src/core/dev/harness/hosts.ts) — busy/removed/starting interleaving looked suspicious at first (concurrent invoke, mid-settle removal), but assertIdle plus the release→retire in the catch and stream paths handles it correctly, and pump is tracked in running so the abort-on-shutdown wait reliably awaits container cleanup.
  • token() retry — swallowing all errors from workloadAccessToken() and retrying once is slightly broad (not just 404), but the retry is bounded and errors from the second attempt surface normally. Minor, not worth changing.
  • AWS env forwarding change — intentionally dropping AWS_PROFILE matches the PR description. If a user sets AWS_PROFILE via .env.local or runtime envVars, it still flows into the container but will fail against the [default]-only config; that's an acceptable edge case to leave as-is since the error is self-explanatory.
  • Telemetry — the existing pattern only instruments cli.command_run at the router level, so agentcore dev --harness and agentcore invoke --local are automatically covered. No per-feature instrumentation is required by current conventions.

Lockfile churn is large but is just the knock-on from adding @aws-sdk/credential-provider-node and transitive dedup.

@agentcore-devx-automation agentcore-devx-automation Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Oct 5, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 5, 2026
@tejaskash
tejaskash force-pushed the feat/harness-local-dev branch from 8f743f8 to 2776775 Compare October 5, 2026 20:06
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Oct 5, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 5, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 5, 2026
@tejaskash
tejaskash force-pushed the feat/harness-local-dev branch from 2776775 to 61f2f72 Compare October 5, 2026 20:39
@github-actions github-actions Bot added size/xl PR size: XL and removed size/xl PR size: XL labels Oct 5, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 5, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@jariy17

jariy17 commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

rft says no.

@jariy17 jariy17 closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants