Skip to content

feat(agent): add prompt replacement and Pi tool controls - #5144

Merged
RyaliNvidia merged 3 commits into
NVIDIA-NeMo:mainfrom
RyaliNvidia:codex/g10-native-code-generation-controls
Nov 2, 2026
Merged

RyaliNvidia merged 3 commits into
NVIDIA-NeMo:mainfrom
RyaliNvidia:codex/g10-native-code-generation-controls

Conversation

@RyaliNvidia

@RyaliNvidia RyaliNvidia commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Expose existing harness controls for custom system instructions and built-in tool selection. Pi and OpenCode gain system_prompt_mode: append | replace, retaining append defaults. Pi's enable_tools: false uses --no-builtin-tools while preserving separately granted MCP tools, output limits and outcome reporting.

OpenCode replacement explicitly selects build, keeps the user task separate and rejects conflicting --agent arguments. Local replacement uses a per-invocation config override, preserving sibling configuration without writing request-specific prompts into a persistent repository. Existing OpenCode tool selection, sandbox restrictions, plugins and continuation behavior remain intact.

Preserve injected replacement prompts literally through OpenCode's pre-JSON {env:...} / {file:...} substitution. Escape opening braces only in the serialized agent.build.prompt value, so JSON decoding restores the original text. Both local and sandbox paths share this handling. Intentional substitutions elsewhere—including identical text in another agent's prompt—remain active. Append mode and configurations without an injected replacement use ordinary serialization.

The branch incorporates main c69f3b15ff794d6cb49418117748bb16be195cbb, retaining its sandbox runtime changes, canonical package imports and documentation additions. No separate issue is needed for this focused exposure of existing native controls.

Validation

  • Current-main necessity audit: the named prompt-mode and Pi built-in-tool controls are absent from main's config models and serialized config; supplied values are ignored. Main retains append-only prompt commands and ordinary JSON config serialization. No obsolete feature hunks were found.
  • Both isolated adapter wrappers ran from independent environments installed from uv.lock --extra dev --extra sandbox through Gym's setup helper, then reused with ++skip_venv_if_present=true. Commands use gym env test +entrypoint=responses_api_agents/<pi_agent|opencode_agent> +should_validate_data=true. Pi: 335 passed, 27 skipped. OpenCode: 592 passed, 50 skipped on the complete repeat.
  • The first parallel OpenCode run had 591 passed, 1 cleanup-error fixture failure, 50 skipped (Sandbox.params() raised KeyError('')). The unchanged case passed individually on both the PR and verified main; the complete unchanged repeat passed. No assertion or source was altered to obtain that result.
  • Prompt-control and serialization suites: 15 passed, including 9 native CLI cases, using Pi 0.80.2, OpenCode 1.17.11, synthetic loopback services and isolated configuration. Tests cover replacement and tool selection in both execution paths, literal env/existing-file/missing-file references, Unicode, whitespace, quotes, backslashes and literal \\u007b text.
  • Replacing only the prompt serializer with ordinary json.dumps makes both native literal regressions fail on the missing-file reference. Removing explicit build selection makes its native replacement regression fail because the replacement marker is absent. Normal substitutions in other configuration remain active.
  • Independent verifier: 23 current-main MCP/plugin/deadline tests passed with replacement mode and literal template prompts enabled. Required-tool validation, failure recovery and continuation validation remain intact. Independent simplifier found no redundant production code.
  • Public-import policy checks: 2 passed. pre-commit run --all-files, Ruff/format and PR-relative whitespace checks passed. The private guardwords scan was skipped because its catalog was unavailable.
  • Fern validation passed with zero errors; its redirect-history check was skipped without authentication.

The merge from main was clean and required no manual source changes. Existing capture, task-tool staging, cancellation and continuation behavior remains. Earlier optional KiloCode conformance failures are outside this PR; incorporated main includes a later KiloCode fix. These local results do not claim a new-head CI result.

Compatibility and limits

Defaults preserve existing behavior. Replacement and tool selection are evaluation treatments and should be recorded. OpenCode replacement selects build even if another default agent is configured; append retains custom selection. Native environment context may remain, so replacement does not promise byte-identical whole model requests.

No new dependency, runtime image, authentication or retention change. Real harness checks use synthetic local model/MCP services; no live provider, remote sandbox, benchmark-quality or full-core qualification is claimed. New source carries Apache-2.0 SPDX headers and PR commits include DCO sign-off.

Proposed maintainer labels: feature, area:agent, needs-review.

Signed-off-by: RyaliNvidia <ryali@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nemo-automation-bot nemo-automation-bot Bot added the community-request Issue reported or requested by someone from the community label Oct 27, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added area:agent Agent harnesses and Responses API agent behavior feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer labels Oct 27, 2026
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 28, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the waiting-on-maintainers Waiting on maintainers to respond label Oct 29, 2026

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One prompt-serialization issue below.

@github-actions github-actions Bot removed the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 30, 2026
Signed-off-by: RyaliNvidia <ryali@nvidia.com>
@svcnvidia-nemo-ci svcnvidia-nemo-ci added waiting-on-maintainers Waiting on maintainers to respond and removed waiting-on-maintainers Waiting on maintainers to respond labels Oct 30, 2026
@RyaliNvidia

Copy link
Copy Markdown
Contributor Author

/ok to test 0a67829

@nemo-automation-bot nemo-automation-bot Bot removed the community-request Issue reported or requested by someone from the community label Nov 2, 2026
@nemo-automation-bot
nemo-automation-bot Bot requested review from a team November 2, 2026 17:30
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Signed-off-by: RyaliNvidia <ryali@nvidia.com>
@RyaliNvidia

Copy link
Copy Markdown
Contributor Author

/ok to test 39774fe

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 39774fe.

The earlier {env:...} / {file:...} thread is resolved. Only the injected agent.build.prompt is escaped, in both the local and sandbox paths, and substitutions elsewhere in the config stay active. I round-tripped _serialize_replacement_config on about 5,000 generated prompts (braces, backslashes, a literal \u007b, control and non-BMP characters): each decodes to the original text, and neighbouring keys, their order and sibling {env:...} / {file:...} text come through unchanged. I did not re-run the pinned-CLI cases locally.

The merge from main is clean and the branch still merges into current main without conflicts. CI is green apart from the non-blocking harness-conformance KiloCode job already noted in the description. No blockers or risks left from my side.

@yaoyu-33 yaoyu-33 added ready-to-merge PR is approved, current, and waiting for final CI or merge and removed waiting-on-maintainers Waiting on maintainers to respond needs-review PR is ready for code review and waiting on a reviewer labels Nov 2, 2026
@RyaliNvidia
RyaliNvidia added this pull request to the merge queue Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/41513131179

Merged via the queue into NVIDIA-NeMo:main with commit 84de1b5 Nov 2, 2026
51 of 54 checks passed
@RyaliNvidia
RyaliNvidia deleted the codex/g10-native-code-generation-controls branch November 2, 2026 20:28
AdamRajfer added a commit that referenced this pull request Nov 2, 2026
Brings in main up to 94545e7 (#5141, #5384, #5144, #4615, #4954, #5001,
#5175).

Textual conflict: nemo_gym/cli/env.py, imports only. This branch imports
PIPE from subprocess (the server start error capture); main imports SIGKILL
from signal (#5141, the shutdown drain). Both are kept:
"from signal import SIGINT, SIGKILL" and
"from subprocess import PIPE, Popen, TimeoutExpired".

Semantic conflict: #5141 starts each server in its own session
(run_command(..., start_new_session=True)) so shutdown can signal its
process group, while this branch starts servers through
RunHelper._spawn_server, which keeps the tail of the error stream. The
merged call passed start_new_session to _spawn_server, which did not accept
it (11 test_cli failures). _spawn_server now takes start_new_session and
forwards it to run_command, so both behaviours hold. The dry-run path keeps
this branch's process.wait(): the tail thread of _spawn_server owns the
piped stream, so main's communicate() would compete with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Adam Rajfer <arajfer@nvidia.com>
copy-pr-bot Bot pushed a commit that referenced this pull request Nov 3, 2026
Resolve the conflicts with main's move of the agent modules into nemo_gym.agents (#5228) and OpenCode
system prompt replacement (#5144): import from nemo_gym.agents, keep this PR's OpenCode config fields
beside main's prompt-agent validator, and keep main's replacement config serialization with the shared
OpenCode isolation variables.

In a sandbox session, prompt replacement now adds the build agent's prompt instead of replacing the
agent config, so request temperature and top_p still reach every OpenCode agent. A test covers it.

Import the names main moved from their new packages (scripts/migrate_imports.py).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Katherine Huang <katherineh@nvidia.com>

This branch was previously deployed

1 inactive deployment
public — 39774fe6 Deployed Nov 2, 2026 by copy-pr-bot[bot] via release / finalize / notify #9242
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior feature New capabilities, enhancements, or enablement work ready-to-merge PR is approved, current, and waiting for final CI or merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants