Skip to content

fix(opencode-agent): stable sessions under token capture - #5657

Open
bg51717 wants to merge 4 commits into
NVIDIA-NeMo:mainfrom
bg51717:biguo/opencode-capture-stability
Open

bg51717 wants to merge 4 commits into
NVIDIA-NeMo:mainfrom
bg51717:biguo/opencode-capture-stability

Conversation

@bg51717

@bg51717 bg51717 commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

What does this PR do?

Three fixes to opencode_sandboxed_agent for RL training:

  • opencode_context_limited reports whether the session ended on a full context (its last assistant message stopped on length), so a recipe can mask those sessions like overlong rows instead of folding them into opencode_failed.
  • When the sandbox kills the run (for example at its timeout), opencode_error_type takes the sandbox's error type instead of the error left in the session's last message, so mask rules keyed on timeouts match. Based on Jiaqi Zeng's c546efaaa; its provider side is already on main (fix(opensandbox): report a command that hit its timeout as a timeout #5157).
  • With token capture on, the stable-date.js plugin pins the system prompt's date to the session's first call. Source-history capture links calls by exact prompt text, so a session running past 00:00 UTC lost its parent link and the rollout was discarded. The plugin goes through the same registration and verified upload as the other plugins, and a run that finishes without its marker fails, so a future OpenCode change cannot silently bring that back.

Proposed labels: bug, area:agent.

Validation

  • gym env test +entrypoint=responses_api_agents/opencode_sandboxed_agent +should_validate_data=true (with Node available, so the .mjs plugin tests run)
  • pre-commit run --from-ref origin/main --to-ref HEAD

Rollouts

N/A

Compatibility and benchmark impact

Adds the opencode_context_limited verify field. opencode_error_type can now report the sandbox's error instead of OpenCode's. The date pin applies only with token capture on.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Documentation added or updated, or N/A with justification.
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format checks pass).
  • New source files include the required Apache-2.0 SPDX header.
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

bg51717 and others added 3 commits November 4, 2026 17:21
opencode_failed folds a context-limited session (its last assistant
message stopped on length: the generation hit the context limit, or the
next request no longer fit and the model server answered with a length
stop) together with real failures, so a trainer could not treat the two
differently. Expose it as opencode_context_limited, so a recipe can mask
such sessions like overlong rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: bg51717 <biguo@nvidia.com>
…'s last message

When the sandbox kills the OpenCode run (for example at its timeout), the
session's export still ends with the error the kill left in its last
assistant message. opencode_error_type took that message's error name, so
a timeout reported by the sandbox showed up as an OpenCode error and mask
rules keyed on the timeout never matched. Prefer the sandbox's error type.

Based on Jiaqi Zeng's c546efa. The provider side of that change, which
classifies a kill at the deadline as a timeout, is already on main (NVIDIA-NeMo#5157).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: bg51717 <biguo@nvidia.com>
…en capture

OpenCode renders today's date into every call's system prompt. Source-history
capture links a call to its parent by exact prompt text, so the first call
after 00:00 UTC in a running session found no parent and the rollout was
discarded.

With token capture on, load stable-date.js, which uses OpenCode's
experimental.chat.system.transform hook to pin the date line to the
session's first call. The plugin writes a marker once it pins a date; a run
that finishes without the marker fails like a required-MCP setup error, so a
hook or date-format change in a future OpenCode cannot silently bring the
discarded rollouts back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: bg51717 <biguo@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 5, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@bg51717
bg51717 marked this pull request as ready for review November 5, 2026 07:23
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Nov 6, 2026
Resolve conflicts with NVIDIA-NeMo#5128: clear the stable-date marker in the split
setup command, and report a missing marker as an NgFailureEvent like the
other capture checks. Timed-out test runs now carry the launch marker
that main requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: bg51717 <biguo@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant