Skip to content

feat(environment): sign episode tokens for Environment Server tool grants - #5346

Open
ffrujeri wants to merge 1 commit into
mainfrom
ffrujeri/episode-tokens
Open

ffrujeri wants to merge 1 commit into
mainfrom
ffrujeri/episode-tokens

Conversation

@ffrujeri

@ffrujeri ffrujeri commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

What changed

nemo_gym/server_runtime/episode_tokens.py adds EpisodeTokens, which mints and reads the credential an Environment Server grants for one episode's tools:

  • mint(episode_id) returns a signed token and the EpisodeTokenClaims (episode ID and a random nonce) that the server keeps.
  • read(token) returns the claims of a token this server signed, or None for anything else, including an MCP session token signed with the same key.
  • EpisodeTokenClaims.matches compares the episode and the nonce in constant time.
  • EPISODE_TOKEN_HEADER names the header that carries the token on a direct HTTP tool call.

Why

This is the first PR of a stack that lets an Environment Server own an episode's state and tools without a Resources Server (Linear GYMINT-641, under the design in the user-simulation RFC's layer 1, BaseEnvironmentServerWithTools). The tool calls arrive as separate requests, so they need a credential for the episode:

  • A rollout ID is predictable: <task index>-<rollout index> unless the caller sets one, and the agent sees its own in its capture path. A bare ID would let a policy call another episode's tools, for example to read hidden knowledge without paying for it.
  • A Gym server's signing key is derived from its class and instance names (SimpleServer.get_session_middleware_key), so a signature alone is forgeable.

The nonce makes the token unguessable; the server accepts it only while it holds the same claims. The signature keeps the claims intact, and lets a worker that does not hold the episode read which episode a token names, so a later PR can route the call to the worker that does.

Nothing uses the module yet; the next PR in the stack, the Environment Server base class that serves episode tools, does.

Validation

Command Result
pytest tests/unit_tests/server_runtime/test_episode_tokens.py tests/unit_tests/test_import_linter_coverage.py (after the rebase onto main at dda609f86) 15 passed. The module is now nemo_gym/server_runtime/episode_tokens.py, next to the other server runtime modules
pytest tests/unit_tests/server_runtime/test_episode_tokens.py 11 passed: round trip with an attempt, a new nonce for each grant, another episode's token, a forged token with a guessed nonce, a tampered token, and six strings this server did not mint (another key, an MCP token with the same key, a missing nonce, an invalid episode, a payload that is not an object, garbage)
pre-commit run --files nemo_gym/server_runtime/episode_tokens.py tests/unit_tests/server_runtime/test_episode_tokens.py passed

Rollout evidence

N/A: a library module that no server uses yet. The BIRD-Interact migration onto the Environment Server base class will run rollouts.

Compatibility

New module only. No existing behavior changes.

🤖 Generated with Claude Code

@copy-pr-bot

copy-pr-bot Bot commented Oct 30, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ffrujeri

Copy link
Copy Markdown
Contributor Author

/ok to test e050173

@ffrujeri

ffrujeri commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 3dc43be

@ffrujeri

ffrujeri commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test fa0b016

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

@ffrujeri
ffrujeri marked this pull request as ready for review November 6, 2026 17:07
@ffrujeri
ffrujeri requested a review from a team as a code owner November 6, 2026 17:07
…ants

An Environment Server that serves an episode's tools needs a credential
that names one episode and cannot be guessed. A rollout ID is
predictable, and a Gym server's signing key is derived from its class
and instance names, so neither is enough. EpisodeTokens signs the
episode's capture key with a random nonce; the server accepts a token
only while it holds the same claims for that episode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Felipe Vieira Frujeri <ffrujeri@nvidia.com>
@ffrujeri
ffrujeri force-pushed the ffrujeri/episode-tokens branch from fa0b016 to 8f232d4 Compare November 6, 2026 17:15
@ffrujeri

ffrujeri commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 8f232d4

This branch was previously deployed

1 inactive deployment
public — 8f232d48 Deployed Nov 6, 2026 by copy-pr-bot[bot] via release / finalize / notify #10671
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation feature New capabilities, enhancements, or enablement work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants