[yaoyu-33]

Stack: this is the bottom of a stack. #5491 (finance_agent_v2 resources sessions) builds on it, and #5492 (finance_agent agent sessions plus single_agent_turn_legacy configs) builds on #5491. The branch merges main at 379ae226d; its diff against main is only resources_servers/finance_sec_search/.

What and why

finance_sec_search keeps per-rollout state: _data_storage, which holds parsed documents, and _session_start_times, which tracks each rollout's time budget. But it only served the agent-run seed, so an Environment Server's episode seed got {} back. It also had no close_resources_session, so a rollout that never reached /verify left its state in memory for as long as the server ran.

This PR implements the resources sessions contract as tightened in #4679:

• Episode seed. /seed_session accepts ResourcesSeedSessionRequest | BaseSeedSessionRequest. An episode seed is keyed by resources_session_id and serialized per ID with ResourcesSessionLocks.locked.
  • A retried seed returns the open session without clearing its documents.
  • A seed that binds the ID to another episode or task raises ResourcesSessionConflictError (409).
  • The cookie's SESSION_ID_KEY is bound to resources_session_id only after these checks pass, so a refused seed's reply doesn't hand out the live session. The same fix was made in GDPVal in fix(resources): tighten the session contract for close, refusals and lost sessions #4679.
  • The agent-run seed still resets the session and returns {}.
• Close. close_resources_session releases the session's storage, time budget and episode seed by ID. That works with or without the session cookie, which covers the case where the seed's reply was lost.
  • It then fences the ID with ResourcesSessionFences.fence. Until the fence expires, a later episode or agent-run seed gets 409.
  • A close that names another episode gets 409.
  • Repeated closes succeed, and so do closes of sessions that were never seeded.
  • Releasing only removes entries from in-memory dicts, so it has no failure case to raise.
• Verify releases an agent-run session. finance_agent runs behind legacy_agent, which never calls /close_session, so its verify releases the session. An episode session keeps its seed and documents until its close, so that close still checks the seed's episode. A /verify for a session that was already closed returns a masked lost_session_verification sample even though the score comes only from submit_final_result and the judge; this is kept on purpose so every server on the contract treats an out-of-order verify alike (single_agent_turn always verifies before it closes).
• Stateful tools report a lost session. parse_html_page and retrieve_information raise ResourcesSessionLostError (404, failure_kind=session_lost) when the session isn't open. Before, they created storage that nothing would ever release.
  • finance_agent passes a non-200 tool body to the model unchanged.
  • The search tools don't touch session state, so they are unchanged.
• A retired attempt releases its session. A checkpoint retire refuses the retired attempt's later requests, so its verify or close never arrives. retire_session_state now releases the session, as workplace_assistant and example_session_state_mgmt do.
• Docs. The README gains a Sessions section: what a session holds, episode and agent-run seeds, release at verify and close, the 404 for a lost session, and session_ttl_seconds.
• Abandoned sessions are evicted. Each seed discards sessions older than the new session_ttl_seconds (default 6 h). That limits the leak on the agent-run path when a rollout fails before /verify. A verify for an evicted session is masked with failure_kind=session_expired, and an evicted episode session is fenced as after a close. The TODO in SimpleResourcesServer.close_resources_session plans a general version of this.

Why the cookie is bound explicitly

An earlier review said binding the cookie is unnecessary because the session router already names the cookie by resources_session_id. That only holds when the router middleware is installed, which run_webserver does. An app built with setup_webserver() alone gets a random cookie ID. #4679's close_resources_session docstring now says the same thing. The first version of this PR failed the route-level conformance test until the cookie was bound.

Not in this PR:

• finance_agent_v2 (same pattern): feat(finance_agent_v2): serve the resources sessions API #5491, stacked on this PR.
• Overlap with feat(agent): honor skip_verification in six agents that ignored the global flag #4154's /discard_session in the same file: if it lands first, its discard should route to close_resources_session.
• The agent side, finance_agent agent sessions and the configs that run it behind single_agent_turn_legacy: feat(finance_agent): enable session execution in default configs #5492.

No separate public issue: this is one server's step in the resources-sessions migration that #4679 defines; the stack's other PRs are listed above.

Compatibility

• New config field: session_ttl_seconds (default 21600). It must be longer than the longest rollout.
• Behaviour change: calling parse_html_page or retrieve_information on a session that is not open now returns 404 with failure_kind=session_lost. Before, it returned an observation. finance_agent always seeds before calling tools.
• Behaviour change: a verify for a session that outlived session_ttl_seconds is masked (failure_kind=session_expired) instead of scored, because its tool calls after the eviction got 404.
• The agent-run seed and verify responses are otherwise unchanged. No benchmark-result impact is expected.

Current stack validation (November 6)

Real-model validation is now complete at stack head f0b200ce6568ff02cbc2d51a6dd5d46ccb3d6b5b in #5492. The resources_servers/finance_sec_search/ implementation and tests are identical to this PR's current head; only README/config changes differ in the top PR. The resource suite passed 116 tests there, and full pre-commit checks passed (the unavailable private guardwords catalog was skipped). The entire selected stack validation passed 822 tests.

The canonical resources config passed a bounded real-NIM HTTP smoke with reward 1.0, actual policy/retrieval/judge calls, successful close responses, zero active agent state and empty captured document/time maps. The judge recovered from two upstream 429s. The renamed resources seed map is covered by source/tests, not directly captured by that driver. See #5492's validation for the complete scope and compatibility notes.

This supersedes the earlier no-real-model statements below for the integrated stack. It does not establish standalone lower-PR episode activation, CLI collection, live SEC/Tavily/Tiingo coverage, full benchmark results or relay score parity. Historical evidence retains its original revision.

Historical validation

Update on 788316605 (merged main at c6cad9e52, added retire_session_state and the README section):

• resources_servers/finance_sec_search/.venv/bin/python -m pytest resources_servers/finance_sec_search/tests: 106 passed, including the new test_retired_attempt_releases_its_session.
• pre-commit run --files on the three touched files: clean.
• Both commits are signed off.

Earlier validation, on the pre-merge head:

All on the final commit, on top of #4679 a49da9c9b:

• gym env test +entrypoint=resources_servers/finance_sec_search +should_validate_data=true: 104 passed.
  • This includes check_resources_session_contract(..., keeps_state=True, holds_state=...) from fix(resources): tighten the session contract for close, refusals and lost sessions #4679, run through the real routes. It covers the conflicting seed, the conflicting close, a cookie-less close after a lost seed reply (state released), and late seeds.
  • It also includes 8 server-specific session tests.
• Mutation check: moving the cookie bind back in front of the checks makes test_typed_seed_for_another_episode_is_refused_without_its_cookie fail.
• Deployed single-worker path, probed with setup_webserver() plus install_session_routing(socket_dir=None, ...):
  • episode seed: 200;
  • retried seed: 200;
  • close: 200, storage released;
  • repeated close: 200;
  • late seed: 409;
  • agent-run seed: 200 with {}.
• pre-commit run --all-files: clean.
• DCO: the commit is signed off.

Review fixes on 031b0022d7c9 (team review)

Merged main (clean) and fixed what a review team found:

• A agent-run seed for a session that holds a episode seed now answers 409. Before, a {} seed, or a model calling a tool named seed_session through the grant, wiped the episode session's documents and restarted its time budget.
• Tests: the eviction test now installs routing through the public install_session_routing instead of setting the router's app.state attribute by hand, so a rename in core fails it (and the test no longer imports a submodule). New tests cover the episode seed's cookie reaching the tools through the routes (removing the explicit bind now fails it), the route-level contract check with a verify body, a seed that keeps other sessions within their TTL, and a re-sent verify before close getting the same score.

Validation at 031b0022d7c9: pytest resources_servers/finance_sec_search/tests: 111 passed (worktree modules confirmed via __file__); pre-commit run --files on the changed files: passed. Not run: a real-model smoke at this head (the paired smoke below is from 33939a380).

Review fixes on 2fa1099b4 (second team review)

Merged main (now 379ae226d). Main's #5083 split ResourcesSessionFence into ResourcesSessionLocks and ResourcesSessionFences with no alias, so after the merge the server no longer imported. Fixed, plus what the review found:

• Port. ResourcesSessionLocks.locked, and ResourcesSessionFences (fence, get, raise_if_fenced); the tests use owned_session_routing. Names follow refactor(sessions): one name per session lifecycle concept before release #5083's glossary: episode seed/session, agent-run seed.
• Close after verify. Verify no longer drops an episode session's seed, so a close that names another episode is refused after verify too, and the documents stay until the close (as example_single_tool_call does).
• Expired sessions. A session evicted after session_ttl_seconds is masked at verify as session_expired instead of scored; an evicted episode session is fenced; a new agent-run seed for the ID clears the mark.

Validation at 2fa1099b4: pytest resources_servers/finance_sec_search/tests with the server venv: 116 passed (worktree modules confirmed); tests/unit_tests/test_session_contract_repo.py + tests/unit_tests/config/test_global_config.py: 233 passed; pre-commit run --files on the PR's files: clean. CI at 031b0022d failed only openclaw_agent/tests/test_exec_transport.py::test_exec_only_supervision_leaves_task_children_for_sandbox_owner[cancel] (a known four-workers-per-shard flake, issues #5638 and #5672) and main's test_each_cleanup_has_its_own_budget timing test; the session-routing e2e failure is fixed by main's d45f9cf, now merged in. Not run: a real-model smoke at this head.

Rollout evidence

Environment Server plumbing smoke on this stack (no real model). No model endpoint or API key is configured on the machine that ran it, so this used a scripted OpenAI-compatible stub for the policy, retrieval and judge, behind real openai_model servers. gym env start --config resources_servers/finance_sec_search/configs/finance_sec_search_single_agent_turn.yaml (from #5492) at #5492's head, then gym eval run --no-serve --agent finance_agent on the first row of data/example.jsonl:

• Result type single_agent_turn_legacy, reward 1.0 (judge [[2]]), no failure rows.
• Tool sequence parse_html_page → retrieve_information → submit_final_result. retrieve_information found the document parse_html_page stored, so the episode seed's session reached both tools through the agent's direct HTTP grant.
• The same input through the unchanged relay config (finance_sec_search.yaml, result type legacy_agent) gave the same tool sequence and reward.

This proves the wiring, not model quality. The paired real-model smoke below is from the earlier head.

Paired real-model smoke on an internal cluster. Setup: Qwen3-8B on vLLM (TP=1, temperature 0). The run drives the production legacy_agent → finance_agent → finance_sec_search path over real local HTTP, behind the session router:

1. seed
2. parse_html_page (a synthetic financial report)
3. retrieve_information
4. submit_final_result
5. verify, with the stock judge

Retrieval and the judge also run against the real model. Both arms use identical task settings.

Current head, rerun after the rebase. One job held the model and controller; it completed 0:0 in 2m21s, and model shutdown returned 0.

Revision	Model calls	Tool sequence	Retrieved figure	Judge / reward	State after verify
33939a380 (this PR)	5	parse, retrieve, submit	172.90 USD million	[[2]] / 1.0	all finance maps empty
a49da9c9b (#4679, parent)	5	parse, retrieve, submit	172.90 USD million	[[2]] / 1.0	all finance maps empty

• All HTTP calls returned 200. The parsed document was stored before verify.
• Both arms ended with mask_sample=false and no failure kind.
• In both arms, the imported server sources hash-match the exact git revisions.
• The focused finance resources and agent suites passed 171/171 on Linux at 33939a380, with no skips.

An independent reviewer had already run the same paired smoke on the pre-rebase head (ac5aaec46 vs 2b5faf08d), with the same outcome.

This is plumbing and compatibility evidence, not benchmark accuracy. Not covered: live SEC search, multi-worker routing, and cancellation. The episode-seed path has no real caller until #5492, so the route-level conformance test covers it.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[svcnvidia-nemo-ci]

HOLD — one RISK: TTL eviction frees finance data but leaves the evicted sessions open in lifecycle tracking.

Reviewed both changed files and traced the affected seed, close, verify, routing, and checkpoint contracts. Static inspection only; tests and model rollouts were not executed in this read-only review.

Note: formal approval is not available for this request, so this review is posted as a comment only.

Review generated by codex. This was a full review. Request a full review with /review reset=true.

[yaoyu-33]

/ok to test 33939a3

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-4948-65276d07ab37.docs.buildwithfern.com/nemo/gym

[yaoyu-33]

/ok to test 65276d0

[yaoyu-33]

/ok to test bee3a52

[yaoyu-33]

/ok to test 031b002

[yaoyu-33]

/ok to test 2fa1099

[svcnvidia-nemo-ci]

SHIP — no eligible findings remain in this follow-up.

I verified that the prior eviction finding is fixed: stale-session eviction now ends checkpoint tracking and routing ownership. The regression test checks that the abandoned session’s lifetime expires while the fresh session remains held. I also checked the follow-up lock/fence integration, seed guards, verification and expiry behavior, retirement cleanup, and session documentation against their directly affected contracts.

The previously reviewed tree was unavailable, so I couldn’t compare the complete inter-review source delta. This was a targeted follow-up, not a fresh whole-PR review. I found no proven introduced regression above the prior moderate impact ceiling. Static inspection only; I didn’t execute tests or model rollouts.

Note: formal approval is not available for this request, so this review is posted as a comment only.

Review generated by codex. This was a findings-only review. Request a full review with /review reset=true.