[qiaochuz-nv]

What does this PR do?

Makes six more agents report rollout token usage per the documented contract: the sum over the policy-model
responses that finished, with an unknown detail kept unknown. Before this PR they reported either nothing or
fabricated zeros:

Agent	response.usage before this PR	After
tau2	always null	sum of every agent-model call, incl. responses tau2 or the malformed-tool-call retry discarded; user simulator excluded
osworld_agent	always null	sum of Gym's own policy calls (gym_pyautogui, prompt_agent*, nemotron/qwen omni runners); m3_agent / pointer_agent stay null
anyterminal_agent	always null (inner agent's usage dropped)	the inner agent's usage, as anyswe_agent already does
vcqa_agent	summed, but cached_tokens / reasoning_tokens = 0 from the 2nd call on	summed details; null when a call reported none
conversational_tool_use_simulation	same as vcqa	same
image_tools_agent	same as vcqa	same

nemo_gym/reward_profile.py copies response.usage into each rollout row, so these rows had no token counts or
wrong cache/reasoning counts. Contract: rollout-evidence.mdx ("Report the agent's response usage as the sum over
model responses that finished … Omit usage only when a finished response's usage was lost"); policy calls only per
rollout-health.mdx "Binding Policy-Model Calls" and switchyard.mdx.

Follow-up to #4716 and #4753, NVBug 6900411. No separate issue: the
6900411 sweeps found these agents.

Uses chat_completion_usage_to_response_usage from #4753 (merged). Rebased onto main after #4753 merged, so the
diff is this PR's two commits only: the original change and the review-round fix.

Cause

• tau2/app.py builds the response dict without usage; the per-message counts only feed min/mean/max metrics.
  The messages are not a complete record anyway: tau2-bench regenerates an empty agent message (up to 5 attempts,
  agent/llm_agent.py) and Gym's malformed-tool-call retry discards attempts.
• osworld_agent/app.py: _build_model_fn returns only the text and _build_messages_model_fn only the normalized
  message, so resp.usage is dropped; _build_response sets no usage.
• anyterminal_agent/app.py: _inner_responses rebuilds the response from the saved output and tools only.
• vcqa_agent, conversational_tool_use_simulation (_merge_usage), image_tools_agent (_response_usage_add)
  hand-roll the merge and write cached_tokens = 0 / reasoning_tokens = 0.

Fix

• tau2: a ContextVar set in Tau2Agent.run(). Gym's Tau2ToolValidatingAsyncOpenAI (already installed as
  tau2's client) records each finished response whose model is the agent model name Gym configures, so retried
  and discarded responses count and user-simulator calls (other model name, same server) do not. run() adds the
  sum to success and failure rows: a rollout that failed on a malformed tool call still spent those tokens.
• osworld: _PolicyUsage is passed into both Gym-owned callers and records right after create() returns,
  before finish-reason or normalization rejections (that response finished). The Ray task returns usage;
  _build_response sets it. m3_agent and pointer_agent call models through their own Anthropic clients
  (pointer_agent also through gate/planner/verifier/summarization models), so Gym cannot attribute those calls;
  their usage stays null, as does the error path.
• In both, a usage block that does not parse (e.g. a provider's prompt_tokens: null) counts as unknown; it never
  fails the policy call or the rollout. A finished call whose usage is unknown makes the rollout's usage null, not
  the other calls' subtotal (rollout-evidence.mdx: omit usage when a finished response's usage was lost). tau2 also
  keeps an empty record null: its recorder sees only calls made through its own client, so an empty record does not
  prove that no policy call finished.
• anyterminal: keep saved.usage.
• vcqa / conversational simulation / image_tools: accumulate_response_usage, like simple_agent.
• Tests: for the three zeroing agents, two calls with distinct usage asserting the exact sum, plus a call without
  details → null details. anyterminal: usage kept, and null without response.json. osworld: sum over three
  Gym-owned calls incl. one rejected for finish_reason=length, and null for a runner that calls models through its own client. tau2: client
  records a discarded malformed attempt and not a user call; end-to-end through the real tau2-bench orchestrator
  (empty agent message regenerated, user simulator on the same server) asserting the exact sum; failure row usage.
  tau2 and osworld: a malformed usage block is recorded as unknown without raising, and a known plus an unknown call
  (both orders) gives null; two concurrent tau2 runs keep their own sums.

Before the fix

This branch's tests against main's (aa5d277e3) six app.py files:

$ pytest responses_api_agents/<agent>/tests -k usage   # each agent's venv; main's six app.py swapped in
vcqa_agent           2 failed   cached_tokens 0 != 120, reasoning_tokens 0 != 10; unknown details 0 != None
conversational_sim   2 failed   same numbers
image_tools_agent    2 failed   same numbers
anyterminal_agent    1 failed   usage None (inner response.json had 5300/812/6112)
osworld_agent        2 failed   KeyError: 'usage' (child result has none); response.usage None
tau2                 3 failed   usage None after 3 agent calls through the real tau2 orchestrator;
                                recorder empty; failure row usage None

After the fix

$ gym env test +entrypoint=responses_api_agents/<agent>   # full suites
tau2 38 passed | vcqa_agent 51 passed, 1 skipped | conversational_tool_use_simulation 38 passed
image_tools_agent 10 passed | anyterminal_agent 86 passed | osworld_agent 387 passed

Validation

• Agent suites (gym env test +entrypoint=responses_api_agents/<agent>): all six green (counts above).

• pre-commit run --all-files: clean.

• Rollout evidence (hosted provider models, model-call capture on, one example row, main control vs this branch):
  main 0/2, this branch 2/2. Policy calls = tau2 requests with the agent model name / conversational-simulation
  requests carrying the domain tools; the other captured calls are the user simulator and judge.

  Case	main	This branch
  tau2 (Nemotron-3-Nano-30B-A3B)	5 policy calls of 9 captured, sum 28089 in / 6065 out → usage null	5 of 7, sum 29649 / 5840 / 35489 → usage 29649 / 5840 / 35489
  conversational simulation (nemotron-3.5-super-vl-preview)	5 of 17, sum 25396 / 2008, cached 6720, reasoning 955 → usage 25396 / 2008, cached 0, reasoning 0	5 of 13, sum 26792 / 2324, cached 11200, reasoning 1024 → usage identical

  (Separate runs per checkout, so the dialogues and counts differ; each row is compared with its own capture. Run on
  an earlier revision of this commit with the same accounting; the final commit adds only the malformed-usage handling
  and the rebase onto current main.)

• Not run live: osworld_agent (VM image), anyterminal_agent and vcqa_agent (Apptainer), image_tools_agent
  (no in-repo example data for this agent). Their changes are covered by the unit tests above.

Review round 1

yaoyu-33's review: tau2 and osworld summed the known calls when another finished call's usage was unknown. Both now
report null in that case (commit fix(agent): report unknown usage when a finished policy call lost its usage).
Also added: a tau2 test with two concurrent runs, and comments on osworld's timeout and Ray-failure returns.
vcqa_agent, image_tools_agent and conversational_tool_use_simulation still sum the known calls when one lacks
usage, as before this PR; per the review, a shared completeness helper for every agent follows separately.

Compatibility

Rows from these agents now carry token counts where they had null, and cache/reasoning sums (or null) where
they had 0. tau2 failure rows now carry the usage of the calls that finished. No config or API change.

🤖 Generated with Claude Code

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[qiaochuz-nv]

/ok to test 07df8b9

[yaoyu-33]

Thanks for sweeping these six agents onto the documented usage contract; the tau2 ContextVar recorder, the osworld _PolicyUsage, and the exact-sum tests (including the discarded-attempt and user-simulator cases) look solid. I reviewed only the last commit, since the branch still carries #4753's commit.

Needs a change (1 inline, 2 anchors): a finished policy response with missing or malformed usage is recorded as unknown, but the sum then skips it, so the rollout reports the other calls' subtotal as a complete total. rollout-evidence.mdx / rollout-health.mdx say a finished response whose usage was lost should make the usage unknown. See the inline comments on tau2 and osworld.

Follow-ups (not blocking):

• vcqa_agent, image_tools_agent and conversational_tool_use_simulation have the same shape (accumulate_response_usage(total, None) returns the total). It is no worse than before this PR, so a shared completeness helper in a follow-up would keep all agents consistent.
• tau2: a small test that gathers two Tau2Agent.run calls with different usages and asserts each row keeps only its own sum would pin the ContextVar isolation.
• osworld timeout and Ray-failure rows report usage: null (as the description says); a one-line comment at those return sites would make that intentional-looking.

PR hygiene: please rebase onto main once #4753 merges so the diff shrinks to this commit. The live rollout evidence covers tau2 and conversational simulation only, and the description gives reasons for the other four agents.

[yaoyu-33]

tau2/app.py:146

When a finished policy response has no usable usage this appends None, and the fold in run() then skips it (accumulate_response_usage(total, None) returns total), so the row reports the other calls' subtotal as if it were complete. rollout-evidence.mdx says to omit usage when a finished response's usage was lost. Could you keep response.usage as None when any recorded entry is None (as scicode_agent does with usage_complete) and add a test with one known and one unknown call, in both orders?

[qiaochuz-nv]

Thanks, fixed in 45cd858. run() now reports NeMoGymResponseUsage.sum_from_list(...) only when every recorded policy call has usage, and None otherwise, so a lost usage block no longer leaves a subtotal. An empty record also stays None: the recorder only sees calls made through Tau2ToolValidatingAsyncOpenAI, so an empty record can't prove that no policy call finished.

Tests: test_run_usage_is_unknown_when_a_finished_policy_call_lost_its_usage (the unknown call first and last, through /run), plus the isolation test you suggested: two Tau2Agent.run calls under asyncio.gather with different usages, each row keeping only its own sum.

[yaoyu-33]

osworld_agent/app.py:585

Same gap here: a malformed or missing usage block becomes None and accumulate_response_usage(self.total, None) leaves the earlier subtotal in place, so the result looks like a complete sum. Could _PolicyUsage track a complete flag and report None once any finished response lacked usable usage, with a test for known and unknown calls in both orders?

[qiaochuz-nv]

Fixed in 45cd858. _PolicyUsage now tracks complete, and reported() returns None once any finished response lacked usable usage; the child result uses reported().

Tests: test_child_result_usage_is_unknown_when_a_finished_response_lost_its_usage (known and unknown calls in both orders, checked through _build_response). The malformed-block test now also checks that a later known call doesn't restore a total. The timeout and Ray-failure returns have a comment saying their usage stays unknown.

[qiaochuz-nv]

/ok to test 45cd858

[qiaochuz-nv]

@yaoyu-33 thanks for the review. Round 1 is in 45cd858:

• tau2 and osworld now report null when a finished policy call's usage is missing or malformed, instead of the other calls' subtotal (details inline).
• Rebased onto main now that fix(agent): sum token usage across every model call in four loop agents #4753 has merged, so the diff is just this PR's two commits.
• Follow-up, tracked in [NVBug 6978294] Report null usage when a finished call's usage is lost, and zero when no call finished #4940: a shared completeness helper. It covers the same skip-None shape in vcqa_agent, image_tools_agent and conversational_tool_use_simulation, and in the fix(aviary): sum token usage across every model call in an episode #4716/fix(agent): sum token usage across every model call in four loop agents #4753 agents. It also covers the "zero when no call finished" case (conversational_tool_use_scenario_generation when every request fails). I'll start it once this PR merges, so one change covers every agent.
• CI: the one unit-test failure (test_checkpoint_agent.py::...[restart_only_resources_seed]) is outside this diff and passes 5/5 locally on this head. I've re-run the failed jobs.

[qiaochuz-nv]

/ok to test 45cd858

[yaoyu-33]

Approving at 45cd858: usage is now reported as unknown when a finished policy call lost its usage (tau2 and osworld), which resolves the inline comments.

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/39856697805