[gwarmstrong]

What does this PR do?

Let Codex run against chat-completions models (e.g. self-hosted on vLLM) through vllm_model's Responses-to-chat conversion. Codex speaks the Responses API, and today that conversion refuses or loses part of every Codex request. #4880 covered one part: with custom_tool_strategy: function_bridge, Codex's freeform apply_patch reaches the model as a function tool and the call comes back as a custom_tool_call. This PR adds the rest, as new settings, all off by default:

• forward_unvalidated_reasoning_effort: pass an effort the pinned SDK doesn't list (Codex sends max) to the chat route as reasoning_effort instead of answering 422, and render it the same way when measuring context (/context/measure) so the measured prompt matches generation. drop_unrepresentable_request_fields (feat(model): optionally omit unsupported Responses fields #3976) now also drops reasoning.context auto/all_turns, since the conversion already forwards every reasoning item; current_turn is still refused.
• inline_tool_output_images: view_image output goes to the model as an image_url part of the chat tool message (feat(model): accept image parts in chat tool messages #4688 added image parts there) instead of being refused or replaced by a placeholder. GenRM, which builds its own converter, honors it too.
• stream_keepalive_s (model servers that build their own responses: this conversion, azure_openai_model, inference_provider): keepalive events while a buffered streaming response is computed. A buffered response sends no headers until the answer is ready, and Codex's stream idle timer starts only once a stream has begun (checked with Codex 0.154.0 at its stock 300 s timeout and a 400 s turn), so a direct connection doesn't need this; it is for a client behind something that opens the stream early or drops idle connections, such as a proxy or a relaying gateway. The events name the response before it exists, so the response takes that id, and capture records the same id. They are liveness, not output: a checkpoint holds the call as undelivered and its capture row is published before the first output or terminal event. Model-call capture records Gym's own usage for these streams too, as fix(capture): keep unknown usage counts for streamed /v1/responses calls #3317 does for buffered ones.

Saved model-call records also list custom_tool_call items among a response's tool calls, so a bridged apply_patch call is recorded like any other.

Options that cannot take effect are refused at startup: forward_unvalidated_reasoning_effort with use_completions_api (the completions body has no reasoning_effort) or on vllm_model_with_compaction, the conversion options with is_responses_native, and stream_keepalive_s on a server whose responses carry an upstream Responses API's ids (is_responses_native, openai_model, switchyard_model), which must reach the client unchanged.

No issue: opt-in conversion settings for one harness's requests; no default changes.

Validation

• vllm_model: 896 passed; vllm_model_with_compaction: 9 passed; genrm_model: 12 passed; converter, streaming, base-model, model-call and function-bridge unit tests: 579 passed; checkpoint unit tests: 786 passed. pre-commit clean.
• A Codex-shaped streaming request (effort max, context: all_turns, include, and apply_patch with its lark grammar) round-trips through vllm_model with these options and custom_tool_strategy: function_bridge: the call comes back as a custom_tool_call, and its replay on the next turn reaches the engine as the matching chat tool call and tool message.
• Codex 0.160.1's first request captured at a local stub: the DeepSeek catalog sends effort max and include; the GPT-6 catalog adds context: all_turns.
• Earlier runs on an older main: an earlier version of this change, with its own custom-tool bridge in place of feat(model): bridge Responses custom tools over function-only chat backends #4880's, served Codex 0.154.0 with self-hosted DeepSeek V4.1 Flash on Terminal-Bench-Science, 70 tasks × 3 trials, 8 h: 31/210, against the public leaderboard's 33/210 for the same model and harness. A 2-task, 30-minute smoke: 202 model calls, all completed, effort max on every turn, 21 view_image calls passed inline.

Rollouts

N/A: model-server options.

Compatibility and benchmark impact

Defaults unchanged. Users who already set drop_unrepresentable_request_fields will now also have reasoning.context auto/all_turns dropped instead of refused.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[gwarmstrong]

/ok to test ff92c43

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-5155-66d4c71fa5ec.docs.buildwithfern.com/nemo/gym

[svcnvidia-nemo-ci]

HOLD — 2 BLOCKERs and 1 RISK.

The new Responses keepalives bypass checkpoint capture publication and announce a different response ID from the eventual successful completion. The inherited image-inlining option is also ignored by GenRM's converter override.

Reviewed all 14 changed files and directly affected conversion, capture, checkpoint, and subclass paths. This was a static review; no tests were executed.

Note: formal approval is not available for this request, so this review is posted as a comment only.

---

Inline comments (escaped v1):

• nemo_gym/_checkpoint/model.py line 933: BLOCKER — Lifecycle keepalives bypass capture publication

  With checkpoint.policy, external token staging, and stream_keepalive_s enabled, let a streaming Responses call exceed its first interval. The new created/progress events enter this branch before generation finishes. deliver_response() marks the ticket started, but publish() returns because no deferred commit exists yet. Finalization subsequently defers the completed ledger row, while every answer chunk skips publication because ticket.response_started is already true. Thus the client receives an answer without its capture row. Source-history capture leaves a pending intent, and the cleanup sweep skips deferred commits. Checkpoint preparation also incorrectly waits for this still-generating call as a draining answer.

  Could you classify lifecycle-only events as liveness, not answer delivery? For the inspected producer's complete, one-event-per-chunk framing, replace this condition with:

  lifecycle_only = (
      event_stream
      and body.startswith((b"event: response.created\n", b"event: response.in_progress\n"))
      and body.endswith(b"\n\n")
      and body.count(b"\n\n") == 1
  )
  if body and not (event_stream and body == SSE_KEEPALIVE) and not lifecycle_only:

  This suggestion needs verification before adoption. The single-event guard preserves delivery/publication when a chunk contains both lifecycle and answer events. The invariant is that the first actual output or terminal event—not liveness—starts delivery and publishes the deferred row.

  Proposed acceptance: delay generation behind the real gate/external-capture harness. After created/progress events, assert the ticket remains undelivered and checkpoint prepare completes without releasing generation. Resume and finish; assert exactly one committed ledger row exists before output/terminal bytes leave, with no pending intent. Also check an empty-output terminal response triggers publication. Retain comment-keepalive cutting and held-output assertions, publication-before-answer assertions, and draining of already-started output while admission is closed. These checks were not executed.

• nemo_gym/base_responses_api_model.py line 660: BLOCKER — Successful keepalive streams change response identity

  For a call that outlasts stream_keepalive_s, this generator announces a freshly minted resp_A in created/progress events, then passes the backend response unchanged to _serve_responses_sse(). If that response has ID resp_B, the terminal synthesizer emits resp_B, with no terminal event for resp_A. A client tracking response state by the announced ID therefore receives completion for an unknown response and leaves the announced response in progress. This affects slow successful and incomplete turns; the exception path correctly reuses the announced ID.

  Minimal reproduction: configure a 0.02-second interval and have responses() return a response with fixed ID resp_final after 0.3 seconds. The opening ID is a generated UUID-based ID, while response.completed.response.id is resp_final. The current slow-success test checks ID consistency only among keepalives, so it accepts this mismatch.

  Could you arrange for lifecycle and terminal events to identify one actual served response? Select/reserve its identity before announcing it, or use a liveness representation that does not announce a different response. This needs coordination with capture rather than a terminal-only ID rewrite: _invoke_responses() captures before this point, and external finalization records the served envelope ID. Preserve provider-issued IDs on native Responses paths and the equality between client-visible IDs and capture IDs. I cannot recommend a safe one-line replacement without resolving those identity contracts.

  Proposed acceptance: extend the delayed-response test to assert that every lifecycle and terminal envelope has the same ID, for completed and incomplete responses, and that a client keyed by the created ID reaches a terminal state. Retain the existing late-failure same-ID assertion and fast-call event sequence. For captured calls, assert the ledger/token entry ID equals the terminal wire ID; for native Responses, assert the provider's ID remains unchanged. These are proposed checks, not executed tests.

• responses_api_models/genrm_model/app.py line 45: RISK — GenRM silently ignores the inherited image-inlining option

  GenRMModelConfig inherits inline_tool_output_images, but this converter override does not forward it. The PR adds forwarding only in VLLMModel.get_converter(), which GenRM bypasses. Consequently, setting inline_tool_output_images: true on GenRM still rejects a function_call_output containing an input_image; with dropping enabled it substitutes the placeholder instead. The accepted option should retain the image as an image_url tool-message part.

  Could you add the missing keyword to GenRMModelMixin.get_converter()?

              custom_tool_strategy=self.config.custom_tool_strategy,
              inline_tool_output_images=self.config.inline_tool_output_images,
  

  Verify before adopting. This preserves the existing token-ID, reasoning-parser, and custom-tool strategy arguments and does not change GenRM's role preprocessing.

  Proposed acceptance: extend the existing converter-wiring test setup with inline_tool_output_images=True; convert a tool output containing {"type":"input_image","image_url":"https://example.com/a.png","detail":"high"} and assert its content is [{"type":"image_url","image_url":{"url":"https://example.com/a.png","detail":"high"}}], including when dropping is enabled. With the option false, retain strict rejection and placeholder behavior. Keep the existing function_bridge assertion and the GenRM custom-role assertions. Static reasoning only; tests were not run.

Review generated by codex. This was a full review. Request a full review with /review reset=true.

[gwarmstrong]

/ok to test 66d4c71

[gwarmstrong]

All three findings reproduced, and each now has a test that fails on the previous head (ff92c43).

1. Lifecycle keepalives and capture publication (BLOCKER). The checkpoint gate now treats a chunk that is exactly one response.created or response.in_progress event as liveness, like a keep-alive comment (a47b5d3). The check is a shared helper, is_responses_lifecycle_chunk, which time-to-first-token now uses as well; a chunk that carries any other event still starts delivery. test_responses_lifecycle_events_are_liveness_not_the_answer sends the lifecycle events before generation finishes: the call does not start delivery, checkpoint prepare holds it instead of waiting for it, and after resume its ledger row is published before the terminal event leaves. The keep-alive cut, publication-before-answer and draining tests are unchanged. test_checkpoint_gate_keeps_framework_capture_pending_until_delivery now lets the buffered stream's response.created precede publication, as a keep-alive comment may (f672e78).

2. One response id across a keepalive stream (BLOCKER). The keepalive path now reserves the id before it announces it, and _invoke_responses gives the response that id before capture, so the lifecycle events, the terminal event, model-call capture and token capture all carry one id (9bcefb3). With stream_keepalive_s set, every streamed response takes its reserved id, fast ones included, so the choice cannot race the first interval. Provider-issued ids stay unchanged: a server opts in through _mints_response_ids(). vllm_model's Chat Completions conversion, azure_openai_model and inference_provider do; a server that serves an upstream Responses API's ids (vllm_model with is_responses_native, openai_model, switchyard_model) refuses stream_keepalive_s at startup. Tests: test_every_event_names_the_response_the_stream_ends_with[completed|incomplete]; test_capture_records_the_id_the_stream_announced (the id passed to token capture equals the wire id); test_refused_on_a_server_that_serves_upstream_ids; vllm_model's slow Codex turn now asserts one id across the stream; and test_keepalive_streams_are_refused_on_the_native_responses_path. The late-failure same-id and fast-call sequence tests are unchanged.

3. GenRM and inline_tool_output_images (RISK). GenRMModelMixin.get_converter() now forwards it (66d4c71). test_get_converter_forwards_inline_tool_output_images[False|True] converts a tool output with an image and checks its image_url part, with and without drop_unrepresentable_request_fields. The function_bridge and custom-role tests are unchanged.

Tests: vllm_model 896, vllm_model_with_compaction 9, genrm_model 12, azure_openai_model 7, inference_provider 30; PR 6's unit files 579; checkpoint unit tests 786. pre-commit and detect-secrets clean.

[yaoyu-33]

Thanks for this, @gwarmstrong. The new conversion options are tightly scoped, and the startup refusals make misconfiguration loud.

Earlier review. We re-checked the three findings from the review at ff92c43c5 against 66d4c71fa, and all three are resolved:

• A lone response.created / response.in_progress chunk is now liveness at the checkpoint gate (_checkpoint/model.py#L934-L937).
• The keepalive id is reserved before it is announced and applied before capture (#L621-L624, #L940-L953), and servers that serve upstream ids refuse the setting.
• GenRM forwards inline_tool_output_images (genrm_model/app.py#L46).

The named regression tests pass at head and fail against ff92c43c5's sources. We also streamed slow calls through vllm_model with external staging, with the checkpoint gate both on and off. The wire events, the token entry and the ledger row all carried one id.

This pass: no blockers. Five notes are inline:

• The keepalive task can outlive a request that is cancelled before the stream body starts. The chat keepalive path has the same pre-existing shape.
• With inlining on, a tool-output image without a usable URL now fails the request.
• Two option combinations could be refused at config time.
• Two test gaps: the keepalive cleanup paths, and the ledger id on a keepalive stream.

Two asks for the PR body:

• A short Codex + vllm_model smoke on this head. For example, the 2-task smoke from the body, with all four options on and one turn longer than 300 s, confirming that the streamed ids match the captured ids. The real rollouts in the body ran before the switch to #4880's bridge and before the keepalive id and lifecycle-gate changes, and AGENTS.md asks for rollout evidence on behavior changes.
• Which harness needs the server-side keepalive. Gym's own codex_agent already sets Codex's stream_idle_timeout_ms from its timeout (codex_agent/app.py#L1226). Keepalive is the largest part of the change, so a line on the harness it serves would help, for example an external harness that runs Codex with its own config, as in the Terminal-Bench-Science comparison.

What we ran (macOS, no GPU):

Suite	Passed
The PR's core unit test files	620
vllm_model	896
vllm_model_with_compaction	9
genrm_model	12
azure_openai_model	7
inference_provider	30

genrm_model ran with vllm stubbed. Not verified: a real vLLM engine on GPU, and a real Codex binary.

[yaoyu-33]

nemo_gym/base_responses_api_model.py:624-638

pending is cancelled only in _responses_keepalive_events's finally, and that block never runs if the generator never starts. Production servers install ClientDisconnectCancellationMiddleware (server_utils.py#L2007). If a disconnect lands after this return but before Starlette's first __anext__, _invoke_responses keeps generating with nothing waiting on it. We reproduced this with the middleware; without it, the task is cleaned up.

The window is small, and the chat keepalive path (#L845) already has the same shape, so this isn't blocking. Could the response own the task instead? For example, a StreamingResponse subclass whose __call__ wraps super().__call__() in a try/finally that cancels and gathers pending.

[yaoyu-33]

nemo_gym/responses_converter.py:622-630

With inline_tool_output_images on, a tool-output image that has only a file_id, or an empty image_url, now raises in _input_image_to_chat. That happens even under drop_unrepresentable_request_fields, which on its own turns the image into the text placeholder. Because the transcript is re-converted on every turn, the rest of that rollout fails too.

Codex's view_image always sends a data URL, so only other clients hit this. Could this fall back to _IMAGE_OUTPUT_PLACEHOLDER when dropping is on and the part has no usable URL?

[yaoyu-33]

responses_api_models/vllm_model/app.py:533-548

Two more combinations could be refused here at config time, like the others:

• stream_keepalive_s with is_responses_native is refused only in setup_webserver (base_responses_api_model.py#L403). LocalVLLMModel and GenRM reach that only after start_vllm_server() (local_vllm_model/app.py#L92-L96), so the GPU engine starts before the config is refused. Adding "stream_keepalive_s" to the is_responses_native tuple above would refuse it first.
• inline_tool_output_images with use_completions_api isn't refused at startup. Each inlined image then fails its request in the text-only completions path (#L1967):

Suggested change

	)
	return self
	)
	if self.inline_tool_output_images and self.use_completions_api:
	raise ValueError(
	"inline_tool_output_images requires use_completions_api=false: /v1/completions takes text-only "
	"messages, so every inlined image would be refused"
	)
	return self

[yaoyu-33]

tests/unit_tests/test_responses_api_model_streaming.py:1264

Three of the Responses keepalive cleanup paths have no test yet:

• a request cancelled during the first interval (base_responses_api_model.py#L629-L632);
• a stream closed mid-keepalive (#L690-L693);
• an HTTPException raised after keepalives (#L716).

The chat suite has tests to port for the first two: test_closing_stream_cancels_pending_backend and test_cancel_before_headers_cancels_backend (test_chat_completions_streaming.py#L342, #L428). Running one of them through the real middleware stack (setup_cancellation_middleware) would also cover the disconnect case in the comment on base_responses_api_model.py:635.

[yaoyu-33]

responses_api_models/vllm_model/tests/test_framework_context.py:768

No test runs a keepalive stream through external capture, where vllm_model normally keeps the chatcmpl-* envelope id. So nothing checks that the ledger row's response_id equals the id the stream announced. That id is the join key for terminal attribution (external_capture.py#L291-L296), and the earlier review's acceptance criteria asked for this check.

Our probe shows it holds today. Could you add a case here with stream_keepalive_s=0.02 and a slow worker, asserting that manifest.records[0].response_id equals every event's id, with and without the checkpoint gate?

[yaoyu-33]

Approving at 66d4c71fa. The inline notes are non-blocking; the head smoke rollout asked for in the review summary would still be good to add to the PR body.

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/40733684068

[gwarmstrong]

Smoke on this head (66d4c71, in a build with PR 4's 04ed490 and the Terminal-Bench-Science Codex profile, on main c155b77): Codex 0.154.0 in two Terminal-Bench-Science task sandboxes on OpenSandbox, through vllm_model with all four options and custom_tool_strategy: function_bridge, against one self-hosted DeepSeek V4.1 Flash replica behind a router, 30 minutes per trial.

221 model calls, all 200 except the one in flight when the cap stopped Codex. Every request had effort max, and replaying one through a recorder showed reasoning_effort: "max" in the chat body the engine receives, with include and reasoning.summary dropped. Codex called view_image 80 times and the images went inline in later requests without a rejection. For all 220 completed calls, every streamed event carries the captured response id; 39 calls sent keepalive events. Both trials were graded.

No turn here passed 300 s (the longest was 69 s), so I checked the long-turn case on a local rig: a real Codex 0.154.0 binary at its stock 300 s idle timeout and a model turn that takes 400 s. With keepalive on, the turn completes and every event carries the captured id. With it off, it also completes: the buffered path sends no headers until the answer is ready, and Codex's idle timer only starts once a stream has begun. The keepalive matters when something between Codex and the server opens the stream early or drops idle connections; I've corrected the body's sentence about which harness needs it.