[linj-glitch]

This optional adapter routes OpenHands policy calls through the legacy Switchyard beta proxy and reconstructs Gym training rollouts from exact prompt/generated token IDs and log probabilities, preserving patch/evaluation results and masking invalid traces; it is off by default and needs no separate issue beyond Switchyard #63.

It requires that PR's rl-token-capture-beta implementation at 97424fc108e267a0885ce6d7f34806fd3b9ba3bd, which did not merge into Switchyard main: current native deployments must not be assumed compatible, and startup rejects simultaneous native Gym capture or OpenCode use, as explained in the integration guide.

Head 4e0b01b5023fead76df081becb6e9b254d2bd30f additively merges main c155b77c8420283982ec30347a46b92ae6ee1af5, resolving an obsolete whitespace conflict while retaining main's ng_verifier_invocations field and the existing adapter behavior.

With Python 3.13.14 and imports asserted to use this checkout, python -m pytest -q -o addopts= --import-mode=importlib responses_api_agents/swe_agents/tests tests/unit_tests/test_switchyard_trace.py tests/unit_tests/test_base_resources_server.py passed 323 tests; full pre-commit, actual PR-diff whitespace checks, and complete diff/body/comment guardword scans passed, and the merge commit has DCO sign-off.

Earlier validation at cde8b3d included 489 SWE/Switchyard/converter/base-response tests plus 43 AnySWE tests; prior regressions verified prefilled-history reconstruction and incompatible-mode rejection, and the changed-file secret scan passed with only a documented fixed test-session ID allowlisted.

Keep this PR draft. Earlier Apptainer/OpenHands/Qwen3-1.7B trials captured 42 schema-1 records but produced no patch. A new bounded run at this head used Qwen3-8B revision b968826d9c46dd6066d109eabc6255188de91218, pinned OpenHands, the original psf__requests-1766 issue, and the beta proxy. It produced an auth.py patch that the real verifier applied; the suite returned 45 passed / 46 failed, including httpbin DNS failures, with resolved=false. This demonstrates patch application and evaluation execution, not a reliable correctness result.

The new run captured six individually valid schema-1 records, with generated-token/log-probability lengths 771, 1225, 692, 1132, 1093, and 2048. Final run() postprocessing of the saved actual response against the live capture endpoint rejected record 1 because its prompt did not extend the prior prompt plus generation. Qwen reasoning tokens were absent from the next OpenHands prompt, so the guard correctly returned mask_sample=true without emitting a token trajectory. An offline check of the official Qwen template with explicit enable_thinking=false also drops the empty reasoning prefix on the next tool round trip; disabling thinking alone does not establish compatibility. No token-prefix guard was relaxed.

A later bounded trial used the official Qwen2.5-7B-Instruct template at revision a09a35458c702b33eeacc393d103063234e8bc28, using vLLM 0.22.0 with the Hermes tool parser and without a reasoning parser or custom template, through the actual SWEBenchWrapper.run() path. The original coveragepy R2E row c1bfa7352368b63f3a9b30c02f242408d07a7ab2 and its public image were used; the agent-visible data excluded solutions and grading fields. The original native R2E verifier separately established an unpatched baseline of 6 passed / 1 failed.

That live trial recorded four individually valid captures but also failed the token-history contract. The first response generated two task-tracker calls followed by prose; both the parsed assistant message and the next request retained only the calls. Its 7,367 prompt tokens plus 1,456 generated tokens made an 8,823-token sequence; the actual next prompt had 7,704 tokens and diverged at token index 7,613. Later matching transitions cannot repair that earlier loss. After this conclusive mismatch, the trial was intentionally stopped and the real wrapper completed cleanup with mask_sample=true and the record-1 prefix error. No patch was produced or patched verifier run completed; the separate unpatched baseline is not a rollout grading result.

Keep this PR draft. The observed blocker includes loss of generated text during tool parsing/history serialization, beyond Qwen3 reasoning-template behavior. A representative model → patch → reliable verifier → valid token-trajectory run remains outstanding for a compatible deployment. No prefix guard was weakened, no custom template was added, and these tests do not establish compatibility with native Switchyard. No PR code changed for these trials. Required Nemo_CICD_Test, Lint check, and DCO checks pass at this head; the runtime compatibility gap and maintainer review still prevent readiness.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[linj-glitch]

How to activate Switchyard trace collection (YAML)

Everything is off by default — enabling it is one key on the swe_agents server config. The three OpenHands configs now carry a commented example; uncomment and point it at your proxy:

swe_agents:
  responses_api_agents:
    swe_agents:
      agent_framework: openhands
      # Route policy calls through Switchyard and rebuild the rollout with
      # exact token ids. Must be exactly http://<host>:<port> (validated at
      # startup) and reachable from this process AND the agent container.
      switchyard_base_url: http://switchyard-host:4000

Equivalently, keep the shipped configs untouched and set it per run via a Hydra override / overlay config on top of swebench_openhands.yaml.

Prerequisite — the Switchyard side must be a capture-enabled serve (PR NVIDIA-NeMo/Switchyard#63 at eb351a7+), with a route whose id matches this config's model_server.name (policy_model in the shipped configs):

# single-vllm.yaml
routes:
  policy_model:
    type: model
    target:
      model: <actual-model-served-by-vllm>
      base_url: http://<vllm-host>:<port>/v1
      api_key: dummy
      format: openai
      token_capture_engine: vllm

switchyard \
  --routing-profiles single-vllm.yaml \
  --enable-rl-logging \
  --rl-log-dir /data/switchyard-traces/<job-id> \
  -- serve

(Use the --routing-profiles ... -- serve form: serve --config is the Rust profile server and rejects --enable-rl-logging.)

What you get / what can happen:

key absent	behavior byte-identical to main — no Switchyard codepath runs
enabled, capture healthy	rollout input/output items carry exact prompt_token_ids / generation_token_ids / generation_log_probs; response metadata records the session id + record UUIDs
enabled, anything fails	sample is masked (mask_sample: true, reason in switchyard_trace_error) — patch, eval metrics, and reward are preserved

Full details in gym_switchyard_integration.md.

🤖 Generated with Claude Code

[linj-glitch]

/ok to test 05f448b

[linj-glitch]

/ok to test cde8b3d

[linj-glitch]

Correction to the older activation example above: this adapter targets Switchyard's rl-token-capture-beta implementation, merged from Switchyard #63 as 97424fc108e267a0885ce6d7f34806fd3b9ba3bd; it should not be assumed compatible with current native Switchyard deployments.

In the beta routing YAML, token_capture_engine: vllm belongs at the route level, alongside type and target, rather than inside target. The current gym-switchyard-integration.md has the corrected example and deployment restrictions. Do not enable this external adapter together with native Gym token capture.

The PR remains a draft: the previous real OpenHands/beta-proxy run captured valid token records but produced no patch, so an end-to-end model → patch → grading smoke is still missing.

The additive merge 4e0b01b5023fead76df081becb6e9b254d2bd30f resolves the main-branch conflict while preserving the new verifier-invocation metadata. The final source-bound SWE/Switchyard/base-response suite passed 323 tests, and full pre-commit checks passed. Draft status and the rollout limitation are unchanged.

[linj-glitch]

/ok to test 4e0b01b

[linj-glitch]

Updated evidence at 4e0b01b5023fead76df081becb6e9b254d2bd30f: the earlier Qwen3-8B Requests run produced an applied patch but had DNS-limited grading and a reasoning-token prefix failure. A subsequent official Qwen2.5-7B trial used the actual SWEBenchWrapper.run() path with the original coveragepy R2E task, vLLM 0.22.0 and the Hermes tool parser, no custom template, and no reasoning parser. Agent-visible data excluded solutions and grading fields; the original verifier separately reproduced the unpatched 6-passed / 1-failed baseline.

Four real capture records have aligned valid token/log-probability arrays, but the first response's prose after two tool calls disappeared from the parsed assistant/history. The first 7,367 prompt + 1,456 generated tokens total 8,823; the next actual prompt has 7,704 tokens and diverges at index 7,613. The adapter correctly rejected that history. We stopped after the conclusive mismatch; the real wrapper returned mask_sample=true and the prefix error after cleanup. No model patch or patched-verifier result was obtained in this trial.

The PR remains draft. This identifies a tool-parser/history serialization limitation beyond reasoning mode; neither the guard nor the template was weakened. The body preserves the failed attempts and distinguishes baseline tests from fresh rollout outcomes.

Required Nemo_CICD_Test, Lint check, and DCO checks now pass at this head. Draft status, the runtime compatibility gap, and maintainer review remain; the head is unchanged.