[NVBug 6847480] Define judge_model so terminus_judge README launch works #4496

Queue rank 9/48 · Difficulty 2/5 · Status on main: not_fixed

NVBug: https://nvbugspro.nvidia.com/bug/6847480 · NVBug priority: Unprioritized (Severity 3-Functionality) · ARB: Khushi Bhardwaj
GitHub: none found (no PR/issue addresses the undefined judge_model reference; current code state is authoritative for not_fixed)

Summary

The terminus_judge README launch command fails config validation before any model call. The resources server config references a model server named judge_model that no launch selector ever defines, so the documented command aborts with a dangling-reference error. This is a config/docs defect, not a credential failure.

Reproduction

From a Gym checkout, run the exact README command:

gym env start --resources-server terminus_judge --model-type openai_model \
  +terminus_judge_resources_server.resources_servers.terminus_judge.judge_responses_create_params.max_output_tokens=512

CLI exits 1 with: field 'judge_model_server' references responses_api_models/'judge_model', which is not defined in the merged config. Available responses_api_models: 'policy_model'.

Root cause

resources_servers/terminus_judge/configs/terminus_judge.yaml:5-7 sets judge_model_server.name: judge_model, but the config defines no judge_model model server. --model-type openai_model selects responses_api_models/openai_model/configs/openai_model.yaml, which defines only policy_model. The reference is unresolved, so validation rejects the merged config. The sibling terminus_judge_simple.yaml:7 points its judge at policy_model and launches fine.

Status on main (not_fixed)

HEAD 75bb53e. terminus_judge.yaml:7 still name: judge_model; no top-level judge_model: block exists, and openai_model.yaml:1 defines only policy_model. The two commits touching this dir since the bug revision (016bb6b legacy env servers, b4d3fbc Ray declaration) add no judge_model definition. The 2026-10-07 terminus commit 6368213 touches terminus_2_sandboxed_agent only.

Proposed fix

Preferred: add a derived judge model server to terminus_judge.yaml using the supported _copy directive (global_config.py:119), mirroring conversational_tool_use/.../policy_tool_generation.yaml (the _copy: policy_model usage is at lines 24/27):

judge_model:
  _copy: policy_model

Caveat to flag for the maintainer: _copy: policy_model makes the judge the same endpoint as the policy (self-judge); if an independent judge was intended, this only unblocks launch. The README's max_output_tokens=512 override still applies. Trivial alternative: change judge_model_server.name to policy_model (as the simple config does). Add a config-resolution test asserting the README command's merged config resolves judge_model_server.

Validation

Re-run QA testcase testcases/environments/terminus_judge.py (case original_readme) — it must now reach rollout generation and grade all five terminus_1/terminus_2 examples with binary rewards. Optionally gym env resolve the README command to confirm the reference resolves.

Triage

Difficulty 2/5 — localized two-line config edit, root cause pinpointed, unit-testable without GPU; needs a live judge endpoint only for the full smoke run. Needs GPU/model: no (model endpoint, not GPU). External dependency: none.

Reviewer notes

Verifier confirmed not_fixed and difficulty 2 (the mechanical edit is trivial, but the judge-topology choice needs maintainer sign-off plus a config-resolution test). Corrections: the _copy: policy_model reference in the conv config is at lines 24/27, not :26 (line 26 is the policy_tool_judge_model: key); _copy: policy_model yields a self-judge (judge == policy) — flag if an independent judge was intended. PR #1374 attribution unverified; current code confirms the defect.