[NVBug 6790320] Fix Inverse IF quick-start: judge_model reference has no server definition #4503

Queue rank 15/48 · Difficulty 2/5 · Status on main: not_fixed

NVBug: https://nvbugspro.nvidia.com/bug/6790320 · priority: Unprioritized · ARB: Ananth Subramaniam
GitHub: none found (no fix PR; related sibling convention in physics_judge, math_with_autograder)

Summary

The shipped Inverse IF config references a judge_model responses_api_models server that is only present as a commented-out block, so no server defines it. The README quick-start selects inverse_if + vllm_model, and vllm_model defines only policy_model. Config validation raises ServerRefNotFoundError and startup exits 1 before any task generation or judging. Not a model-quality or scoring defect.

Reproduction

From a scratch dir with env.yaml = {}, using the Gym venv (no GPU/inference):
gym env validate --strict --resources-server inverse_if --model-type vllm_model ++policy_base_url=http://127.0.0.1:9/v1 ++policy_model_name=Qwen/Qwen3-8B '++policy_api_key='
Exit 1: field 'judge_model_server' references responses_api_models/'judge_model', which is not defined... Available: 'policy_model'. Adding ++inverse_if.resources_servers.inverse_if.judge_model_server.name=policy_model -> exit 0.

Root cause

inverse_if.yaml:24-26 sets judge_model_server.name: judge_model, but the only judge_model definition (inverse_if.yaml:128-136) is commented out. vllm_model.yaml:1 contributes only policy_model. The validator in global_config.py correctly rejects the unresolved ref (raise at :658-660, within 643-661). The quick-start composition is incomplete.

Status on main (not_fixed)

HEAD 75bb53e (2026-10-12). inverse_if.yaml:24-26 still name: judge_model; :128-136 still commented. README start commands (:12-14, :121-123) unchanged; vllm_model.yaml unchanged. git log on these paths: only #4300 (Ray decoupling), which did not touch the judge ref; last inverse_if config commit 016bb6b (#3724). Present in every release since v0.3.0 — not a regression.

Proposed fix

Maintainer design choice (one of):

1. Point judge_model_server.name -> policy_model (matches README Basic Setup :155-157); minimal, policy-as-judge. This fixes BOTH README start commands with no README edit.
2. Add configs/judge_<flavor>.yaml defining judge_model + update both README commands to add --resources-server inverse_if/judge_<flavor> (sibling convention: physics_judge, math_with_autograder); only this option also requires editing the second README command (:121-123).
   Add a CI check running gym env validate --strict over each README start command (also catches terminus_judge, NVBug 6847480).

Validation

testcases/environments/inverse_if.py (e2e). After fix, update the pinned hashes (:57) and the judge_model_server assertion (:254) to the fixed reference; re-run only after a green run. Config-only fix verifiable via gym env validate --strict without GPU.

Triage

Difficulty 2/5 — root cause pinpointed; localized config/README edit, but needs a small maintainer design decision and QA pin update. Needs GPU: no.

Reviewer notes

Verifier confirmed all paths (QA testcase lives in the test repo) and status not_fixed on main; difficulty held at 2 (borderline 1). Corrections applied: second README command is at :121-123 (not :120-122); the "also fix the second README command" step applies only to Option 2 — Option 1's one-line config edit fixes both commands with no README change; validator raise tightened to global_config.py:658-660.