[athitten]

What does this PR do?

Stacked on yaoyu-33#8 (Pi in GDPval task sandboxes), which is stacked on NVIDIA-NeMo#3626. This PR fixes what
blocked a full 220-task Pi × GDPval run on HSG. It also adds a way to score sandbox-session runs with the GDPval-AA v2
multistage ELO: gym eval run --no-serve cannot drive multistage.

Fixes, grouped by problem:

• Seeding: a seed retry reused a stopped sandbox, and reference lists stored as JSON strings broke staging
  (29edfc233).
• Large references: two tasks have reference files over the 128 MiB limit (up to 689 MB), so they could never
  start. There is now a 1 GiB per-file cap with bounded download retries. A missing file fails the task with a terminal
  424 and an oversized one with a terminal 413, instead of an endlessly retried 500. 408, 425, 429, 5xx and malformed
  responses are retried (ef48c21ee, 6fe5f612a, 65274f632, 802476d34).
• Export layout: a subdirectory, link, unusual name, or more than 100 files in /workspace/output failed the
  episode as an infrastructure error, which dropped the task from the score. The export now copies the top-level
  regular files (all the judge reads). Everything else is recorded in finish_params.json under skipped: the first
  100 entries plus a count. A missing or symlinked output directory is recorded too, and the in-sandbox listing is
  capped at 10,000 entries. The export cap is now 1 GiB, because two gold deliverables exceed 128 MiB (abdec3d38,
  925b786fe, e18f1f5bc, 12f0d65b1, 75aaa5eeb).
• Completion marker: the export writes the finish_params.json marker, so comparison mode judges the export
  instead of reporting it missing (6020761bf).
• Judge failures no longer lose the export:
  • An invalid verdict used to raise a 503. That re-ran the whole episode with the same seeded judge, and the export
    never reached the ELO.
  • A raised judge error, such as a 413 or context-window error, did the same. That happened once in the run below
    (Claude on Bedrock, "Input is too long").
  • Both now return a masked verdict (mask_sample, failure_kind, failure_reason) that keeps deliverables_dir.
    The rubric mean and gym eval profile exclude it, and judge-only scoring re-judges the export (f54d81898,
    7290f2abb, 22f6dede0).
• LibreOffice: clusters without a host LibreOffice can set libreoffice_command, for example to run it through
  the GDP image with apptainer exec. The startup probe checks that the command can see deliverables_root as well as
  TMPDIR (24e59f9c1, 93e6cbb6a).

Adds:

• resources_servers/gdpval/cache_sandbox_deliverables.py copies sandbox exports into the task_<id>/repeat_<n>
  layout that judge-only scoring reads. Multistage ELO then runs as a second, judge-only pass. The cache build stages
  copies outside the task directories, and it fails, naming the tasks, when an export has no marker (eee1fd325,
  4183b4b2a, 9fccc683c, 57ff3ca01).
• Docs: the export rules, judge-only scoring of sandbox runs, and libreoffice_command (42c0f2ddd, 8c0c43579).
  They also give the config-order rule for overriding the judge panel: pass --config benchmarks/gdpval/config.yaml
  before your overlay instead of using --benchmark gdpval. The CLI appends --benchmark after every --config, so
  the benchmark's judge settings silently replace yours. The headers of the Kimi and MiniMax judge overlays showed the
  wrong order and are fixed.

Related:

• MB-1712 (Pi × N35 Super budgets).
• docs(agent): add harness onboarding and upgrade guide NVIDIA-NeMo/Gym#3917 (harness onboarding guide).
• feat(gdpval): run Codex on GDPval task sandboxes #1 (Codex × GDPval, which carries the same GDPval fixes).

Validation

• NEMO_GYM_EXTRA_ROOTS=$PWD resources_servers/gdpval/.venv/bin/python -m pytest resources_servers/gdpval/tests -q:
  655 passed, 3 skipped, 4 failed.
  • The 4 failures are in test_judge_panel.py and are the same at a35bc005e: the openai SDK rejects a Starlette
    TestClient as its http_client.
  • The suite also passes at each commit.
• Each new test was checked by breaking the code it guards. Examples:
  • export skip instead of 503, count and size limits, listing shape and cap, symlinked output root;
  • invalid and raised judge errors masked, checked through the real select_measured; cancellation still propagates;
  • download statuses and fail-closed guard, cache staging and the marker check, the LibreOffice probe directories and
    %PDF check.
• pre-commit run --files <changed files>: all hooks pass with no changes. ruff check and ruff format --check are
  clean.
• Two independent review rounds. Their findings are addressed in the last 16 commits; what remains is listed under
  follow-ups.

Rollouts

220-task Pi × GDPval on HSG:

• Policy: N35 Super GA served by vLLM.
• Harness: Pi 0.80.2 with MB-1712 budgets (thinking high, 32,768 output tokens per call, 3,600 s per episode),
  concurrency 16.
• Code: generation ran at f54d81898, across two chained 2-hour jobs with one automatic resume. The later commits
  are covered by unit tests.
• Scoring: judge-only, with the official AA-v2 panel (GPT-5.5 / Gemini 3.1 Pro / Opus 4.8) and the 18-reference
  ladder.

Metric	Value
Tasks	220. There are 219 rows; 1 export was recovered after a raised judge error, the case 22f6dede0 fixes.
Scoring settings	900 s comparison judge timeout. partial_completion floors: 0.9 / 0.3 / 1 for stage 1 and 0.95 / 0.5 / 1 for the final stage. Availability-only reference repair. The Codex run (#1) used the same settings.
Rubric (re-judge from the cache, official panel)	79.3 / 100, with 218 of 220 judged. Two tasks hit permanent judge size limits.
Multistage ELO (comparison/eval_elo)	1150.6. The final stage judged 216 of 220 tasks against deepseek_v4_pro, glm51_fp8, inkling_small and kimi_k26, and was accepted with partial_completion at 98.2% coverage; it is not degraded and has no imputed losses. Stage 1 (45 tasks × 18 references) gave 1194.8. The 4 omitted tasks are all transport_ineligible: their deliverables are larger than every judge's attachment limit.
Episodes that ended on a length-truncated model call	11 (10 scored 0). Pi has no truncation recovery; see follow-ups.

Compatibility and benchmark impact

• The stateless (Stirrup) GDPval path is unchanged. The default LibreOffice argv is unchanged, and
  libreoffice_command defaults to unset.
• finish_params.json from sandbox exports gains skipped and skipped_count. Its readers only check that the file
  exists.
• In sandbox sessions, a judging failure returns a masked row instead of failing /verify with a 500 or 503. This
  covers invalid verdicts, raised judge errors, and inline comparison failures. Score such runs with the judge-only
  pass from the cache.
• Benchmark impact: none for existing runs. The ELO above is comparable to the 18 anchors: same judge panel and
  multistage protocol.
• Follow-ups, not in this PR:
  • Pi has no equivalent of Stirrup's truncation_recovery. A call that spends the whole output budget on reasoning
    ends the episode.
  • The Apptainer provider copies files synchronously, so a 1 GiB file blocks the resources-server event loop while it
    copies.
  • The CLI's config order (--benchmark after --config) has no test.

Checklist

• I have read the contributing guidelines.
• The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
• Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
• Documentation added or updated, or N/A with justification.
• Pre-commit checks pass locally (on the changed files).
• New source files include the required Apache-2.0 SPDX header.
• All commits have DCO sign-off (git commit -s) (so the DCO check passes).