Skip to content

feat(gdpval): run LibreOffice through a configurable command - #4586

Merged
athitten merged 5 commits into
mainfrom
athitten/gdpval-libreoffice-command
Oct 26, 2026
Merged

athitten merged 5 commits into
mainfrom
athitten/gdpval-libreoffice-command

Conversation

@athitten

@athitten athitten commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds a libreoffice_command setting to the GDPVal resources server: a command (for example a container prefix) that runs LibreOffice for Office-to-PDF conversion instead of a host libreoffice. It is for clusters where the judge host has no LibreOffice and cannot apt-get install it:

libreoffice_command: [apptainer, exec, --bind, /path/to/data, --bind, /tmp, /path/to/libreoffice.sif, libreoffice]
  • It is used in both places that convert files for the judges: comparison-mode preconversion and rubric-mode rendering.
  • At startup, the server runs the command with --version and converts a probe workbook (with a formula and a chart) in the temp directory and in temp directories under persist_deliverables_dir and each reference deliverables_dir. Startup fails with an error naming the command if --version or a probe conversion fails, if a reference deliverables_dir does not exist, or if the command leaves files that the server cannot delete (for example, rootful Docker without --user). The warning suggests --user only when the leftover files belong to a different user than the server's. A reference dir that the server cannot write is skipped and logged: nothing can be converted there.
  • The command gets absolute paths, so it does not depend on the container's working directory.
  • When the setting is unset, the host libreoffice, its arguments and the ensure_libreoffice() install steps are unchanged, except that the same probe conversion now also runs before skipping apt and after an install. Two small fixes apply either way: output that is not valid UTF-8 no longer fails a conversion or the startup check, and conversion failure messages show the end of LibreOffice's error output, where the real error is, instead of the start.
  • The README lists the directories the command must see and gives a Docker example. Gym gives each LibreOffice run 120 s, but with Docker that limit stops only the docker client, so the example also sets a time limit inside the container.

Ported from athitten#2, which built on yaoyu-33#8, and adapted to #4178 (now merged):

  • file_reader.py now lives in resources_servers/gdpval/;
  • the probe uses persist_deliverables_dir.

Related: #3550 renders Office files in a separate preparation step before judging. This PR lets the judge server run LibreOffice itself, through the configured command.

No issue: this is a small opt-in setting that we needed to score #4589 and #4591 on a cluster without LibreOffice on the judge host.

Validation

  • pytest resources_servers/gdpval/tests with the server's requirements.txt installed: main (49ffb81) 870 passed; this branch, rebased onto it, 893 passed (23 new tests); 1 skipped on both (a Stirrup-only check).
    • cryptography was installed with uv pip install --no-config. The root pyproject.toml excludes it, and without it 3 PDF tests skip.
  • Each new or changed test fails when the code it guards is broken (one-at-a-time mutations).
  • Real container check: Docker with an image that has LibreOffice 25.2 (the host has 24.x), and a mocked judge.
    • Startup fails for a command without the persist bind, and for one without --user.
    • With all binds, comparison /verify sent the judge 6 PDFs rendered by the container (docx, xlsx with a chart, pptx; eval and reference). Rubric /verify built 3 PDF blocks with no text fallback.
    • The README's Docker example passes the startup check and converts files, and no containers or files are left behind.
  • An earlier revision of this branch scored the Pi and Codex GDPVal runs on a cluster without host LibreOffice (feat(gdpval): run Pi in place of Stirrup #4589 and feat(gdpval): run Codex in place of Stirrup #4591).
  • pre-commit run --all-files: passed.

Rollouts

N/A for generation: the setting only changes how the judge's inputs are converted. The scoring runs above use it.

Compatibility and benchmark impact

  • When unset, output that is not valid UTF-8 no longer fails a conversion or the startup check. A conversion that failed this way now succeeds, which can change the score of an affected task.
  • When set, startup fails if the command cannot convert in the temp directory, under persist_deliverables_dir or under a writable reference dir, if a reference dir is missing, or if it leaves files that the server cannot delete.
  • When unset, a host libreoffice whose probe conversion fails no longer skips the apt install, and fails the startup check in comparison mode.
  • The judge sees PDFs rendered by the command's LibreOffice, whose version and fonts can differ from a host install, so scores can differ. PDFs already next to the files are reused, so a submission and its reference can be rendered by different builds.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Documentation added or updated, or N/A with justification.
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format checks pass).
  • New source files include the required Apache-2.0 SPDX header.
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

@athitten athitten added feature New capabilities, enhancements, or enablement work area:environment Individual environments, benchmarks, verifiers, and environment-specific resources servers labels Oct 14, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@marta-sd
marta-sd force-pushed the martas/gdpval-decoupling branch from fd1cb9e to 53c4a64 Compare October 14, 2026 08:34
@athitten
athitten force-pushed the athitten/gdpval-libreoffice-command branch from aece9de to 21cd046 Compare October 15, 2026 01:44
Base automatically changed from martas/gdpval-decoupling to main October 19, 2026 17:22
@athitten
athitten force-pushed the athitten/gdpval-libreoffice-command branch 2 times, most recently from 430af36 to 54b51a1 Compare October 20, 2026 10:25
@athitten

Copy link
Copy Markdown
Contributor Author

/ok to test 54b51a1

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

@athitten
athitten marked this pull request as ready for review October 20, 2026 10:49
@yaoyu-33 yaoyu-33 added complexity:medium Single-domain change with interacting parts or a moderate review surface needs-review PR is ready for code review and waiting on a reviewer labels Oct 20, 2026
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 21, 2026
Comment thread resources_servers/gdpval/app.py Resolved
@github-actions github-actions Bot removed the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 22, 2026
@athitten

Copy link
Copy Markdown
Contributor Author

/ok to test 24481c8

Signed-off-by: Abhishree <athittenaman@nvidia.com>
…r use

Signed-off-by: Abhishree <athittenaman@nvidia.com>
Signed-off-by: Abhishree <athittenaman@nvidia.com>
Signed-off-by: Abhishree <athittenaman@nvidia.com>
@athitten
athitten force-pushed the athitten/gdpval-libreoffice-command branch from 24481c8 to 555f0d1 Compare October 26, 2026 04:17
@athitten

Copy link
Copy Markdown
Contributor Author

/ok to test 555f0d1

Signed-off-by: Abhishree <athittenaman@nvidia.com>
@athitten

Copy link
Copy Markdown
Contributor Author

/ok to test 8a436ec

@athitten
athitten added this pull request to the merge queue Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/40354884270

Merged via the queue into main with commit 97ab8a8 Oct 26, 2026
46 of 49 checks passed
@athitten
athitten deleted the athitten/gdpval-libreoffice-command branch October 26, 2026 16:27

This branch was previously deployed

1 inactive deployment
public — 8a436ec3 Deployed Oct 26, 2026 by copy-pr-bot[bot] via release / finalize / notify #7252
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:environment Individual environments, benchmarks, verifiers, and environment-specific resources servers complexity:medium Single-domain change with interacting parts or a moderate review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants