Skip to content

fix: serialize Jest in nv-internal SWE - #2513

Open
tdene wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
tdene:tde/nv_internal_jest
Open

tdene wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
tdene:tde/nv_internal_jest

Conversation

@tdene

@tdene tdene commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Forces Jest invocations in nv-internal eval scripts to run in-band: each jest line in an instance's run_script.sh gains --runInBand (skipped when the row already sets --maxWorkers) and --forceExit.

This has already been done for part of the dataset manually (90 rows), but the rest of the dataset still errors out due to Jest issues, so a comprehensive fix is useful.

Jest sizes its default worker pool from os.cpus(). The eval container gets a memory cgroup but no cpuset, so a single episode on a large node spawns a core-count fleet of Node workers.
The episode runs under rootless Apptainer with a squashfuse rootfs: every node_modules read funnels through one user-space FUSE daemon per container, inside a fixed memory budget, on nodes running many concurrent episodes. Under load the worker fleet starves the FUSE daemon (D-state pileup) or exhausts the cgroup; separately, suites that leak open handles keep the jest process alive after all tests have passed.

Thus we reach the full swebench_tests_timeout, produce no output, and score 0, corrupting reward.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
  • Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
  • Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format/copyright pass).
  • All commits have DCO sign-off (git commit -s) (so the DCO check passes).

@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Aug 13, 2026
@tdene
tdene force-pushed the tde/nv_internal_jest branch from 5dbac71 to fc54306 Compare August 18, 2026 10:59
@tdene
tdene force-pushed the tde/nv_internal_jest branch from fc54306 to 4e55d53 Compare August 21, 2026 15:16
@tdene
tdene force-pushed the tde/nv_internal_jest branch from 4e55d53 to 5442a2e Compare August 25, 2026 08:56
@tdene
tdene force-pushed the tde/nv_internal_jest branch from 5442a2e to 8d040c9 Compare September 1, 2026 11:32
Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@tdene
tdene force-pushed the tde/nv_internal_jest branch from 8d040c9 to a910318 Compare October 2, 2026 02:42
@tdene

tdene commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test a910318

@svcnvidia-nemo-ci svcnvidia-nemo-ci added area:agent Agent harnesses and Responses API agent behavior bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer labels Oct 20, 2026

This branch was previously deployed

1 inactive deployment
public — a9103189 Deployed Oct 2, 2026 by copy-pr-bot[bot] via release / finalize / notify #4156
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants