Skip to content

fix(harbor-agent): restore the pathspec runtime dependency - #5040

Open
jackiectl2 wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
jackiectl2:fix-harbor-pathspec
Open

jackiectl2 wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
jackiectl2:fix-harbor-pathspec

Conversation

@jackiectl2

@jackiectl2 jackiectl2 commented Oct 24, 2026 •

Copy link
Copy Markdown

What does this PR do?

The root dependency override excludes pathspec, but harbor.job imports it through its directory-scanning dependencies. The Harbor agent must restore pathspec in its own environment. The current server uses pyproject.toml and uv.lock, so this PR pins pathspec==1.1.1 there and overrides the root exclusion for that server. The import regression test remains in the Harbor agent tests.

Fixes #4517.

Validation

  • Before the original fix, an isolated Harbor environment raised ModuleNotFoundError: No module named 'pathspec' while importing harbor.job. With the original requirements/overrides fix, that import test passed; a maintainer also reported 123 server-environment tests passing on the cherry-picked change.
  • After the upstream switch to pyproject.toml, uv lock --directory responses_api_agents/harbor_agent resolved 221 packages. uv tree --locked and uv export --locked --no-dev both show active pathspec==1.1.1 for this server. The lock changed only the relevant dependency markers and entries.
  • Targeted pre-commit run --files on the Harbor project manifest, lockfile and import test passed. compileall on the import test and git diff --check on changed Harbor files passed.
  • The full server environment and model rollout were not rerun on the updated branch. New upstream CI requires maintainer approval.

Rollouts

N/A for this dependency manifest fix. No model rollout was run.

Compatibility and benchmark impact

The Harbor agent environment gains pathspec==1.1.1; other server environments retain the root exclusion. Benchmarks were not run.

Checklist

  • I have read the contributing guidelines.
  • The change is focused.
  • The import regression test is retained; it passed in the previous isolated Harbor environment.
  • Documentation N/A: this restores an existing runtime dependency without changing the public interface.
  • Targeted pre-commit checks pass on the current changed files.
  • New source files include the required Apache-2.0 SPDX header.
  • All commits have DCO sign-off (git commit -s).

Keep pathspec available to the Harbor server despite the root override
that removes it from other environments.

Fixes NVIDIA-NeMo#4517

Signed-off-by: Tianlang (Jackie) Chen <305351710+jackiectl2@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 24, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nemo-automation-bot nemo-automation-bot Bot added the community-request Issue reported or requested by someone from the community label Oct 24, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added area:agent Agent harnesses and Responses API agent behavior bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer labels Oct 24, 2026
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Oct 26, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the waiting-on-maintainers Waiting on maintainers to respond label Oct 26, 2026
@qiaochuz-nv

Copy link
Copy Markdown
Contributor

Thanks for the fix. The branch is 128 commits behind main, so I checked it as your commit cherry-picked onto current
main (5911d628c):

  • gym env test +entrypoint=responses_api_agents/harbor_agent (what the changed-server CI job runs) builds the agent
    venv with pathspec 1.1.1: 123 passed, and test_harbor_job_imports runs there instead of skipping.
  • Control: the same command on main with only tests/test_dependencies.py added fails that test with
    ModuleNotFoundError: No module named 'pathspec'.
  • The manifests match what chore: never install gprof2dot/pathspec; purge dbus and gi bindings #4193 did for terminal_bench_4 and miniswe_sandboxed_agent (pathspec==1.1.1 in
    requirements.txt, unpinned pathspec in overrides.txt).
  • End to end on an H100 (Legal Agent Bench, Qwen3-0.6B policy, real judge, limit=1, official full-task reward):
    with this change, Harbor ran a real Docker trial for about 3.5 minutes and wrote result.json and
    verifier/scores.json, with all 23 criteria judged (0 judge errors). On main the same run created no Harbor trial;
    the rollout came back as reward 0.0 with failure_kind=None, and the import error showed up in no log.

Outside this PR: HarborAgent.run still turns any exception into reward 0 with failure_kind=None, which is why this
regression scored 0 silently instead of failing.

LGTM from the QA side.

@qiaochuz-nv

Copy link
Copy Markdown
Contributor

/ok to test e9cb672

@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team October 29, 2026 22:10
@svcnvidia-nemo-ci svcnvidia-nemo-ci added waiting-on-maintainers Waiting on maintainers to respond and removed waiting-on-maintainers Waiting on maintainers to respond labels Oct 30, 2026
@mjmikulski

Copy link
Copy Markdown
Contributor

Could you please share ETA for PR?

@svcnvidia-nemo-ci svcnvidia-nemo-ci removed the waiting-on-maintainers Waiting on maintainers to respond label Nov 6, 2026
Signed-off-by: Tianlang (Jackie) Chen <305351710+jackiectl2@users.noreply.github.com>
@jackiectl2

Copy link
Copy Markdown
Author

Thanks for checking in. I have updated the branch to current main in d367b2b. The Harbor agent now restores pathspec==1.1.1 through its new pyproject.toml and uv.lock; the retired requirements and overrides files are removed. Lock resolution and targeted pre-commit checks pass. I have not rerun the full Harbor server environment on this head. An NVIDIA-NeMo member will need to approve CI with /ok to test d367b2b77d71efb9766a34bf8d300a8494a1b959; I can follow up on any resulting failures.

This branch was previously deployed

1 inactive (outdated) deployment
public — e9cb6720 Deployed Oct 29, 2026 by copy-pr-bot[bot] via release / finalize / notify #8474
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent harnesses and Responses API agent behavior bug Something isn't working community-request Issue reported or requested by someone from the community needs-review PR is ready for code review and waiting on a reviewer sla:triage-overdue Review assignment is over the one-business-day SLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[NVBug 6900605] Re-enable pathspec for harbor_agent venv so Harbor rollouts stop silently scoring 0

4 participants