You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
docs: unclear how to obtain SWE-bench dataset #1417
A user trying to run SWE-bench Verified rollouts was blocked on obtaining the dataset. The YAML config (swebench_openhands.yaml) references gitlab_identifier with dataset_name: swebench_verified_for_sweagent_and_openhands, but responses_api_agents/swe_agents/README.md doesn't explain how to download it. The user tried ng_download_dataset_from_hf and got a 401.
It's unclear from the docs:
Where the dataset is hosted and how to obtain it
Which download command to use (ng_download_dataset_from_hf, ng_download_dataset_from_gitlab, ng_prepare_data, etc.)
What credentials (if any) are needed
Additional issues in responses_api_agents/swe_agents/README.md:
Example commands use internal Lustre paths (e.g. /lustre/fsw/portfolios/llmservice/users/...) that are not useful to external users
SIF image creation is documented only as a link to an external script in the NeMo-Skills repo (nemo_skills/dataset/swe-bench/dump_images.py). The SIF creation workflow should be self-contained within NeMo Gym's own docs — users should not need to go to a separate repo to figure out how to set up container images.
changed the title [-]docs: unclear how to obtain SWE-bench dataset for swe_agents[/-][+]docs: unclear how to obtain SWE-bench dataset[/+]on May 26, 2026
Activity
[-]docs: unclear how to obtain SWE-bench dataset for swe_agents[/-][+]docs: unclear how to obtain SWE-bench dataset[/+]