Repository navigation
[Bug] pytest-xdist workers race on captured_info.txt in patched testing utils #45561
Description
System Info
Transformers=5.6.0.dev0 (local checkout 85099df);
Python=3.13.5;
Platform=macOS-26.2-arm64-arm-64bit-Mach-O;
pytest=8.4.2;
pytest-xdist=3.8.0
Who can help?
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
This issue affects internal test utilities and does not involve example scripts or a user dataset/task.
- In src/transformers/testing_utils.py, _prepare_debugging_info() appends to captured_info.txt and patch_testing_methods_to_collect_info() unlinks that same file.
- Run tests under pytest -n 2 with workers sharing the same _PATCHED_TESTING_METHODS_OUTPUT_DIR.
- Observe that workers write to and reset the same captured_info.txt, so one worker can delete or interleave another worker's captured debugging output.
I checked for overlapping issues and PRs and did not find a direct match.
I have a candidate fix with focused tests on my fork here: https://github.com/oleksii-tumanov/transformers/tree/fix/testing-utils-xdist-captured-info (commit 58dcab6). I am opening the issue first for coordination before submitting an upstream PR.
Expected behavior
Each xdist worker should isolate its captured debugging output instead of sharing one captured_info.txt file. For example, worker-specific filenames such as captured_info_gw0.txt and captured_info_gw1.txt would avoid cross-worker clobbering while preserving the current single-file behavior for non-xdist runs.
Activity
Rocketknight1 commentedon Apr 27, 2026
cc @ydshieh @tarekziade (see also the PRs at #45639 and #45645, but don't be afraid to banish any slop if it's not helpful 😅 )
tarekziade commentedon Apr 29, 2026
The changes seems sound, I will defer to @ydshieh for reviews
github-actions commentedon May 29, 2026
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
oleksii-tumanov commentedon May 31, 2026
I believe this is a useful small fix
github-actions commentedon Jun 24, 2026
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
oleksii-tumanov commentedon Jun 24, 2026
This is still relevant.
I opened #45639 with a focused fix for the xdist
captured_info.txtrace while preserving the existing non-xdist behavior. Happy to adjust the approach if maintainers prefer a different direction.github-actions commentedon Jul 19, 2026
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.