Skip to content

Fix ParallelExecutor destructor deadlock - #173

Merged
copybara-service[bot] merged 1 commit into
google:masterfrom
rootkiller6788:fix-parallelexecutor-deadlock
Sep 10, 2026
Merged

copybara-service[bot] merged 1 commit into
google:masterfrom
rootkiller6788:fix-parallelexecutor-deadlock

Conversation

@rootkiller6788

Copy link
Copy Markdown
Contributor

The ParallelExecutor destructor never returns: it stores next_task = 1
while num_tasks = 1, so the workers' wait predicate
next_task.load() < num_tasks (i.e. 1 < 1) stays false and the worker
threads are never woken to observe the terminate flag. Every construction
of ParallelExecutor (e.g. cbrunsli/dbrunsli built with
BRUNSLI_EXPERIMENTAL_GROUPS) therefore hangs forever at destruction.

Fix: store 0 instead, so 0 < num_tasks is true, workers wake up,
increment busy_count, observe terminate, and exit, letting the
destructor's finish_latch.wait complete.

Verified with a standalone reproduction of the executor (4 threads, one
execute round): before the change it timed out at destruction; after the
change it exits cleanly.

The destructor set next_task to 1 and num_tasks to 1, so the worker wait
predicate (next_task < num_tasks) stayed false and workers never woke to
observe the terminate flag. This deadlocked destruction of every
ParallelExecutor. Store 0 instead so workers wake, observe terminate,
and exit.
@eustas

eustas commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Hi, why it is draft? Do you plan to refine that?

@rootkiller6788
rootkiller6788 marked this pull request as ready for review September 1, 2026 15:51
@rootkiller6788

Copy link
Copy Markdown
Contributor Author

Hi eustas, thanks for checking in - sorry, it sat as a draft for longer than it should have. I opened it as a draft while I reproduced the deadlock locally and double-checked the change against the worker loop; that's done now, and the PR is marked ready for review.

A quick recap of the change in case it helps: in ParallelExecutor::~ParallelExecutor() the destructor set terminate = true and then stored next_task = 1 while leaving num_tasks = 1. Workers wait on next_task.load() < num_tasks, and 1 < 1 is never true, so they never woke up to observe terminate, and the destructor's finish_latch.wait(...) blocked forever. Storing next_task = 0 makes the predicate 0 < 1 true, so each worker wakes, increments busy_count, sees terminate, notifies finish_latch, and exits cleanly.

I verified it with a standalone 4-thread reproduction: destruction hung before the change and completes normally after. All the CI checks for the PR are green. Could you please take another look when you have a chance?

@copybara-service
copybara-service Bot merged commit c18b962 into google:master Sep 10, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants