Poetry install is stuck on specific dependency with "pending..."

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: 10.15.7
Poetry version: 1.1.4, 1.1.3, 1.0.9

Issue

Hello poetry crew. Loving the work that yall do!

I am having issues with poetry while running poetry install in one of my projects on one specific machine.

It appears to get stuck on one dependency during the install and it will hang forever. When I kill the process, I see this in my shell (hinting at a race condtion/locking problem):

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/Users/{}/.pyenv/versions/3.6.8/lib/python3.6/concurrent/futures/thread.py", line 40, in _python_exit
    t.join()
  File "/Users/{}/.pyenv/versions/3.6.8/lib/python3.6/threading.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "/Users/{}/.pyenv/versions/3.6.8/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

I am on the latest poetry (1.1.4) and have tried 1.1.3 as well with the same results (all of which while running python 3.6.8 (I did use 3.6.5 as well)).

After downgrade to 1.0.9, I no longer faced this issue. I am assuming that there was an update to the way poetry can download multiple deps at the same time. Anyone else facing locking issues? At the very least, poetry should recognize a pending download with no progress being made to exit and fail early, but it simply hung and would never exit (left it going for at least 30 mins).

Maybe it would be helpful to see the pyproject.toml. Is that on macos? Were you able to identify if it hangs always on the same dependency or not?

Unfortunately, I cannot share the pyproject.toml as it contains a lot of proprietary names.

Is that on macos?

As noted above, mac OS 10.15.7 (catalina)

Were you able to identify if it hangs always on the same dependency or not?

It does indeed hang on the same dependency every time.

One other interesting thing to note:
I can manually go to the venv created by poetry and go to the pip that lives in its bin directory. When I pip install the problematic dependency (that hangs in the poetry install execution), it does install correctly. So it appears to focus the problem even further on the locking mechanism being used.

Same issue here. I just upgrade from v1.0.0, but got pending on some packages during installing , typically numpy , sklearn... The error message shows it got a read time out from my nexus repo, but it never happened before. And after I try the poetry install several times, it installed all package successfully. Does the parallel installation trigger this issue?

I do not know how to help you all here...

My advice would be to try one of those (maybe in that order) if you have not yet:

delete the poetry.lock file and retry
delete the virtual environment and retry
deactivate the new (parallel) installer with poetry config experimental.new-installer false and retry (do not forget to reactivate it afterwards if it proves to have no influence)
or any combination of those

Also maybe provide some verbose output (-vvv), that might be helpful for when one of the maintainers gets the time to handle this ticket. Of course, the best as always is if you can provide a minimal reproducible example that triggers the failure.

@sinoroc

I do experience the same problem with poetry 1.1.4 within a python 3.9 docker container. The error goes away by following the suggestion of @sinoroc to deactivate the experimental installer.

pyproject.toml:

[tool.poetry]
name = "foobar"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "^3.8"
watchdog = "^1.0.1"

[tool.poetry.dev-dependencies]
docker = "^4.4.0"
pytest = "^6.2.1"

poetry.lock.txt

I can confirm that disabling the experimental parallel installer worked. My environment, although, was Docker, particularly the python:3.8-slim image.

Same problem here, and disabling the experimental installer fixed it for me, too. My environment is this docker image: mcr.microsoft.com/vscode/devcontainers/python:0-3.8 with the following commands:

apt-get update && export DEBIAN_FRONTEND=noninteractive \
    && apt-get -y install --no-install-recommends python3-venv gnupg2
pip install tox
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
source $HOME/.poetry/env
poetry --version
poetry install -v

Could someone on the Poetry team provide a perspective on prioritization for this? It seems like there's meaningful evidence that this is an issue independent of environment or dependencies. Turning off the installer is a good workaround in the interim, but it significantly increases install times. I'd love to know I have some hope of being able to turn it back on at some point.

Disabling experimental installer also fixes this for me, but also able to fix it using the experimental installer by adding a timeout for the requests to pypi, E.g.

diff --git a/poetry/repositories/pypi_repository.py b/poetry/repositories/pypi_repository.py
index 1c7b7be3..a1f0922a 100644
--- a/poetry/repositories/pypi_repository.py
+++ b/poetry/repositories/pypi_repository.py
@@ -321,14 +321,14 @@ class PyPiRepository(RemoteRepository):
 
         return data.asdict()
 
-    def _get(self, endpoint: str) -> Union[dict, None]:
+    def _get(self, endpoint: str, timeout: int = 5) -> Union[dict, None]:
         try:
-            json_response = self.session.get(self._base_url + endpoint)
+            json_response = self.session.get(self._base_url + endpoint, timeout=timeout)
         except requests.exceptions.TooManyRedirects:
             # Cache control redirect loop.
             # We try to remove the cache and try again
             self._cache_control_cache.delete(self._base_url + endpoint)
-            json_response = self.session.get(self._base_url + endpoint)
+            json_response = self.session.get(self._base_url + endpoint, timeout=timeout)
 
         if json_response.status_code == 404:
             return None

Happens on WSL2 ubuntu 20.04 with python3.10

Recent issues possibly related to discussion happening in #2094

On Ubuntu 20.04 WSL2 here. Only disabling IPv6 did the trick:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1

#3352 (comment) suggests a very plausible fix, perhaps someone would like to submit an MR.

FWIW I deleted my poetry cache and artifacts (~/.cache/pypoetry/cache/), and found "things" to work again. If there is any wisdom as to when/why corruption in these locations leads to such an inability to install venvs, that would be educational.

@Purg

@Purg thank you sire! i've been stuck on this for hours... i tried all these:

turn off the experimental.new-installer
```
poetry config experimental.new-installer false
```
but this only worked "for a moment" then stopped working:
delete virtualenv - didn't work
delete poetry.lock - didn't work
clear all cache by running (see Poetry is extremely slow when resolving the dependencies #2094 (comment))
```
$ poetry cache list
$ poetry cache clear --all pypi 
```
didn't work

only yours worked:

cd ~/.cache/pypoetry/cache/
rm -rf artifacts/ cache/

The reason for corrupted caches is either:

cancelled downloads leaving a orphan lock file (fixed by Use filelock to lock the file cache #6471)
multiple concurrent instances of Poetry writing to the same file (fixed by Fix a race condition in simultaneous poetry installations updating the cache #6186)

Both of these will not be backported to 1.2 as they are new changes to Poetry behavior/not fixes for regressions, but they will be available in the near future, and you can test them today by installing from master. Clearing the cache is a standard troubleshooting step, and always worth doing (indeed --no-cache is a handy global flag that lets you ignore the cache without actually clearing it).

my last question: what's the difference between

$ poetry cache list
$ poetry cache clear --all pypi

and

cd ~/.cache/pypoetry/cache/
rm -rf artifacts/ cache/

?

poetry cache clear is like deleting the cache folder, but does not affect the artifacts folder.

artifacts are just whl packages, maybe they are corrupted too? 🤔
i'll remember to clear both artifacts and cache in the future

Artifacts are distfiles (wheels, sdists) and can be corrupted by an interrupted Poetry (ctrl-c, machine shutdown) as well as a concurrent Poetry. This will be fixed in the 1.3 release.

I had problems with this due to "Connection pool is full, discarding connection: pypi.org. Connection pool size: 10". It helped to set poetry config installer.max-workers 10. Should we consider that as an upper bound to the default? I think I might have something like 8 cores, which takes the worker number to 12.

The artifacts and cache dirs were in ~/Library/Caches/pypoetry on my M1 Mac for those who can't find it in ~/.cache/pypoetry/cache/

cd ~/Library/Caches/pypoetry
rm -rf artifacts/ cache/

Doing that cleared things up for me. Before that, it was consistently hanging on trying to fetch flake8-bugbear from pypy

You can always find the cache directory with poetry config cache-dir. If you're having issues you think might be related, try clearing the cache first. If you're still having issues, I'd suggest opening a Discussion or joining Discord first as 90% of the time these are local config issues.

I would ask that if you don't have anything to add that hasn't already been said in this issue, you hold off, as generating notifications for 20+ people for a root cause that is solved is unnecessary (and if you have something novel, we want to identify that and make sure it's tracked as a new issue).

poetry config experimental.new-installer false

This solution helped me. Thank you.

I'm facing this issue in the latest Poetry version.
This helps:

poetry config experimental.new-installer false

I do not know how to help you all here...

My advice would be to try one of those (maybe in that order) if you have not yet:
* delete the `poetry.lock` file and retry

* delete the virtual environment and retry

* deactivate the new (parallel) installer with `poetry config experimental.new-installer false` and retry (do not forget to reactivate it afterwards if it proves to have no influence)

* or any combination of those
Also maybe provide some verbose output (-vvv), that might be helpful for when one of the maintainers gets the time to handle this ticket. Of course, the best as always is if you can provide a minimal reproducible example that triggers the failure.

In my case deactivating the new (parallel) installer solved the problem. It was stuck on installing pycparser 2.21 and six 1.16.0 (maybe it will be useful)

If deactivating the new installer helps, you have a corrupted artifact cache. Please rm -rf $(poetry config cache-dir)/artifacts and try again. I am locking this issue as it keeps misleading people -- most of the issues here are solved, including the artifact corruption issue. Disabling the new installer is not sustainable (as the old installer is scheduled for removal), so please follow the troubleshooting steps outlined.

If, after doing the following, you still reproduce on the latest version of Poetry, please open a new issue so we can fix bugs in the new installer:

poetry env remove --all
poetry cache clear --all .
rm -rf $(poetry config cache-dir)/artifacts

Navigation Menu

Repository navigation

Poetry install is stuck on specific dependency with "pending..." #3352

Issue

10 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Poetry install is stuck on specific dependency with "pending..." #3352

Description

Issue

Activity

sinoroc commented on Nov 11, 2020

cpvandehey commented on Nov 11, 2020

DeepDarkOdyssey commented on Nov 24, 2020

sinoroc commented on Nov 24, 2020

arnegroskurth commented on Dec 21, 2020

vaidik commented on May 17, 2021

ohthehugemanatee commented on Jun 2, 2021

jlucas91 commented on Aug 5, 2021

dbowring commented on Sep 9, 2021

GerevSec commented on Mar 12, 2022

schlich commented on Mar 17, 2022

epomatti commented on Jun 19, 2022

dimbleby commented on Jun 19, 2022

10 remaining items

Purg commented on Sep 21, 2022

zachliu commented on Sep 22, 2022

neersighted commented on Sep 22, 2022

zachliu commented on Sep 22, 2022

neersighted commented on Sep 22, 2022

zachliu commented on Sep 22, 2022

neersighted commented on Sep 22, 2022

TallGirlVanessa commented on Oct 4, 2022

rberger commented on Oct 5, 2022

neersighted commented on Oct 5, 2022

bhuiyanmobasshir94 commented on Nov 21, 2022

Lamarcke commented on Dec 31, 2022

PitterRup commented on Jan 3, 2023

neersighted commented on Jan 3, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions