Which pip version does poetry use?

I have searched the issues of this repo and believe that this is not a duplicate.
I have searched the documentation and believe that my question is not covered.

Question

I was wondering how poetry chooses which pip version it uses when installing new packages? And how can we make it choose a different one?

I am asking this question because of the following problem I encountered:

While trying to install "cryptography" in a Docker container with poetry I got the following error message:

Command ['/root/.cache/pypoetry/virtualenvs/hybrid-py3.7/bin/pip', 'install'  
, '--no-deps', 'cryptography==2.4.2'] errored with the following output:      
Collecting cryptography==2.4.2                                                
  Downloading https://files.pythonhosted.org/packages/f3/39/d3904df7c56f8654  
691c4ae1bdb270c1c9220d6da79bd3b1fbad91afd0e1/cryptography-2.4.2.tar.gz (468k  
B)                                                                            
  Could not find a version that satisfies the requirement cffi!=1.11.3,>=1.7  
 (from versions: )                                                            
No matching distribution found for cffi!=1.11.3,>=1.7                         
You are using pip version 10.0.1, however version 18.1 is available.

I found the error a little bit weird in the first place, because it does not occur when I run poetry install locally, but after a little bit of googling it seemed that the cffi problem was related to the pip version and should be fixable, by upgrading pip => which leads to my question: My dockerfile is based on python:3.7-alpine and has pip 18.1 installed, so why does poetry still try to use pip version 10.0.1?

Thanks a lot to anyone taking the time to read through this :)

Edit: To fix my issue, I just had to do poetry run pip install --upgrade pip before poetry install, but I guess my question where poetry gets the idea to use 10.0.1 from still holds? :)

This is most likely related: #712

I just ran into this as well and think it is a separate issue.

venv creates virtual environments using ensurepip which contain pip==10.0.1 and setuptools==39.0.1. This explains where pip==10.0.1 is comming from on Python >=3.3.

The next issue is that packages that require certain versions of pip or setuptools usually do not declare these as dependencies. Hence, IMHO cryptography should have a requirement pip>=X.Y.Z somewhere, maybe in setup.py/setup_requires or pyproject.toml/[build-system]/requires?

However, even if an explicit dependency on pip or setuptools is declared it is currently not honored by poetry. This is regardless of whether this is done in [tool.poetry.dependencies], setup.py, or [build-system]\nrequires = [...]. pip, setuptools, and distribute seem to get filtered out in two places:
puzzle.provider.Provider.UNSAFE_PACKAGES and repositories.installed_repository.InstalledRepository.load (would need to call pip freeze --all).

For now manually upgrading pip, setuptools, and distribute as decribed above seems to be the only way.

@sdispater

Checked our packages and there are a lot with setuptools>=40.0.0 constraint (on Windows any package with src-layout). However, there are none with < or <= constraints on pip, distribute, setuptools, or wheels. I'd assume < or <= constraints on pip, distribute, setuptools, or wheels are very rare in general. Encountered no build system beyond setuptools and poetry.

As a tactical solution pip, setuptools, and wheels could be upgraded to the version found in the installation running poetry during venv creation. This is not fully deterministic as now the resulting installation depends on the interpreter running poetry at the time of virtual environment creation.

A "proper" implementation of setup requirements/build system might in a worst-case scenario require virtual environments separate from the target virtual environment to build packages. I'd suspect this would be a lot more effort and since today very few packages actually declare their setup requirements properly likely in vain.

@sdispater what do you think?

@asodeur

@asodeur The thing is that it's a tricky subject. Poetry uses pip internally so allowing it to be manipulated as a dependency can lead to undesirable side effects.

Also, if setuptools, or pip for that matter, is treated like any other dependency, it can also lead to them being uninstalled if they are no longer present in the pyproject.toml, which would in turn make the virtualenv corrupted.

I am not sure what the best approach is to be honest.

@timonbimon

Ok, than I'd propose to simply upgrade pip and setuptools to the versions found in the base environment during venv creation for now. This is probably the least confusing behaviour, would resolve the issues encountered by @timonbimon and myself, and be fairly simple to implement (simple enough for me to create a pull request).

The proposed solution does not seem to work anymore, at least for me. Whenever I try to upgrade pip inside of poetry, I run into the following error:

Collecting pip
Using cached https://files.pythonhosted.org/packages/5c/e0/be401c003291b56efc55aeba6a80ab790d3d4cece2778288d65323009420/pip-19.1.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 10.0.1
Uninstalling pip-10.0.1:
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\Users\me\AppData\Local\Temp\pip-uninstall-32ke2pg9\users\me\appdata\local\pypoetry\cache\virtualenvs\my-package-py3.7\lib\site-packages\pip\_vendor\urllib3\packages\ssl_match_hostname\pycache\_implementation.cpython-37.pyc'

I was not able to find the root of this issue. Does this have something to do with the problems that poetry and pip have?

@asodeur

As @asodeur pointed out this is an issue with ensurepip. You can see the related bug report and discussion in the python bug tracker.

It would be nice to fix this but I think any fix should maintain the ability to create virtualenvs without internet access. Perhaps it could be a command line arguement or setting?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

This still seems like a valid bug.

Hi, we are facing the same issue where we did a following on a fresh windows machine:

install python 3.7.5
downgrade pip to 18.1
install poetry 1.0.0b7 through get-poetry.py script
do a poetry install on a project that needs pip 18.1, has it as an explicit dependency
install fails, and it says using "pip 19.2.1"

Can you help me with this? How do i force poetry to use a specific pip version?

@pavanmr94

@pavanmr94 a hacky workaround to get you through these dark times:

$ # Upgrades pip in a newly-installed virtualenv
$ poetry run pip install pip==18.1
$ # Installs your project dependencies, but with the correct version of pip
$ poetry install

I have the same issue. Lots of packages (e.g., opencv-contrib-python-headless) are failing to install because of the old version of pip.

why was this closed? As far as I can tell this is still an issue.

I tried to install scipy on macOS, but because it does not work with the Accelerate backend as of 1.6.0, it can't be built. There is a wheel, but a newer pip is required to install it.

What is the recommended approach in this case?

I am running into this as well; specifically with pyarrow version 3.0.0 which requires the latest pip to install on my MacOS. I'd be willing to look into creating a fix for this? Let me know if that's something you guys would appreciate.

@abn

Maybe @abn can shed some light here?

The pip version used by poetry for all virtual environments created after 1.1.0b1 uses the pip version bundled within the virtualenv package used by poetry. This is because since #2666 was merged poetry uses virtualenv and the bundled pip wheel when creating new virtual environenments.

All this said, depending on how you installed poetry (pip, pipx, get-poetry.py) and to what environment it was installed (system site, user site, virtualenvironment, poetry site), the version of pip that get triggerred will vary.

Further, with #2826, we are no longer installing pip into the virtual environments created. We use the pip wheels in-place from the virtualenv install. In future version's if your package requires an explicit version of pip, you need to define it as part of the development dependencies.

Hope this helps.

@abn

@abn at what point can we begin adding pip as a dev dependency?

The change is in master and the next 1.2.0 pre-release should have it.

Just install with ADMIN problem will solve

On my work VPN pip installs packages successfully while poetry will not. On open internet both work just fine. Platform: Windows 10, pip 21.1, poetry 1.1.8, poetry-core 1.0.4

This is the error I get from poetry:


Updating dependencies
Resolving dependencies...

  ConnectionError

  ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

  at c:\users\[redacted]\appdata\local\programs\python\python38\lib\site-packages\requests\adapters.py:498 in send
      494¦                     low_conn.close()
      495¦                     raise
      496¦
      497¦         except (ProtocolError, socket.error) as err:
    ? 498¦             raise ConnectionError(err, request=request)
      499¦
      500¦         except MaxRetryError as e:
      501¦             if isinstance(e.reason, ConnectTimeoutError):
      502¦                 # TODO: Remove this in 3.0.0: see #2811

Could this be related to poetry using its own version of pip? I don't see this specific error searching the issues.

Do we have an ETA for the 1.2 release to go live?

FWIW, it feels like a similar problem is still around. Certain packages (e.g. mypy) fail to install using poetry when connected to corporate network. It might be related to #6791

The issue you have is unrelated -- it relates entirely to the issues I've linked you to in your previous issue.

For those spelunking, this change has been partially reverted. The present state of things is this:

Poetry does not depend on pip being in the target environment; if it is missing, we will fall back to the embedded copy in the virtualenv module.
However, if it is in the environment, we will use it.
Poetry will manage pip -- you can explicitly depend on it now in your pyproject.toml.
Poetry will always manipulate pip before managing any other packages to prevent breaking the copy of pip it uses for installer operations.

There are a lot of unrelated topics in the comments -- please start a Discussion or join Discord if you would like clarification on any of the points here.

@mirosval

why was this closed? As far as I can tell this is still an issue.

I tried to install scipy on macOS, but because it does not work with the Accelerate backend as of 1.6.0, it can't be built. There is a wheel, but a newer pip is required to install it.

What is the recommended approach in this case?

I am having the exact same issue. Any workaround found for this ? @mirosval

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Navigation Menu

Repository navigation

Which pip version does poetry use? #732

Question

7 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Which pip version does poetry use? #732

Description

Question

Activity

eblume commented on Dec 11, 2018

asodeur commented on Dec 12, 2018

asodeur commented on Dec 13, 2018

sdispater commented on Dec 13, 2018

asodeur commented on Dec 14, 2018

kummerer94 commented on Jun 12, 2019

brycedrennan commented on Aug 21, 2019

stale commented on Nov 13, 2019

chdsbd commented on Nov 13, 2019

pavanmr94 commented on Nov 22, 2019

pappasam commented on Dec 11, 2019

7 remaining items

IamGianluca commented on Aug 29, 2020

mirosval commented on Mar 8, 2021

stinodego commented on Mar 29, 2021

PetterS commented on Mar 29, 2021

abn commented on Mar 29, 2021

austinbutler commented on Apr 19, 2021

abn commented on Apr 19, 2021

vettrivikas commented on Jun 16, 2021

AEHamrick commented on Sep 22, 2021

ericriff commented on Feb 23, 2022

drorata commented on Oct 13, 2022

neersighted commented on Oct 13, 2022

daviibf commented on Jan 17, 2024

github-actions commented on Feb 29, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions