Repository navigation
poetry install does not honor build-system.requires #6154
Description
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: Ubuntu 20.04.4
- Poetry version: 1.2.0b2
- Link of a Gist with the contents of your pyproject.toml file:
pyproject.toml
[tool.poetry]
name = "meow"
version = "0.1.0"
description = ""
authors = [""]
build = "build.py"
[tool.poetry.dependencies]
python = "^3.9"
SoundFile = "0.10.*"
crcmod = "^1.7"
librosa = ">=0.8.0"
numba = "^0.56.0"
protobuf = "3.14.0"
tensorboard = "^2.10.0"
torchaudio = {version = "=0.12.1+cu113", source="torch"}
torch = {version = "=1.12.1+cu113", source="torch"}
[[tool.poetry.source]]
name = "torch"
url = "https://download.pytorch.org/whl/cu113"
secondary = true
[tool.poetry.dev-dependencies]
[build-system]
requires = ["poetry-core>=1.0.0", "setuptools>=40.6.0", "wheel", "numpy", "pybind11"]
build-backend = "poetry.core.masonry.api"
Issue
Running poetry build with the above pyproject.toml does the right thing:
Preparing build environment with build-system requirements poetry-core>=1.0.0, setuptools>=40.6.0, wheel, numpy, pybind11
[...snip...]
Running poetry install fails because it attempts to build the project without installing build-system.requires packages.
I would have expected the install command to create an isolated environment, install the build requirements and perform the build in that isolated environment, and then install the produced artifact in my current environment.
Reacted by Olivier 'reivilibre', Pavel Kulyov, Grzegorz Mrukwa, Sam Grayson, Jairo Llopis, Michael Blaß and benedikt-bartscher
Reacted by Thomas Burghout
Activity
bilelomrani1 commentedon Aug 14, 2022
Same issue on macOS 12.5. A dirty hack is to add the following at the top of
build.py:The exact same issue was described in #2789, which was marked as solved, but the issue is still present on Poetry 1.2.0b3. I didn't check if this is a regression from when this PR was merged.
neersighted commentedon Aug 17, 2022
This is currently intentional -- Poetry has never respected the
build-systementry for a Poetry-managed project. We might want to establish consensus on this being a desired behavior, since currently you can install any needed build dependencies using a Poetry group or similar (but this will result in duplication of deps in pyproject.toml).neersighted commentedon Aug 17, 2022
Re: this being a regression, the changes in that PR were explicitly scoped to use of build scripts.
Also, Poetry does not perform builds in an isolated/pep 517 environment, but instead uses the 'main' environment
poetry installuses, as this is the only way to use lockfile versions during the build with the current architecture. It is possible to re-architect this so that a clean build environment is used and thebuild-systemdeps are only installed there (as opposed to polluting thedevgroup implicitly or similar) -- however that will require the implementation of a lockfile-awarepoetry-corefirst (which we have already discussed/planned).arcbtc commentedon Sep 16, 2022
+1, having exact same issue
tool.poetry.build.scriptis not using dependencies from[tool.poetry.dependencies]randolf-scholz commentedon Nov 13, 2022
@neersighted For me this issue occurs when I have a GIT-dependency that has
jupyter-packagingas a build dependency. (specifically this fork here: https://github.com/randolf-scholz/jupyterlab_templates)Also, it does seem poetry uses an isolated environment to build this package.
`/tmp/tmpx6y_j1ku/.venv/bin/pip list`
neersighted commentedon Nov 13, 2022
Your issue is entirely different @randolf-scholz -- this issue is about
build-system.requiresfor Poetry-managed projects, and not for dependencies that don't use Poetry. Please reach out via Discord or Discussions for support.adamgreig commentedon Mar 22, 2023
I believe @randolf-scholz's issue is what I've reported as a separate issue in #7702, I couldn't find any other issue about it.
Samreay commentedon May 12, 2023
I raised this at #7909 as a potentially separate issue, but it might be the same thing I'm encountering.
The pyproject.toml is available on the linked issue, but for a TLDR here, we have a
build.pythat compiles cython, and so our build requires include both cython and numpy. Apoetry buildworks fine. But apoetry installdoes not, because it seems when "Getting requirements to build editable",setuptools/build_meta.pyis run (via the--isolatedtag in the parent pip invocation) in an environment which does not respect the build requirements.Full stack from our failing
docker buildbelow:radoering commentedon Jun 10, 2023
Trying to clarify some statements:
Not the same issue, only similar. #2789 descibes the use of a build script with
generate-setup-file = falseand #5401 fixed exactly that issue. However, this issue describes the use of a build script withgenerate-setup-file = true.Maybe, that was true in the past, but it's not anymore for all kinds of Poetry-managed projects (at least since #5401). I think it's still true for projects without build scripts but in this case there are no build-system requirements except for poetry-core anyway.
As it seems projects with build scripts are built in an isolated environment (no matter if
generate-setup-fileis true of false).Not the same, but it shows that neither locked dependencies nor build-system requirements are available. Locked dependencies are not available because an isolated environment is used instead of the project's environment.
#7975 looks like a sensible fix. It will make build-system requirements available.
It might still be surprising that locked dependencies are not available when building projects with build scripts but that's really something were consensus is needed (and that's not what this issue actually was about in the first place).
github-actions commentedon Feb 29, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.