Skip to content

poetry publish fails for package with . in its name. #5782

Closed
Closed
poetry publish fails for package with . in its name.#5782
Labels
area/publishingRelated to PyPI/PEP 503 publishingkind/bugSomething isn't working as expectedversion/1.2.1

Description

@nekitdev
nekitdev
opened on Jun 6, 2022

Issue

For packages with . in the name:

[tool.poetry]
name = "example.py"

Poetry correctly builds wheels and sdists with example.py in the file name.
However, when uploading to PyPI with the canonical name (i.e. example-py), poetry publish fails with the following error:

HTTP Error 400: Start filename for 'example-py' with 'example-py'.

Activity

  1. nekitdev
    added
    kind/bugSomething isn't working as expected
    status/triageThis issue needs to be triaged
    on Jun 6, 2022

  2. mkniewallner commented on Jun 6, 2022

    @mkniewallner
    mkniewallner
    on Jun 6, 2022
    Member

    According to https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode, it seems that the right thing to do for wheels would be to replace . with _ for the distribution name.

    Though even if doing so, a bug in warehouse (the backend behind PyPI) may prevent the upload from working: pypi/warehouse#10030.

  3. nekitdev commented on Jun 7, 2022

    @nekitdev
    nekitdev
    on Jun 7, 2022
    Author

    It seems as though Poetry attempts to publish example-py with files containing example.py, causing PyPI to return the error above. I have hooked canonicalize_name function to see if PyPI allows uploading with example.py as the name, which it apparently does. So I'm assuming the mismatch is what causes the error.

  4. nekitdev commented on Jun 7, 2022

    @nekitdev
    nekitdev
    on Jun 7, 2022
    Author

    Once again, I'm not exactly sure how to solve this issue, though.

  5. neersighted commented on Jun 7, 2022

    @neersighted
    neersighted
    on Jun 7, 2022
    Member

    Once again, I'm not exactly sure how to solve this issue, though.

    We've already got a PR under review that uses the same logic pip does to canonicalize wheel names.

  6. nekitdev commented on Jun 7, 2022

    @nekitdev
    nekitdev
    on Jun 7, 2022
    Author

    Oh, I see, thanks! I'm looking forward to it. :)

  7. NiklasRosenstein commented on Jul 4, 2022

    @NiklasRosenstein
    NiklasRosenstein
    on Jul 4, 2022

    Hello, what's the progress on the PR @neersighted? I'm running into the same issue.

    I'm a bit surprised because on 20th of June (so only two weeks back), I was able to publish a package with . in the name: https://pypi.org/project/databind.core/2.0.7/#files

    But today I get

    400 Client Error: Start filename for 'databind-core' with 'databind-core'. for url: https://upload.pypi.org/legacy/

    I don't see any release to poetry-core since February though.

  8. NiklasRosenstein commented on Jul 16, 2022

    @NiklasRosenstein
    NiklasRosenstein
    on Jul 16, 2022

    I just realized that this is an issue with poetry-core==1.1.0b2, but not with the latest final release poetry-core==1.0.8.
    The way I had set up my build process was that I ended up using the a poetry-core beta release.

    I also confirmed the issue with 1.1.0b3.

  9. aaronhnsy commented on Sep 3, 2022

    @aaronhnsy
    aaronhnsy
    on Sep 3, 2022

    I am still getting this issue with poetry==1.2.0 and poetry-core==1.1.0.
    Building works fine but I am unable to publish the sdist file and publishing the wheel file results in the package having a dash in the name on pypi which is undesirable.

  10. dimbleby commented on Sep 3, 2022

    @dimbleby
    dimbleby
    on Sep 3, 2022
    Contributor

    Situation:

    • poetry always normalizes the package name at upload
    • poetry-core normalizes package names when building .whl but not when building .tar.gz
    • pypi insists that uploaded files start with a prefix that matches the package name

    So per this issue, folk who try for un-normalized package names find that poetry normalizes their package name at upload, and they can't upload their source distribution because it has the wrong prefix.

    Probably the sensible next step is to normalize .tar.gz names in poetry-core (and also make the corresponding change in poetry itself - the uploader class currently expects unnormalized source distributions, because it 'knows' what poetry-core did).

    That gets to a place where poetry insists on normalization, but at least it is possible to upload sdists.

    Not planning on doing this myself, no doubt MRs are welcome.

  11. mkniewallner commented on Sep 3, 2022

    @mkniewallner
    mkniewallner
    on Sep 3, 2022
    Member

    Probably the sensible next step is to normalize .tar.gz names in poetry-core (and also make the corresponding change in poetry itself - the uploader class currently expects unnormalized source distributions, because it 'knows' what poetry-core did).

    Last time I checked, there was no accepted PEP formally stating that sdists should be canonicalised, though the closest thing there is to that is this packaging specification which states:

    The file name of a sdist is not currently standardised, although the de facto form is {name}-{version}.tar.gz, where {name} is the canonicalized form of the project name (see PEP 503 for the canonicalization rules) with - characters replaced with _, and {version} is the canonicalized form of the project version (see Version specifiers).

    There is PEP 625 about canonicalising the name the same way as for wheels, but it is still in draft.

    Although PEP or not, it would seem more sensible to canonicalise sdists the same way as Poetry already does for wheels, for better consistency, especially since this (probably) shouldn't be a problem for any tool consuming the sdists.

  12. dimbleby commented on Sep 3, 2022

    @dimbleby
    dimbleby
    on Sep 3, 2022
    Contributor

    Sure, the proposal to canonicalize sdists in the same way as bdists wasn't intended to rule out the possibility of supporting un-canonicalized project names at some later point (nor to rule it in, come to that!).

    I suggest sdist canonicalization as a relatively low risk next step that can be taken without getting into that discussion at all. It's clearly a bug at the moment that some packages can't upload their sdists, start by solving that.

    Then later debate the merits or otherwise of supporting un-canonicalized names.

  13. neersighted commented on Sep 3, 2022

    @neersighted
    neersighted
    on Sep 3, 2022
    Member

    If anything this bug is PyPI being too strict in validation when different names are allowed by the spec -- technically Poetry is doing nothing wrong. However, from a practical standpoint I believe they use filenames to identify releases, so the requirement to match makes sense (even if it's undocumented/not required by spec).

  14. SvenBecker commented on Sep 23, 2022

    @SvenBecker
    SvenBecker
    on Sep 23, 2022

    Is there some way to opt out of the package name normalization or would it be possible to provide such an option? We are using some internal PyPI so I'm not sure if those PyPI validations even affect us.

  15. dimbleby commented on Sep 23, 2022

    @dimbleby
    dimbleby
    on Sep 23, 2022
    Contributor

    No. You will either need to put together an MR changing the code, or hope that someone else does.

  16. neersighted
    added
    area/publishingRelated to PyPI/PEP 503 publishing
    version/1.2.1
    and removed
    status/triageThis issue needs to be triaged
    on Sep 23, 2022

  17. NiklasRosenstein commented on Sep 27, 2022

    @NiklasRosenstein
    NiklasRosenstein
    on Sep 27, 2022

    What worked for me is to pin poetry-core==1.0.8:

    [build-system]
requires = ["poetry-core==1.0.8"]
build-backend = "poetry.core.masonry.api"

    But I build with Poetry build and publish with Twine; not sure if this still works with poetry publish.

  18. neersighted commented on Sep 27, 2022

    @neersighted
    neersighted
    on Sep 27, 2022
    Member

    This will not work with poetry publish as Poetry does not use the requires = field.

  19. neersighted commented on Oct 10, 2022

    @neersighted
    neersighted
    on Oct 10, 2022
    Member

    Closing as python-poetry/poetry-core#484 is merged.

  20. neersighted
    closed this as completedon Oct 10, 2022

  21. github-actions commented on Mar 1, 2024

    @github-actions
    github-actionsBot
    on Mar 1, 2024 – with GitHub Actions

    This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

  22. github-actions
    locked as resolved and limited conversation to collaborators on Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/publishingRelated to PyPI/PEP 503 publishingkind/bugSomething isn't working as expectedversion/1.2.1

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions