Repository navigation
Poetry install and update resolve versions differently when requirements aren't pinned #2883
Description
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: Debian running in WSL2
- Poetry version: 1.1.0b2
- Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/iserko/ae0b5240bdb6af0a63264f834a3b4bee
Issue
Using the pyproject.toml I can run poetry install (log) and get the latest dbt installed (at the time that's 0.17.2), but then when I run poetry update (log) it downgrades dbt and updates requests.
The workaround for this is to pin dbt to dbt = "^0.17.2", however I was surprised to see poetry just downgrade.
Activity
dimbleby commentedon Jun 19, 2022
In general I don't think there's any guarantee that locking will give the same answer if you do it twice.
If the latest versions of two packages -
dbtandrequests- are not compatible, then at most one of them can 'win', and which one it is will depend on the exact path that poetry takes through the search.The promise that poetry makes is to find a solution where the packages are compatible with one another: not that it always finds the same solution.
If you care more about
dbtbeing at the latest than you do aboutrequestsbeing at the latest, then telling poetry so (as you have done) is the right thing to do.So while I agree that this behaviour is slightly surprising, I don't think it's a bug.
TBBle commentedon Jun 23, 2022
I would say it's a bug because the path that poetry takes through a search shouldn't be different for the same inputs. Either whatever is randomising that order needs to stop being random, or (better) the solution-evaluation rule needs to be extended in what is I guess a tie-break case.
I'd also suggest that trivially, in a tie-break case, if one of the options is "What's already locked", then it should choose that one, using hysteresis to avoid flip-flopping. That would take care of all the existing bug reports about this behaviour, since none are "It's choosing wrong when this", they're all "It's choosing differently from what it chose 20 seconds ago". Then whatever randomness is being done now would stop actually being a problem except when one deletes the lockfile and relocks, in which case a different solution is less-surprising.
That might actually be a simple change, since the flip-flopping around
poetry updatesuggests that it knows it's in a tie-break, but is choosing the option that isn't the currently-locked result.You could also define more rules for the tie-break cases, but I suspect no one will really care that much which one it's choosing, as long as it's consistent.
dimbleby commentedon Jun 25, 2022
It still isn't, though ;-)
FWIW I can't reproduce this today even with old versions of poetry, presumably because the universe of packages that poetry has to choose from has changed in the mean time.
If anyone cares about this enough to dig into it: one possibility that comes to mind is the code in
_choose_package_versionwhich decides which package to try and resolve next. It does that by looking for the package with fewest remaining versions available, afaik in case of a tie there's no particular reason that one or the other should win. Perhaps you could construct a repro or unit test with that clue...But IMO this remains at worst "slightly surprising" and not worth bothering about.
JacobHayes commentedon Sep 10, 2022
There's a reproducible example in #6470, as of today at least.
radoering commentedon Sep 11, 2022
From #6470:
For those who are interested:
poetry lockignores an existing lock file and creates a new lock file from scratch.poetry updatetakes a whitelist of packages to update. That's why it's more like apoetry lock --no-updateexcept for the whitelisted packages. It considers an existing lock file and tries not to update packages that are not on the whitelist.If you don't pass any names to
poetry updateall locked packages are on the whitelist, but the lockfile is still taken into consideration which can result in a different resolution order.I drafted #6477 to make
poetry updatemore determistic and similar topoetry lock.dimbleby commentedon Sep 17, 2022
duplicate #3779 can also be closed
github-actions commentedon Mar 1, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.