Repository navigation
Improve intersection and union of markers #5192
Description
The handling of markers offers great potential for improvement. Even if marker simplifications seem intuitive to a human being, the subject is more complex than it might seem at first glance. Thus, it's best to proceed in small steps.
This is a meta issue to keep track of these steps:
- Improve intersection of multi marker union poetry-core#226
- special case union of single marker with inverse poetry-core#255
- Marker simplifications poetry-core#282
- Improve constraint union poetry-core#283
- improve detection of Any marker union poetry-core#293
- merging python_version and python_full_version markers poetry-core#294
- Constraint intersection poetry-core#297
- Single marker union simplifications poetry-core#299
- Improve union of multi markers poetry-core#308
- markers: fix union of multi markers if one marker is a subset of the other marker poetry-core#352
- Marker fixes poetry-core#380
- don't merge python_version and python_full_version poetry-core#382
- preserver marker order at union poetry-core#383
- Treat marker intersection more symmetrically poetry-core#384
- Merge python_version and python_full_version markers poetry-core#388
- union of adjacent ranges should collapse poetry-core#390
- to be continued...
Rationale
When resolving dependencies, markers play an important role. Especially, when multiple-constraints-dependencies come into play, overrides with contradicting markers should be ignored to avoid unnecessary failures in dependency resolution and speed up the resolution. See #4695 for details. In order to detect contradicting markers, intersections of markers that cannot be fulfilled have to be simplified to EmptyMarker. Analoguous, unions of markers that are always fulfilled have to be simplified to AnyMarker. At the moment, markers are not always simplified enough so that poetry is not able to detect if a marker can never be fulfilled (empty marker) or is always fulfilled (any marker).
Apart from this use case, simplifying markers seems to be mainly a cosmetic issue when looking into poetry.lock or an exported requirements.txt. (When checking if a dependency has to be installed in the current environment, it does not matter if a marker has been simplified or not.)
Activity
dimbleby commentedon Feb 12, 2022
I think there's a good win to be had by teaching poetry-core that "python_version" and "python_full_version" are really the same thing. It's relatively common in my experience to end up with a marker like
python_full_version < "3.0.0" and python_version >= "3.6".Conceptually I'd suggest: convert both into a
python_full_version. Probably that would be a mechanically tedious change to make though, poetry-core leans in the direction of favouringpython_versionat the moment.dimbleby commentedon Feb 13, 2022
python-poetry/poetry-core#294
radoering commentedon Feb 13, 2022
From a technical perspective I agree this is the most obvious solution. From a user's point of view I would prefer
python_versioncompared topython_full_versionbecause it's shorter and simpler. Maybe, after converting both topython_full_versionand doing simplifications,python_full_verisoncould be converted topython_versionif the full version ends with.0? Maybe, I would even only want to have a conversion if this results into a simplification and otherwise keep the original marker name. However, I am not sure if such a sophisticated solution is worth the effort because in the end it is only cosmetic.By the way, a similar but maybe more challenging case is
sys_platformvs.platform_system. Often you end up with a marker likesys_platform == "win32" and platform_system != "Windows". In order to simplify these kind of markers, some kind of mapping is required. However, I am afraid it might be impossible to create a complete mapping and missing entries may result in issues during dependency resolution...dimbleby commentedon Feb 13, 2022
As you've seen I had a crack at the version / full_version case and it seems to have turned out fairly straightforward, albeit with room for still further improvement.
I think I've probably run out of ideas and enthusiasm in this area for now; not to mention that I'm already beginning to risk merge conflicts between the various MRs that are currently queued up. Over to you if you have more that you want to do - and to the maintainers to get these merged!
dimbleby commentedon Aug 28, 2022
Time to call this one done?
radoering commentedon Aug 28, 2022
Yeah, should be quite good now.
Closing since there haven't been much improvements/fixes anymore in the last time and I can't come up with a good solution for merging
sys_platformandplatform_system.github-actions commentedon Mar 1, 2024
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.