Synapse doesn't detect URL preview dependencies if built with poetry-core 1.3.0, raising runtime errors

This is causing CI failures:

Synapse 1.69.0rc1 does not provide the feature 'url_preview'

even though the requisite dependencies are installed. The problem is that python-poetry/poetry-core#476 now enforces https://peps.python.org/pep-0685/#specification. Wheels built by poetry-core include metadata describing an extra url-preview with a - but NOT url_preview with an _. This means the dependency-checking code in Synapse here

synapse/synapse/config/repository.py

Lines 206 to 208 in 7b67e93

    
           self.url_preview_enabled = config.get("url_preview_enabled", False) 
        
           if self.url_preview_enabled: 
        
               check_requirements("url_preview")

will fail.

People who've also installed from source after the release of poetry-core 1.3.0 are going to be seeing this.

Assuming we want to be able to use future poetry-core releases, we'll need to update the dependency-checking code to account for this.

#14080 should avoid the problem by pinning the build-system requirements. Leaving this issue open though in case we do want to use newer versions of poetry-core in the future.

Isn't the fundamental problem here the whole notion of dependecy checking used by this project?

For example, https://github.com/matrix-org/synapse/blob/develop/pyproject.toml#L177-L185 causes hard failures when setuptools_rust is not available at runtime even though it only exists as a hack around build-system problems.

Merge tag 'v1.69.0rc2' into develop

Isn't the fundamental problem here the whole notion of dependecy checking used by this project?

The checking is long-standing (circa 2015). My understanding is that it was originally intended to protect people who pip installed once, then git pulled a new version with stricter version requirements but didn't pip install again to process them.

Distributors who are specifically managing Synapse's dependencies are welcome to patch out the this mechanism.

For example, develop/pyproject.toml#L177-L185 causes hard failures when setuptools_rust is not available at runtime even though it only exists as a hack around build-system problems.

See #13926 and #13952.

I definitely share your distaste for Python packaging noted in #13926. The ever-changing official view on how to build and distribute Python software, ill-formed build ecosystem and tendency to laugh off distribution package problems with a "just use pip" response have been increasingly large obstacles to sensible handling of Python in Void Linux.

We've already patched out the setuptools-rust dependency but generally try to avoid carrying downstream patches unless they fix problems uniquely related to our build infrastructure.

This caused a startup error for me after building 1.67 AND 1.68 from source.

Any plans to long term support newer poetry-core versions?

Any plans to long term support newer poetry-core versions?

There are some thoughts on this in #14085 (comment).

~~This should be fixed in poetry-core, have you tried 1.3.2? There is already a change regarding the normaloized directory name: python-poetry/poetry-core#495~~

Edit: I was wrong. The core metadata specification mandates a hyphen.

Closing as #14085 has been merged into develop via the 1.69 release.

Why is poetry-core pinned to >=1.0.0,<=1.3.1 in pyproject.toml? Version 1.3.2 was released almost two weeks ago.

And what about "cache_memory"? I don't use that feature, but there's a check for "cache_memory" in synapse/config/cache.py.

Why is poetry-core pinned to >=1.0.0,<=1.3.1 in pyproject.toml? Version 1.3.2 was released almost two weeks ago.

This is a defensive choice---we've been bitten twice now by problems introduced by unbounded poetry-core requirements: here, and indirectly in #13849. Quoting #14085 (comment):

Discussed among the team. We'll constrain to <= 1.3.1 and will raise that upper bound to 1.3.2/1.4.0/another version if a) someone wants to use that version, and b) we can check that it's safe.

This happened in #14217.

And what about "cache_memory"? I don't use that feature, but there's a check for "cache_memory" in synapse/config/cache.py.

Thanks, I missed this and will fixup. Edit: #14221.

Merge tag 'v1.69.0' into merge-1.69.0

Navigation Menu

Repository navigation

Synapse doesn't detect URL preview dependencies if built with poetry-core 1.3.0, raising runtime errors #14079

8 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	self.url_preview_enabled = config.get("url_preview_enabled", False)
	if self.url_preview_enabled:
	check_requirements("url_preview")

Synapse doesn't detect URL preview dependencies if built with poetry-core 1.3.0, raising runtime errors #14079

Description

Activity

DMRobertson commented on Oct 6, 2022

8 remaining items

ahesford commented on Oct 6, 2022

DMRobertson commented on Oct 7, 2022

ahesford commented on Oct 7, 2022

ShadowJonathan commented on Oct 9, 2022

darix commented on Oct 14, 2022

DMRobertson commented on Oct 14, 2022

bnavigator commented on Oct 15, 2022

DMRobertson commented on Oct 17, 2022

voegelas commented on Oct 18, 2022

DMRobertson commented on Oct 18, 2022

DMRobertson commented on Oct 18, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions