feat: relax sentence-transformers and transformers version pins for 5.x compat

[j-sperling]

Summary

• Bump sentence-transformers from == 5.1.1 to == 5.3.0
• Cap transformers at < 5.0.0 (was <= 4.56.2) to allow 4.57.x while deferring v5 until numerical equivalence is confirmed
• Remove deprecated overwrite_output_dir from 3 test files (no-op on v4, required for eventual v5 support)

Numerical verification

ColBERT embeddings are bit-identical between transformers 4.56.2 and 5.4.0 (both with ST 5.3.0, torch 2.9.0):

Model	max abs diff	min cosine sim	Verdict
lightonai/GTE-ModernColBERT-v1	0.00	1.000000	Bit-identical
lightonai/colbertv2.0	0.00	1.000000	Bit-identical

The known transformers v5 divergences (huggingface/transformers#42889, huggingface/transformers#43697) affect T5/encoder-decoder and vision models due to weight tying refactoring. BERT-based architectures used by pylate's ColBERT models are unaffected.

Test plan

Tested with ST 5.3.0 + transformers 4.57.6 on Python 3.12 (Apple Silicon):

• Model loading works: GTE-ModernColBERT-v1, ColBERT-Zero, colbertv2.0
• Encoding produces correct shapes (per-token 128d embeddings)
• 13/16 tests pass (3 training tests fail on MPS -- pre-existing, unrelated)
• 1 model-loading test fails (jina-colbert-v2 HF config issue -- pre-existing)

Motivation

The current exact pin on sentence-transformers==5.1.1 prevents PyLate from being installed alongside other packages that depend on newer ST versions. This is a common integration pain point (see #144, #190).

The transformers<=4.56.2 upper bound blocks 4.57.x which includes bug fixes and performance improvements.

Related

• [Draft] Bump ST #190 (Draft: Bump ST to 5.2.0)
• Update CI, bump fast-plaid to 1.4.4, and add low_memory parameter #204 (CI updates for Python 3.10-3.14)

[NohTow]

Thanks for the PR!
Let me run the test and try to have a look ASAP

Edit:
I might take a bit of time to review because:

1. I am quite drowning lately but most importantly
2. This is a major update for transformers, so I need to check with @tomaarsen if there isn't any pitfall

In any case, we'll have to pin the exact version of ST to the latest one: we are not using >= because it can break when Tom release a new version. I know it's a pain and we also have to make updates manually each time, but this is the only way to keep something stable. This is the main argument in favor of merging PyLate upstream in ST.

[Samoed]

Suggested change

	"sentence-transformers >= 5.1.1",
	"sentence-transformers >= 5.1.1, <6.0.0",

[matospiso]

if you're using uv, you can temporarily avoid this inconvenience by adding

[tool.uv]
override-dependencies = [
    "transformers>=some_version",
    "sentence-transformers>=another_version",
]

to your pyproject.toml (until this PR is merged)

[NohTow]

Hey,
I chatted quickly with @tomaarsen and the new ST version indeed allows to use Transformers v5 but I have a few things to consider before making this update:

1. Transformers v5 seems to yield different values than v4.X for some models. Until we know why (Tom is currently working on multimodal update and did not have time to dig further), I would be a bit worried about lifting the version lock, as any new install would use v5 by default
2. Although all tests are passing, I need to take a bit of time to check the release note, as Tom did some internal modifications and I want to make sure everything is fine

I am sorry to be slow with some updates lately, I've been quite busy, but as raised above, you can change overcome those issues locally. Is there any very blocking setup that you are encountering? I'll try to fix everything ASAP

[j-sperling]

Thanks for the detailed feedback @NohTow -- totally understand both concerns. I dug into the transformers v5 numerical issue before responding.

ColBERT numerical comparison: transformers 4.56.2 vs 5.4.0

I ran pylate's encode() on identical inputs with transformers 4.56.2 and 5.4.0 (both with ST 5.3.0, torch 2.9.0):

Model	max abs diff	min cosine sim	Verdict
lightonai/GTE-ModernColBERT-v1	0.00	1.000000	Bit-identical
lightonai/colbertv2.0	0.00	1.000000	Bit-identical

The known transformers v5 divergences (huggingface/transformers#42889 for T5 weight tying, huggingface/transformers#43697 for RTDetrV2) affect encoder-decoder and vision model families. BERT-based architectures -- which all pylate ColBERT models use -- produce identical outputs.

Happy to test additional models if you'd like.

Updated PR

Based on your feedback, I've updated the branch:

1. ST pin: == 5.3.0 (exact pin per your policy)
2. Transformers: removed the <= 4.56.2 ceiling, kept the original >= 4.41.0 floor unchanged
3. overwrite_output_dir removal: kept (needed for transformers 5.x TrainingArguments)

If you'd prefer to keep the transformers ceiling until Tom confirms independently, I can revert that part and just ship the ST bump. Your call.

[TheAdamEvans]

The transition to 5.*+ is a breaking change and feels too big right now.

In this PR, perhaps we can compromise and make this change to:
"transformers >= 4.41.0, <= 4.57.1",

[TheAdamEvans]

Should be no breaking changes this way --> transformers <= 4.57.1

[TheAdamEvans]

Don't need this addition in this PR.

AGENTS.md is another whole discussion and task.

[j-sperling]

Thanks @TheAdamEvans. Reverted the .gitignore additions -- clearly out of scope.

For the transformers ceiling: updated to <= 4.57.1 as you suggested. I'd initially gone with < 5.0.0 to avoid the pin going stale on each patch, but a tight ceiling is the safer call while the v5 numerical story is still being sorted out (see discussion above with @NohTow).

Kept the overwrite_output_dir removal -- it's a no-op on v4 (default was already False) and will be needed whenever v5 support lands.

[NohTow]

Hey,
Sorry for the delay, Raphael and I were at ECIR last week and are in vacation this week
Will focus on the various PRs when I get back
Apologies for the delay again

[tomaarsen]

Hello!

I've ran some more tests between transformers v4 and v5, and the discrepancies I spotted earlier were likely caused by differences in default options for dtype and attn_implementation. With those fixed, I'm only getting some discrepancies for ~3 rerankers and a handful of Sparse Encoder models, and no regressions for dense embedding models, across a total of 169 model tests.
In short, moving to transformers v5 seems pretty safe.

• Tom Aarsen

[TheAdamEvans]

No worries team, I know you have a lot on and these are small things, am patient! 😄 Looking to get PyLate included in Canva's monorepo. It can be tricky because the entire company has to share the exact same version specs. Thanks @j-sperling for updating 🙏

Re: sentence-transformers, I'd selfishly prefer <=5.1.2 because that's what we have here today and will simplify things for me later on the next PyLate release.

@j-sperling could I convince you to also update ninja to be widened to >=1.11.1? That === pin conflicts with other packages that currently resolve to 1.11.1.1 (a more commonly available version from what I can see).

Happy to open separate issues for these if we'd prefer.

[TheAdamEvans]

Might I also suggest

Suggested change

	"fast-plaid>=1.2.4.260,<=1.4.6.280",