You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Separate inference dependencies from training and evaluation dependencies #2
uv sync --no-dev installs all packages declared in [project.dependencies].
For users who only want to run jeff-serve with an existing checkpoint, this also installs dependencies used for training, data preparation, evaluation, visualisation, or optional tooling.
This makes the installation larger and slower than necessary for inference-only use cases.
Proposal
Please consider providing a documented minimal inference dependency profile.
Possible approaches include:
a server optional extra;
separate dependency groups for serving, training, and evaluation;
a dedicated inference requirements file.
For example:
[project.optional-dependencies]
server = [
# dependencies required to load a checkpoint and run jeff-serve
]
train = [
# training-specific dependencies
]
evaluate = [
# evaluation and benchmark dependencies
]
Then an inference-only setup could use something like:
uv sync --extra server --no-dev
Expected outcome
A smaller, faster installation for checkpoint serving.
Training and evaluation functionality remains available as opt-in dependencies.
The existing full-development workflow remains unchanged.
The inference profile is covered by a small smoke test that imports and starts jeff.server.
Thanks, good suggestion. Done in db4a13d: uv sync --no-default-groups now installs only what jeff-serve needs (torch, torchvision, transformers, pillow, fastapi, uvicorn, safetensors, numpy, huggingface-hub): 63 packages on CPU against 123 for the full install. Add --extra cuda on NVIDIA GPUs for the fast Qwen3.5 kernels, or --extra mac for MLX on Apple silicon. Training, data and evaluation packages are in a train dependency group that plain uv sync still installs, so the development workflow is unchanged. A new test checks that importing the server pulls in no training packages. The README quick start shows the serving install; note that uv run also needs --no-default-groups there, or it re-adds the training group.
Activity
Serving-only install: split training dependencies from serving (#2)
firelex commentedon Sep 29, 2026
Thanks, good suggestion. Done in db4a13d: uv sync --no-default-groups now installs only what jeff-serve needs (torch, torchvision, transformers, pillow, fastapi, uvicorn, safetensors, numpy, huggingface-hub): 63 packages on CPU against 123 for the full install. Add --extra cuda on NVIDIA GPUs for the fast Qwen3.5 kernels, or --extra mac for MLX on Apple silicon. Training, data and evaluation packages are in a train dependency group that plain uv sync still installs, so the development workflow is unchanged. A new test checks that importing the server pulls in no training packages. The README quick start shows the serving install; note that uv run also needs --no-default-groups there, or it re-adds the training group.
Serving-only install: split training dependencies from serving (#2)