Skip to content

Separate inference dependencies from training and evaluation dependencies #2

Closed

Description

@WavesMan

Problem

uv sync --no-dev installs all packages declared in [project.dependencies].

For users who only want to run jeff-serve with an existing checkpoint, this also installs dependencies used for training, data preparation, evaluation, visualisation, or optional tooling.

This makes the installation larger and slower than necessary for inference-only use cases.

Proposal

Please consider providing a documented minimal inference dependency profile.

Possible approaches include:

  • a server optional extra;
  • separate dependency groups for serving, training, and evaluation;
  • a dedicated inference requirements file.

For example:

[project.optional-dependencies]
server = [
  # dependencies required to load a checkpoint and run jeff-serve
]
train = [
  # training-specific dependencies
]
evaluate = [
  # evaluation and benchmark dependencies
]

Then an inference-only setup could use something like:

uv sync --extra server --no-dev

Expected outcome

  • A smaller, faster installation for checkpoint serving.
  • Training and evaluation functionality remains available as opt-in dependencies.
  • The existing full-development workflow remains unchanged.
  • The inference profile is covered by a small smoke test that imports and starts jeff.server.

Activity

  1. firelex commented on Sep 29, 2026

    @firelex
    Owner

    Thanks, good suggestion. Done in db4a13d: uv sync --no-default-groups now installs only what jeff-serve needs (torch, torchvision, transformers, pillow, fastapi, uvicorn, safetensors, numpy, huggingface-hub): 63 packages on CPU against 123 for the full install. Add --extra cuda on NVIDIA GPUs for the fast Qwen3.5 kernels, or --extra mac for MLX on Apple silicon. Training, data and evaluation packages are in a train dependency group that plain uv sync still installs, so the development workflow is unchanged. A new test checks that importing the server pulls in no training packages. The README quick start shows the serving install; note that uv run also needs --no-default-groups there, or it re-adds the training group.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions