Skip to content

Notes from integrating Jeff v1.2 into a coding-agent harness: template slots, consumer GPU, torchvision #8

Closed

Description

@JonoGitty

Thanks for Jeff. Three small things came up while we wired it into a coding-agent harness. Each one is independent.

1. jeff-kit check-rows rejects real template files. We built evaluation rows from real tool output a coding agent had read. One was a blog template containing {{AUTHOR}} and {{BODY_HTML}}, and it fails with "unfilled template slots". That check is right for generated training data. For evaluation rows made of real content, slots can be legitimate. An opt-out would help, keeping the strict check as the default: either a flag such as --allow-template-slots, or a per-row marker in source.

2. Numbers from a consumer GPU.

  • Setup: an RTX 3070 Ti Laptop GPU (8 GB) under WSL2, with torch 2.8.0+cu128, torchvision 0.23 and transformers 5.17, without flash-linear-attention.
  • Memory: with the v1.2 base, guard, ground, and a small separate classifier server loaded, nvidia-smi showed 2,098 MiB in use out of 8,192.
  • Latency: median per decision, measured from the client, was:
    • 144 ms for grounding rows with evidence capped at about 1,500 characters
    • 199 ms for ground adapter rows
    • 274 ms for guard on texts of up to about 4,000 characters
  • Versions: pyproject.toml pins torch 2.14, but serving worked on 2.8. That may be useful for people reusing an existing environment.

3. Text-only serving still needs torchvision. If torchvision is missing, startup fails with:

Qwen3VLVideoProcessor requires the Torchvision library but it was not found in your environment.

The cause is that AutoProcessor loads the vision-language processor. torchvision is already in the serving dependencies, so this only affects people who reuse an environment. A one-line note in the docs might save someone a few minutes.


✓ Patchwork Harness approved: drafted from measured runs, and checked before posting by a cross-vendor review (GPT-6.1 Sol) in Patchwork Harness, which now uses Jeff for its prompt-injection guard.

Activity

  1. firelex commented on Oct 6, 2026

    @firelex
    Owner

    Thank you for these notes. All three are dealt with on main (13e0123):

    1. Template slots: a row whose source has "real_template_slots": true may contain {{...}} slots. The strict check stays the default for generated rows. This is documented in the adapter-kit README.
    2. Consumer GPU numbers: the README's Speed and size section now quotes your RTX 3070 Ti Laptop numbers (about 2.1 GB, 144–274 ms medians), linking to this issue.
    3. torchvision: the README now says text-only serving still needs torchvision, because the processor also handles images.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions