Repository navigation
testing: move adapter e2e and automation to llama-server (llama.cpp) #1751
Copy link
Copy link
Open
Labels
area/adapter-functionsGranite adapter functions: framework and adaptiers including RAG, Guardian, CoreGranite adapter functions: framework and adaptiers including RAG, Guardian, Corearea/backendsProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLMProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLMp2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.testing
Description
Summary
Our adapter serving and testing story is split across Ollama and llama.cpp, and the
ground is moving under the Ollama half: adapter creation is gone and the runtime path
is deprecated (#1633 has the decision on what we do there). llama-server already
serves Granite Switch adapter functions (PR #1736), and it's the layer Ollama itself
sits on.
This issue tracks focusing adapter testing and automation on llama-server.
Scope
- Adapter e2e and nightly coverage against
llama-server, alongside the LocalHF path. - Migrate what the Ollama removal blocked:
test/scripts/build_ollama_uncertainty_adapter.sh
and theuncertainty_ollama.pyexample need a llama.cpp-based provisioning route.
Two cautions carry over from the Ollama removed LoRA-adapter GGUF *creation* (v0.34.1+) — breaks adapter provisioning for #1634, serving unaffected #1651 thread: merging is not equivalent for an
aLoRA, and a served adapter should be checked for actually activating rather than
assumed (bug(adapters): detect when an aLoRA cannot activate, and stop selecting it silently #1678 has the detection approach). - A CI fixture strategy for llama-server-based adapter tests: models, adapter GGUFs,
pins.
Notes
- Deliberately less urgent than the current consolidation work; nothing blocks on it.
- Different axis from Retarget adapter-resolution e2e tests off the temporary private repo to a public fixture #1655: that one retargets resolution e2e off a private HF fixture
repo, this one is about the serving backend for adapter e2e. - If design(adapters): freeze Ollama adapter support and redirect to llama.cpp, or maintain #1633 goes to freeze, this becomes the main home for adapter automation.
Related
- design(adapters): freeze Ollama adapter support and redirect to llama.cpp, or maintain #1633: the Ollama freeze-or-maintain decision.
- PR feat(backends): serve Granite Switch adapter functions on llama-server #1736: Granite Switch adapter functions on llama-server.
- Retarget adapter-resolution e2e tests off the temporary private repo to a public fixture #1655: adapter-resolution e2e retarget (different axis).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/adapter-functionsGranite adapter functions: framework and adaptiers including RAG, Guardian, CoreGranite adapter functions: framework and adaptiers including RAG, Guardian, Corearea/backendsProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLMProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLMp2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.testing
Activity