design(adapters): freeze Ollama adapter support and redirect to llama.cpp, or maintain #1633

Status (2026-10-20). Reframed as a decision issue. This issue now absorbs:

• Design a custom (non-catalog) adapter registration path for OllamaModelBackend #1650, custom (non-catalog) adapter registration for Ollama; its registration
  ergonomics are folded into "What maintaining involves" below;
• Ollama removed LoRA-adapter GGUF *creation* (v0.34.1+) — breaks adapter provisioning for #1634, serving unaffected #1651, Ollama's removal of LoRA-GGUF creation and the deprecation of the
  runtime adapter path; the evidence for this decision, with the discussion quoted
  below.

The original binding-semantics scope is preserved unchanged under "What maintaining
involves".

The decision

Ollama's adapter support is on a removal clock. Adapter creation has been dead since
v0.34.1 (three independent rejection gates: CLI pre-flight, request-level 400,
metadata-level; none behind a flag), and runtime serving of already-built adapter
models still works but carries Ollama's own warning: "LoRA adapters are deprecated and
will be removed in a future release". Mellea's Ollama adapter support (#1634) works
today against pre-built tags and nothing breaks at runtime, but every direction we
could invest in assumes a platform feature Ollama has committed to removing.

Option A: freeze and redirect (proposed). Stop investing in Ollama-specific adapter
machinery, declare the current support frozen, and redirect adapter serving and testing
to llama.cpp's llama-server. What freezing involves:

• Sweep the Ollama adapter refs: the backend-coverage notes in
  docs/docs/advanced/intrinsics.md, the weights-binding support table,
  lora-and-alora-adapters.md, the uncertainty_ollama.py example, and
  test/scripts/build_ollama_uncertainty_adapter.sh (already broken by the creation
  removal).
• Follow-up testing: move adapter e2e and automation to llama-server (llama.cpp) #1751 tracks moving adapter e2e and nightly coverage to
  llama-server; PR feat(backends): serve Granite Switch adapter functions on llama-server #1736, currently open, already serves Granite Switch adapter
  functions there.
• ServerMediatedBinding stays as it is for whoever needs it; no new semantics work.

Option B: maintain. Complete the binding semantics and registration ergonomics (the
full original scope below), accepting that Ollama may remove the runtime path
underneath it.

The evidence (from #1651, quoted)

• Creation is dead on three independent gates (CLI pre-flight, request-level HTTP
  400, metadata-level), enforced since v0.34.1, none behind a flag. Our build script
  hits the CLI gate. Verified against Ollama's own source and reproduced on macOS and
  on a Linux/CUDA node; v0.34.0 still had the old conversion path.

• One workaround was identified (LLAMA_ARG_MODELS_PRESET to point Ollama's
  underlying llama-server at raw adapter GGUFs) and immediately characterised by its
  proposer as "a pretty brutal hack" (gabe-l-hart, 2026-09-18).

• gabe-l-hart, in two comments on 2026-09-18, after tracing the deprecation through
  Ollama's source:

  Confirmed the logical path for any adapters is now officially deprecated. […]
  I think this makes it pretty clear: we should focus our efforts on
  https://llama.app/ for compositional adapters going forward.

• Merging is not equivalent for an aLoRA (planetf1, 2026-09-25): the weights are
  trained to apply only from an activation point in the prompt onwards, with the base
  model's KV cache reused for the prefix. Merging applies them at every position,
  including the prefix, which is not the arrangement they were trained in. A merged
  uncertainty-alora is a different model from the adapter served with activation, and
  its output would need checking rather than assuming.

• gabe-l-hart, 2026-09-25, on the path forward if we stay on Ollama:

  Wouldn't creating a Granite Switch out of the adapters and the base model be the
  right path forward? This would avoid the deep-merge of the weights (thus breaking
  non-adapted use cases) while still packing a single GGUF. For Granite-based models,
  this should be well supported in Ollama passing through to llama-server since
  Granite-Switch Architecture ggml-org/llama.cpp#25107.

• The activation question (planetf1, 2026-09-29, from the fix(backends): activate aLoRA adapters on the LocalHFBackend generate path #1685 work): an aLoRA can
  be loaded and reported active while never activating, because the activation sequence
  its config declares never appears in the assembled prompt, and PEFT reports nothing
  when that happens (see bug(adapters): detect when an aLoRA cannot activate, and stop selecting it silently #1678 for the detection, bug(adapters): published requirement-check aLoRA never activates, and scores worse than the LoRA #1679 for a concrete artefact mismatch).
  Whatever serves an adapter, we need a way to know activation actually fired; the
  ten-probe comparison harness from the local path transfers to a served tag.

What maintaining involves (original scope, unchanged)

Problem

ServerMediatedBinding represents a real deployment shape, but it remains a lifecycle
stub. PR #1622 adds Ollama support by selecting bundled adapter model tags through
backend configuration, outside the binding, and bypassing the generic adapter-function
lifecycle.

• A binding does not fully describe how its adapter is selected on a server.
• Selected adapter targets are neither validated nor prepared before first use.
• Adapter-function lifecycle telemetry cannot report server-mediated outcomes
  consistently.
• A future server-mediated backend would need to reproduce provider-specific target
  selection and concurrency handling.

The earlier vLLM-specific tracking issue #1143 was closed because its upstream API
dependency remained unavailable. This issue covers the provider-neutral binding contract
and an Ollama implementation, without assuming that vLLM support is now unblocked.

Requirements

1. Binding-owned server target selection
2. Per-request activation isolation
3. On-demand adapter availability
4. Complete invocation outcome reporting
5. Explicit adapter_models migration contract
6. Backwards-compatible registration
7. Clear unsupported-operation behaviour
8. Extensible multi-provider support

Proposed approach

Make ServerMediatedBinding a concrete deployment binding rather than a marker. It
should own the server-side target needed to select an adapter and expose a per-request
activation operation aligned with EmbeddedBinding.apply_activation() where the
provider request shapes permit it.

For Ollama, resolve the selected bundled model from the binding at invocation time.
Availability should be checked or retrieved lazily on first use, with per-target
single-flight coordination so concurrent first requests neither duplicate retrieval nor
race on shared backend state. Provider selection must remain request-local; it must not
use shared activate/deactivate state that could route concurrent calls to the wrong
adapter.

The implementation must define exactly-once adapter-function outcomes for success,
provider failure, JSON or schema failure, and cancellation. It should preserve ordinary
generation telemetry, which already records the selected provider model.

adapter_models in PR #1622 is provisional configuration. The implementation must
decide and document whether it remains a compatibility input that constructs bindings,
is deprecated in favour of direct binding registration, or remains the public Ollama
convenience API. It must define precedence and conflict behaviour. This work does not
migrate raw io.yaml execution into IOContract; composed-adapter configuration
caching remains unchanged.

Alternatives:

• Keep adapter_models on each backend. This is smaller, but leaves the binding
  incomplete and repeats provider wiring.
• Reuse EmbeddedBinding. This avoids another implementation, but conflates
  chat-template activation with selecting a separately served model.
• Add provider-specific bindings without a common server-mediated contract. This keeps
  core simpler initially, but makes each future backend solve target selection,
  availability, and telemetry independently.

The work can ship in phases: establish the binding contract and request-isolation tests
first, migrate Ollama with compatibility support second, then evaluate other providers
independently. It does not change IOContract execution, catalogue semantics, or shim
removal tracked separately in #1621.

Registration ergonomics (from #1650)

OllamaModelBackend.resolve_adapter() (and the cold-resolve path inside the
Requirement-rerouting logic in _generate_from_context) only resolves adapter
functions in the Granite intrinsics catalog. add_adapter() accepts a custom
(non-catalog) adapter function, but registering one without a matching adapter_models
tag is confusing: a direct Intrinsic call raises ValueError, and automatic
Requirement→aLoRA rerouting logs a warning and falls back to LLM-as-a-judge. Neither
is a registration API; both are safety nets for a misconfiguration with no ergonomic
way to avoid it. The config= dict add_adapter() requires for a
ServerMediatedBinding also has no documented shape of its own, and
docs/docs/tutorials/07-custom-adapter-function.md is HF/LocalFileBinding-specific.

Candidate shapes: a parameter on add_adapter() (or a sibling method) that requires the
serving Ollama model tag at registration time, so adapter_models and the composed
adapter register together instead of drifting out of sync; and documenting the config=
shape, extending the custom-adapter tutorial to cover the Ollama/server-mediated case.

Testing and documentation (for option B)

• Unit tests for binding construction, configuration migration, invalid targets, and
  first-use coordination.
• Mock integration tests proving request-local provider selection under concurrent
  invocations.
• Tests for success, provider error, JSON or schema error, and cancellation outcomes.
• Deterministic live Ollama e2e coverage, included in the nightly Ollama run.
• Updated adapter API docstrings, advanced adapter documentation, and the Ollama
  example.
• No qualitative test is needed because target selection, lifecycle outcomes, and
  structured adapter output are deterministic contracts.

Related

• feat(backends): support Ollama adapter functions #1634: Ollama adapter support (merged; what option A would freeze).
• feat(backends): serve Granite Switch adapter functions on llama-server #1736 (open PR): Granite Switch adapter functions on llama-server; the redirect
  target once it merges.
• Ollama removed LoRA-adapter GGUF *creation* (v0.34.1+) — breaks adapter provisioning for #1634, serving unaffected #1651 (closed, absorbed here): full evidence and discussion; its comment thread
  holds the quotes above.
• Design a custom (non-catalog) adapter registration path for OllamaModelBackend #1650 (closed, absorbed here): registration ergonomics.