Skip to content

Detect aLoRA adapters and prevent incorrect eager activation #321

Closed

Description

@leehack

Problem

llamadart currently applies every native LoRA adapter globally from the start of inference. aLoRA adapters require activation only after their invocation-token sequence. Treating them as ordinary LoRA adapters can silently produce incorrect output.

llama.cpp added aLoRA support in ggml-org/llama.cpp#15327, and the pinned b10333 bindings expose the invocation-token inspection functions.

Immediate scope

  • Inspect loaded adapters for aLoRA invocation tokens.
  • Reject aLoRA through a typed, actionable unsupported exception until invocation-aware execution is implemented.
  • Document the current limitation so native LoRA support is not interpreted as aLoRA support.
  • Add ordinary-LoRA and aLoRA/version-skew regression tests.

Follow-up/full support

  • Detect the invocation-token boundary in the prompt.
  • Decode the prefix with the adapter disabled and enable it at the boundary.
  • Define prompt-cache behavior and prevent unsafe prefix reuse.
  • Reject unsupported multiple-aLoRA combinations.
  • Validate output against upstream llama-server with a real aLoRA adapter.

The safety guard should land independently of full aLoRA support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:P1Act next: current correctness, contract, or repository-integrity risk

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions