Skip to content

LoRA: adapter loading#3

Merged
lmangani merged 5 commits intomasterfrom
lora-support
Feb 28, 2026
Merged

LoRA: adapter loading#3
lmangani merged 5 commits intomasterfrom
lora-support

Conversation

@lmangani
Copy link

  • Add safetensors reader and dit_ggml_load_lora (PEFT adapter_model.safetensors)
  • Apply LoRA at linear layers in DiT (self/cross-attn, MLP) when base weights are separate
  • CLI: --lora and --lora-scale in dit-vae
  • Example: examples/lora.sh + lora.json (duckdbot/acestep-lora-cryda)
  • README: LoRA section, example list, dit-vae options

qxip added 2 commits February 28, 2026 21:21
- Add safetensors reader and dit_ggml_load_lora (PEFT adapter_model.safetensors)
- Apply LoRA at linear layers in DiT (self/cross-attn, MLP) when base weights are separate
- CLI: --lora <path> and --lora-scale <float> in dit-vae
- Example: examples/lora.sh + lora.json (duckdbot/acestep-lora-cryda)
- README: LoRA section, example list, dit-vae options

Made-with: Cursor
- dit-graph.h: apply LoRA deltas when base uses fused QKV/gate_up/ca_qkv
  so self-attn, MLP, and cross-attn all use adapters (fixes no audible effect)
- dit-lora.cpp: fix safetensors parse (rewind fp before parse); normalize
  keys for base_model.model.layers.* and .lora_A.weight/.lora_B.weight
- request: add custom_tag (LoRA trigger) and genre; parse language,
  is_instrumental, formatted_lyrics
- dit-vae: append custom_tag to caption for condition encoder when set
- examples/lora.json: nu-disco example with custom_tag crydamoure

Made-with: Cursor
@lmangani lmangani marked this pull request as ready for review February 28, 2026 22:20
@lmangani lmangani merged commit 5a1e7a3 into master Feb 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant