Repository navigation

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA) - #2571

Merged

BenjaminBossan merged 17 commits into

huggingface:mainfrom

gapsong:qa_lora

Jun 26, 2025

Merged

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA)#2571
BenjaminBossan merged 17 commits into
huggingface:mainfrom
gapsong:qa_lora

Commits on Jun 4, 2025
- Add support for Quantization-Aware Low-Rank Adaptation (QALoRA) in LoraLayer and GPTQLoraLinear
  gapsong
  committed
  edc88cb
Commits on Jun 9, 2025
- Refactor QALoraLinearVariant methods to raise NotImplementedError for unsupported operations.
  gapsong
  committed
  c5f4bf9
- Fix dtype handling and use foward in GPTQLoraLinear forward method
  gapsong
  committed
  af27857
Commits on Jun 11, 2025
- Enhance QALoRA support by refining group size handling and error management in variants
  gapsong
  committed
  6ebf4e5
Commits on Jun 14, 2025
- Enhance GPTQ and QALoRA implementations with improved error handling and refactor training tests
  gapsong
  committed
  ed2a7cc
Commits on Jun 16, 2025
- Refactor QALoRA implementation by removing unused memory printing function, enhancing gradient checks in tests, and improving adapter handling in model training.
  gapsong
  committed
  cad05f5
Commits on Jun 17, 2025
- Refactor test for QALoRA with GPTQ quantization by improving indentation and structure for better readability and maintainability.
  gapsong
  committed
  45c0c65
- Changed structure
  gapsong
  committed
  4f8d336
Commits on Jun 21, 2025
- Update README and config for QALoRA clarity; remove redundant line in tests
  gapsong
  committed
  1d76b08
Commits on Jun 22, 2025
Commits on Jun 23, 2025
- Update QALoRA examples and validation: change default group size to 32 in README and add validation for divisibility by group size in QALoraLinearVariant.
  gapsong
  committed
  6cdb5cd
- Refactor QALoraLinearVariant error messages for clarity and consistency; streamline argument handling in training script.
  gapsong
  committed
  8d4f9dd
Commits on Jun 24, 2025
- Refactor forward method in QALoraLinearVariant: streamline weight access, improve pooling logic, and enhance LoRA computation for clarity and efficiency.
  gapsong
  committed
  2fd3f77
- Remove redundant QALoRA gradient tests from RequiresGradTester for improved clarity and maintainability.
  gapsong
  committed
  8a08a10
Commits on Jun 26, 2025
- Moved QALoraLinearVariant to the bottom of variants.py
  gapsong
  committed
  8d85f75

Navigation Menu

Repository navigation

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA) - #2571

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA)#2571
BenjaminBossan merged 17 commits into
huggingface:mainfrom
gapsong:qa_lora

Commits on Jun 4, 2025

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA) in LoraLayer and GPTQLoraLinear

Commits on Jun 9, 2025

Refactor QALoraLinearVariant methods to raise NotImplementedError for unsupported operations.

Fix dtype handling and use foward in GPTQLoraLinear forward method

Commits on Jun 11, 2025

Enhance QALoRA support by refining group size handling and error management in variants

Commits on Jun 14, 2025

Enhance GPTQ and QALoRA implementations with improved error handling and refactor training tests

Commits on Jun 16, 2025

Refactor QALoRA implementation by removing unused memory printing function, enhancing gradient checks in tests, and improving adapter handling in model training.

Commits on Jun 17, 2025

Refactor test for QALoRA with GPTQ quantization by improving indentation and structure for better readability and maintainability.

Changed structure

Commits on Jun 21, 2025

Update README and config for QALoRA clarity; remove redundant line in tests

Commits on Jun 22, 2025

Refactor README and training script for QALoRA: improve readability by breaking long lines, update dataset loading to use provided data path, and remove unused arguments.

Refactor README for QALoRA: remove outdated command examples and unnecessary parameters for clarity and conciseness.

Refactor train_model function in QALoRA script: add data_split parameter, update dataset loading, and remove unused parameters for improved clarity and functionality.

Commits on Jun 23, 2025

Update QALoRA examples and validation: change default group size to 32 in README and add validation for divisibility by group size in QALoraLinearVariant.

Refactor QALoraLinearVariant error messages for clarity and consistency; streamline argument handling in training script.

Commits on Jun 24, 2025

Refactor forward method in QALoraLinearVariant: streamline weight access, improve pooling logic, and enhance LoRA computation for clarity and efficiency.

Remove redundant QALoRA gradient tests from RequiresGradTester for improved clarity and maintainability.

Commits on Jun 26, 2025

Moved QALoraLinearVariant to the bottom of variants.py