Integrate AutoRound into Diffusers

docs/source/en/_toctree.yml

@@ -180,6 +180,8 @@
     title: quanto
   - local: quantization/modelopt
     title: NVIDIA ModelOpt
+  - local: quantization/autoround
+    title: AutoRound
   title: Quantization
 - isExpanded: false
   sections:

docs/source/en/quantization/autoround.md

@@ -0,0 +1,206 @@
+<!-- Copyright 2026 The HuggingFace Team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License. -->
+
+# AutoRound
+
+[AutoRound](https://github.com/intel/auto-round) is an advanced quantization toolkit. It achieves high accuracy at ultra-low bit widths (2-4 bits) with minimal tuning by leveraging sign-gradient descent and providing broad hardware compatibility. See our papers [SignRoundV1](https://arxiv.org/pdf/2309.05516) and [SignRoundV2](https://arxiv.org/abs/2512.04746) for more details.
+
+
+Install `auto-round`(version ≥ 0.13.0):
+
+```bash
+pip install "auto-round>=0.13.0"
+```
+
+To use the Marlin kernel for faster CUDA inference, install `gptqmodel`:
+
+```bash
+pip install "gptqmodel>=5.8.0"
+```
+
+## Load a quantized model
+
+Load a pre-quantized AutoRound model by passing [`AutoRoundConfig`] to [`~ModelMixin.from_pretrained`]. The method works with any model that loads via [Accelerate](https://hf.co/docs/accelerate/index) and has `torch.nn.Linear` layers.
+
+You can use [`PipelineQuantizationConfig`] to quantize specific components of a pipeline:
+
+```python
+import torch
+from diffusers import DiffusionPipeline, PipelineQuantizationConfig, AutoRoundConfig
+
+pipeline_quant_config = PipelineQuantizationConfig(
+    quant_mapping={"transformer": AutoRoundConfig(backend="auto")}
+)
+pipe = DiffusionPipeline.from_pretrained(
+    "INCModel/Z-Image-W4A16-AutoRound",
+    quantization_config=pipeline_quant_config,
+    torch_dtype=torch.bfloat16,
+    device_map="cuda",
+)
+
+image = pipe("a cat holding a sign that says hello").images[0]
+image.save("output.png")
+```
+
+Or load a quantized model component directly:
+
+```python
+import torch
+from diffusers import ZImageTransformer2DModel, ZImagePipeline, AutoRoundConfig
+
+model_id = "INCModel/Z-Image-W4A16-AutoRound"
+
+quantization_config = AutoRoundConfig(backend="auto")
+transformer = ZImageTransformer2DModel.from_pretrained(
+    model_id,
+    subfolder="transformer",
+    quantization_config=quantization_config,
+    torch_dtype=torch.bfloat16,
+    device_map="cuda",
+)
+
+pipe = ZImagePipeline.from_pretrained(
+    model_id,
+    transformer=transformer,
+    torch_dtype=torch.bfloat16,
+    device_map="cuda",
+)
+
+image = pipe("a cat holding a sign that says hello").images[0]
+image.save("output.png")
+```
+
+> [!NOTE]
+> AutoRound in Diffusers only supports loading *pre-quantized* models. To quantize a model from scratch, use the [AutoRound CLI or Python API](https://github.com/intel/auto-round) directly, then load the result with Diffusers.
+
+## torch.compile
+
+AutoRound is compatible with [`torch.compile`](../optimization/fp16#torchcompile) for faster inference. You can compile the quantized transformer (DiT) for better performance:
+
+```python
+import torch
+from diffusers import DiffusionPipeline, PipelineQuantizationConfig, AutoRoundConfig
+
+pipeline_quant_config = PipelineQuantizationConfig(
+    quant_mapping={"transformer": AutoRoundConfig(backend="auto")}
+)
+pipe = DiffusionPipeline.from_pretrained(
+    "INCModel/Z-Image-W4A16-AutoRound",
+    quantization_config=pipeline_quant_config,
+    torch_dtype=torch.bfloat16,
+    device_map="cuda",
+)
+
+pipe.transformer = torch.compile(pipe.transformer, mode="default", fullgraph=False)
+```
+
+## Backends
+
+AutoRound supports multiple inference backends for Weight-only quantized model. The backend controls which kernel handles dequantization during the forward pass. Set the `backend` parameter in [`AutoRoundConfig`] to choose one:
+
+| Backend | Value | Device | Requirements | Notes |
+|---------|-------|--------|--------------|-------|
+| **Auto** | `"auto"` | Any | — | Default. Automatically selects the best available backend. |
+| **PyTorch** | `"torch"` | CPU / CUDA | — | Pure PyTorch implementation. Broadest compatibility. |
+| **Triton** | `"tritonv2"` | CUDA | `triton` | Triton-based kernel for GPU inference. |
+| **ExllamaV2** | `"exllamav2"` | CUDA | `gptqmodel>=5.8.0` | Good CUDA performance via the ExllamaV2 kernel. |
+| **Marlin** | `"marlin"` | CUDA | `gptqmodel>=5.8.0` | Best CUDA performance via the Marlin kernel. |
+
+
+```python
+from diffusers import AutoRoundConfig
+
+# Auto-select (default)
+config = AutoRoundConfig()
+
+# Explicit Triton backend for CUDA
+config = AutoRoundConfig(backend="tritonv2")
+
+# Marlin backend for best CUDA performance (requires gptqmodel>=5.8.0)
+config = AutoRoundConfig(backend="marlin")
+
+# ExllamaV2 backend for good CUDA performance (requires gptqmodel>=5.8.0)
+config = AutoRoundConfig(backend="exllamav2")
+
+# PyTorch backend for CPU/CUDA inference
+config = AutoRoundConfig(backend="torch")
+```
+
+
+## Save and load
+
+<hfoptions id="save-and-load">
+<hfoption id="save">
+
+AutoRound requires data calibration to quantize a model. This is done outside of Diffusers using the [AutoRound library](https://github.com/intel/auto-round) directly:
+
+```python
+from auto_round import AutoRound
+
+autoround = AutoRound(
+    "Tongyi-MAI/Z-Image",
+    scheme="W4A16",  # W4G128 symmetric
+    enable_torch_compile=True,
+    num_inference_steps=3,
+    guidance_scale=7.5,
+    dataset="coco2014",
+)
+autoround.quantize_and_save("Z-Image-W4A16-AutoRound")
+```
+
+For more details on calibration options, see the [AutoRound documentation](https://github.com/intel/auto-round).
+
+</hfoption>
+<hfoption id="load">
+
+
+```python
+import torch
+from diffusers import ZImageTransformer2DModel, ZImagePipeline
+
+model_id = "INCModel/Z-Image-W4A16-AutoRound"
+
+# The inference backend will be automatically selected.
+pipe = ZImagePipeline.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="cuda",
+)
+
+image = pipe("a cat holding a sign that says hello").images[0]
+image.save("output.png")
+```
+</hfoption>
+</hfoptions>
+
+
+### Supported Quantization Schemes
+
+AutoRound supports several Schemes:
+
+- **W4A16**(bits:4,group_size:128,sym:True,act_bits:16)
+- **W8A16**(bits:8,group_size:128,sym:True,act_bits:16)
+- **W3A16**(bits:3,group_size:128,sym:True,act_bits:16)
+- **W2A16**(bits:2,group_size:128,sym:True,act_bits:16)
+- **GGUF:Q4_K_M**(all Q*_K,Q*_0,Q*_1 provided by llamacpp are supported)
+- **NVFP4**(Experimental feature, recommend exporting to `llm_compressor` format.data_type nvfp4,act_data_type nvfp4,static_global_scale,group_size 16)
+- **MXFP4**(**Research feature, no real kernel**, Standard MXFP4, data_type mxfp,act_data_type mxfp,bits 4, act_bits 4, group_size 32)
+- **MXINT4**(**Research feature, no real kernel**, Standard MXINT4, data_type mxint,act_data_type mxint,bits 4, act_bits 4, group_size 32)
+- **MXFP4_RCEIL**(**Research feature,no real kernel**, NVIDIA's variant, data_type mxfp,act_data_type mxfp_rceil,bits 4, act_bits 4, group_size 32)
+- **MXFP8**(**Research feature, no real kernel**, data_type mxfp,act_data_type mxfp_rceil,group_size 32)
+- **FPW8A16**(**Research feature, no real kernel**, data_type fp8,group_size 0->per tensor )
+- **FP8_STATIC**(**Research feature, no real kernel**, data_type:fp8,act_data_type:fp8,group_size -1 ->per channel, act_group_size=0->per tensor)
+
+Besides, you could modify the `group_size`, `bits`, `sym` and many other configs you want, though there are maybe no real kernels.
+
+## Resources
+
+- [Pre-quantized AutoRound models on the Hub](https://huggingface.co/models?search=autoround)

setup.py

@@ -130,6 +130,7 @@
     "onnx",
     "optimum_quanto>=0.2.6",
     "gguf>=0.10.0",
+    "auto-round>=0.13.0",
     "torchao>=0.7.0",
     "bitsandbytes>=0.43.3",
     "nvidia_modelopt[hf]>=0.33.1",

src/diffusers/__init__.py

@@ -7,6 +7,7 @@
     OptionalDependencyNotAvailable,
     _LazyModule,
     is_accelerate_available,
+    is_auto_round_available,
     is_bitsandbytes_available,
     is_flax_available,
     is_gguf_available,
@@ -123,6 +124,18 @@
 else:
     _import_structure["quantizers.quantization_config"].append("NVIDIAModelOptConfig")
 
+try:
+    if not is_auto_round_available():
+        raise OptionalDependencyNotAvailable()
+except OptionalDependencyNotAvailable:
+    from .utils import dummy_auto_round_objects
+
+    _import_structure["utils.dummy_auto_round_objects"] = [
+        name for name in dir(dummy_auto_round_objects) if not name.startswith("_")
+    ]
+else:
+    _import_structure["quantizers.quantization_config"].append("AutoRoundConfig")
+
 try:
     if not is_onnx_available():
         raise OptionalDependencyNotAvailable()
@@ -982,6 +995,14 @@
     else:
         from .quantizers.quantization_config import NVIDIAModelOptConfig
 
+    try:
+        if not is_auto_round_available():
+            raise OptionalDependencyNotAvailable()
+    except OptionalDependencyNotAvailable:
+        from .utils.dummy_auto_round_objects import *
+    else:
+        from .quantizers.quantization_config import AutoRoundConfig
+
     try:
         if not is_onnx_available():
             raise OptionalDependencyNotAvailable()

src/diffusers/dependency_versions_table.py

@@ -37,6 +37,7 @@
     "onnx": "onnx",
     "optimum_quanto": "optimum_quanto>=0.2.6",
     "gguf": "gguf>=0.10.0",
+    "auto-round": "auto-round>=0.13.0",
     "torchao": "torchao>=0.7.0",
     "bitsandbytes": "bitsandbytes>=0.43.3",
     "nvidia_modelopt[hf]": "nvidia_modelopt[hf]>=0.33.1",

src/diffusers/quantizers/auto.py

@@ -18,10 +18,12 @@
 
 import warnings
 
+from .autoround import AutoRoundQuantizer
 from .bitsandbytes import BnB4BitDiffusersQuantizer, BnB8BitDiffusersQuantizer
 from .gguf import GGUFQuantizer
 from .modelopt import NVIDIAModelOptQuantizer
 from .quantization_config import (
+    AutoRoundConfig,
     BitsAndBytesConfig,
     GGUFQuantizationConfig,
     NVIDIAModelOptConfig,
@@ -41,6 +43,7 @@
     "quanto": QuantoQuantizer,
     "torchao": TorchAoHfQuantizer,
     "modelopt": NVIDIAModelOptQuantizer,
+    "auto-round": AutoRoundQuantizer,
 }
 
 AUTO_QUANTIZATION_CONFIG_MAPPING = {
@@ -50,6 +53,7 @@
     "quanto": QuantoConfig,
     "torchao": TorchAoConfig,
     "modelopt": NVIDIAModelOptConfig,
+    "auto-round": AutoRoundConfig,
 }
 
 
@@ -143,6 +147,19 @@ def merge_quantization_configs(
         if isinstance(quantization_config, NVIDIAModelOptConfig):
             quantization_config.check_model_patching()
 
+        if quantization_config_from_args is not None and isinstance(quantization_config, AutoRoundConfig):
+            # For AutoRound, allow overriding fields like `backend` from user args,
+            # since the model config may store a default value (e.g. backend="auto").
+            for key, value in quantization_config_from_args.__dict__.items():
+                if key in ("quant_method",):
+                    continue
+                if hasattr(quantization_config, key) and getattr(quantization_config, key) != value:
+                    warnings.warn(
+                        f"Overriding `{key}` in the model's quantization_config with value {value!r} "
+                        f"from the user-provided `quantization_config`."
+                    )
+                    setattr(quantization_config, key, value)
+
         if warning_msg != "":
             warnings.warn(warning_msg)
 

src/diffusers/quantizers/autoround/__init__.py

@@ -0,0 +1 @@
+from .autoround_quantizer import AutoRoundQuantizer