Skip to content

Cannot load models with bf16 norm ".scale" weights #383

Description

@stduhpf

https://huggingface.co/lodestones/Chroma1-Radiance/discussions/12

If the type of the norm scale tensor is b16, the model won't load, with this error:

VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Requested to load PixelspaceConversionVAE
loaded completely; 14486.01 MB usable, 0.00 MB loaded, full load: True
Warning: This gguf model file is loaded in compatibility mode 'sd.cpp' [arch:flux]
gguf qtypes: BF16 (424), F32 (3), Q4_K (190), Q5_K (38), Q8_0 (4)
model weight dtype torch.bfloat16, manual cast: None
model_type FLUX
!!! Exception during processing !!! Error(s) in loading state_dict for ChromaRadiance:
        While copying the parameter named "distilled_guidance_layer.norms.0.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.1.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.2.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.3.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.4.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "double_blocks.0.img_attn.norm.query_norm.scale", whose dimensions in the model are torch.Size([128]) and whose dimensions in the checkpoint are torch.Size([256]), an exception occurred : ('The size of tensor a (128) must match the size of tensor b (256) at non-singleton dimension 0',).
         [ and a lot more for every ".scale" tensor in the model]
Traceback (most recent call last):
  File "H:\ComfyUI\execution.py", line 516, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
  File "H:\ComfyUI\execution.py", line 330, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
  File "H:\ComfyUI\execution.py", line 304, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "H:\ComfyUI\execution.py", line 292, in process_inputs
    result = f(**inputs)
  File "H:\ComfyUI\custom_nodes\ComfyUI-GGUF\nodes.py", line 153, in load_unet
    model = comfy.sd.load_diffusion_model_state_dict(
  File "H:\ComfyUI\comfy\sd.py", line 1496, in load_diffusion_model_state_dict
    model.load_model_weights(new_sd, "")
  File "H:\ComfyUI\comfy\model_base.py", line 308, in load_model_weights
    m, u = self.diffusion_model.load_state_dict(to_load, strict=False)
  File "H:\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format([same spam as above]):

It seems to detect each bf16 weights as if it was two elements?

Activity

  1. added a commit that references this issue on Dec 17, 2025
  2. city96 commented on Dec 17, 2025

    @city96
    Owner

    Pushed a fix for that, though the conversion code in this repo shouldn't actually allow 1D tensors to be quantized (they're always stored in FP32). Still, now we just dequantize them on the fly for external models that do have them like the one you linked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions