[pluvium27]

Gemma-4 E4B models have global_head_dim inside text_config rather than at the top level. Add fallback to support both layouts.

Overview

This PR adds fallback logic for the global_head_dim field when converting the Gemma-4 E4B model. In the Google Gemma-4 E4B config.json, this parameter is stored inside text_config rather than at the top level of config.json. The current script raises a KeyError when the field is not found in hparams.

This fix first checks text_config, then falls back to hparams, and sets a reasonable default value of 256. This ensures compatibility with both the official Google model and its derivatives.

Additional information

Link affected: https://huggingface.co/google/gemma-4-E4B-it/resolve/main/config.json

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[CISC]

Your problem must be something else, text_config is already merged into root hparams at this point:

llama.cpp/conversion/base.py

Lines 1113 to 1115 in d0c407e

	if "text_config" in self.hparams:
	# move the text_config to the root level
	self.hparams = {**self.hparams, **self.hparams["text_config"]}

[CISC]

Your problem must be something else, text_config is already merged into root hparams at this point

Ah yes, huggingface/transformers#47384 changed the behavior of this entry, it now lives in per_layer_config:
https://github.com/huggingface/transformers/blob/7d5bc4f33b4555949cc0c0eb2c782a76181856c6/src/transformers/models/gemma4/configuration_gemma4.py#L210-L223

Please update fallback accordingly.

[pluvium27]

Your problem must be something else, text_config is already merged into root hparams at this point

Ah yes, huggingface/transformers#47384 changed the behavior of this entry, it now lives in per_layer_config: https://github.com/huggingface/transformers/blob/7d5bc4f33b4555949cc0c0eb2c782a76181856c6/src/transformers/models/gemma4/configuration_gemma4.py#L210-L223

Please update fallback accordingly.

Now it tries text_config.global_head_dim -> per_layer_config[*].head_dim -> head_dim default.

It works well on my laptop, logs at https://invent.kde.org/-/snippets/3862

Please take a look.

[CISC]

Now it tries text_config.global_head_dim -> per_layer_config[*].head_dim -> head_dim default.

That's still not quite right, as mentioned you should not read text_config, it has already been merged into root hparams, also only the full_attention per_layer_config entries actually holds the global head_dim, it's not good enough to just grab any.

[CISC]

Additionally, the same PR also moved num_global_key_value_heads, so that needs handling too.

[pluvium27]

I understand that text_config is merged into the root-level hyperparameters of TextModel, but gemma-4-E4B is a multi-modal model (Gemma4ForConditionalGeneration) that uses the MmprojModel path. And since MmprojModel does not inherit from TextModel, the operation of moving it to the root node does not take effect.

llama.cpp/conversion/base.py

Lines 2313 to 2322 in d0c407e

	class MmprojModel(ModelBase):
	model_type = ModelType.MMPROJ
	model_arch = gguf.MODEL_ARCH.MMPROJ
	preprocessor_config: dict[str, Any]
	global_config: dict[str, Any]
	n_block_keys = ["n_layers", "num_hidden_layers", "n_layer", "num_layers", "depth", "layers", "encoder_layers", "vt_num_hidden_layers"]
	has_vision_encoder: bool = True # by default
	has_audio_encoder: bool = False

llama.cpp/conversion/gemma.py

Lines 304 to 315 in d0c407e

	class ConformerAudioModel(MmprojModel):
	_batch_norm_tensors: list[dict[str, Tensor]] | None = None
	@staticmethod
	def is_audio_tensor(name: str):
	return any(p in name for p in ["audio", "codebook", "conformer", "depth_embedding", "depthformer", "depth_linear"])
	def tensor_force_quant(self, name, new_name, bid, n_dims):
	if ConformerAudioModel.is_audio_tensor(name):
	if ".conv" in name or "_conv" in name and ".weight" in name:
	return gguf.GGMLQuantizationType.F32
	return super().tensor_force_quant(name, new_name, bid, n_dims)

llama.cpp/conversion/gemma.py

Lines 353 to 357 in d0c407e

	@ModelBase.register("Gemma3nForConditionalGeneration")
	class Gemma3nVisionAudioModel(ConformerAudioModel):
	has_audio_encoder = True
	has_vision_encoder = True

The same applies to the other one.

[pluvium27]

Updated to only read head_dim from full_attention layers in per_layer_config, and added fallback for num_global_key_value_heads as well.

[CISC]

I understand that text_config is merged into the root-level hyperparameters of TextModel, but gemma-4-E4B is a multi-modal model (Gemma4ForConditionalGeneration) that uses the MmprojModel path. And since MmprojModel does not inherit from TextModel, the operation of moving it to the root node does not take effect.

These are parameters for the TextModel part though (Gemma4Model -> Gemma3Model -> TextModel).

[CISC]

added fallback for num_global_key_value_heads as well.

You forgot to push that I think.

[pluvium27]

added fallback for num_global_key_value_heads as well.

You forgot to push that I think.

Oh thanks for the reminder! I forgot to save the file before pushing😅.

Just pushed the update with fallback for num_global_key_value_heads.

[pluvium27]

These are parameters for the TextModel part though (Gemma4Model -> Gemma3Model -> TextModel).

You're right, but I'm converting the main model (Gemma4Model), and in that case, it occur the error that says text_config does not get merged into root hparams.

I think when extracting only the vision/audio encoder, only Gemma4VisionAudioModel is used, where the merge does not happen, occuring the weird keyerror.

I think the current version is robust to run.

[CISC]

These are parameters for the TextModel part though (Gemma4Model -> Gemma3Model -> TextModel).

You're right, but I'm converting the main model (Gemma4Model), and in that case, it occur the error that says text_config does not get merged into root hparams.

No, that's not the reason, text_config is merged, but you have the latest transformers installed that pops those keys away.

[pluvium27]

These are parameters for the TextModel part though (Gemma4Model -> Gemma3Model -> TextModel).

You're right, but I'm converting the main model (Gemma4Model), and in that case, it occur the error that says text_config does not get merged into root hparams.

No, that's not the reason, text_config is merged, but you have the latest transformers installed that pops those keys away.

Thanks for the clarification, but I'm afraid I may not install other versions of python-transformers package on my computer to verify your clarification.

Now that fallback solutions have been added for head_dim_full and num_key_value_heads_full, is there anything that needs to be done before merging?

[CISC]

Thanks for the clarification, but I'm afraid I may not install other versions of python-transformers package on my computer to verify your clarification.

I can verify that for you.

Now that fallback solutions have been added for head_dim_full and num_key_value_heads_full, is there anything that needs to be done before merging?

Yes, get entries from root, don't access text_config. :)

[pluvium27]

Thanks for the clarification, but I'm afraid I may not install other versions of python-transformers package on my computer to verify your clarification.

I can verify that for you.

Now that fallback solutions have been added for head_dim_full and num_key_value_heads_full, is there anything that needs to be done before merging?

Yes, get entries from root, don't access text_config. :)

Done. Updated to read them directly from root hparams.

Really appreciate your detailed review and explanations! :)