model: Granite Speech Plus

[gabe-l-hart]

Overview

This PR adds support for ibm-granite/granite-speech-4.1-2b-plus. This model is a slight tweak on the GraniteSpeechForConditionalGeneration architecture that extracts multiple encoder layers and concatenates them into the projected token stream.

Additional information

• Granite Speech Architecture: https://github.com/huggingface/transformers/tree/main/src/transformers/models/granite_speech
• Granite Speech Plus Architecture: https://github.com/huggingface/transformers/tree/main/src/transformers/models/granite_speech_plus

Testing

# Download speech plus from HF
uvx --with transformers hf download ibm-granite/granite-speech-4.1-2b-plus --local-dir ibm-granite/granite-speech-4.1-2b-plus

# Convert
uvx --with-requirements requirements/requirements-convert_hf_to_gguf.txt --extra-index-url https://download.pytorch.org/whl/cpu python convert_hf_to_gguf.py ibm-granite/granite-speech-4.1-2b-plus
uvx --with-requirements requirements/requirements-convert_hf_to_gguf.txt --extra-index-url https://download.pytorch.org/whl/cpu python convert_hf_to_gguf.py ibm-granite/granite-speech-4.1-2b-plus --mmproj

# Grab the sample audio
wget https://huggingface.co/ibm-granite/granite-speech-4.1-2b/blob/main/multilingual_sample.wav

# Build
mkdir -p build && cd build
cmake .. && make -j

# Test inference
./bin/llama-mtmd-cli -m ../ibm-granite/granite-speech-4.1-2b-plus/granite-speech-4.1-2B-plus-BF16.gguf --mmproj ../ibm-granite/granite-speech-4.1-2b-plus/mmproj-granite-speech-4.1-2b-plus-BF16.gguf --audio ../multilingual_sample.wav -p "transcribe this audio to a written format" --jinja

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES

AI Disclosure

AI used in planning and building. All code reviewed and refined by hand.

git-ai-stats

╔══════════════════════════════════════════════════════════╗
║           GIT AI USAGE ANALYSIS                          ║
╚══════════════════════════════════════════════════════════╝

📊 COMMITS BY AGENT

--- Aggregate ---
Commits                        |      Count
---------------------------------------------
IBM Bob, OpenCode + Qwen3.6-35b |          2
---------------------------------------------
TOTAL                          |          2

📊 COMMITS BY USAGE TYPE

--- Aggregate ---
Commits                        |      Count
---------------------------------------------
none                           |          0
draft                          |          1
full                           |          1
---------------------------------------------
TOTAL                          |          2

📈 LINES OF CODE BY AGENT

--- Aggregate ---
Agent                     |    Commits |  Additions |  Deletions
------------------------------------------------------------
IBM Bob, OpenCode + Qwen3.6-35b |          2 |         90 |          1
------------------------------------------------------------
TOTAL                     |          2 |         90 |          1

📈 LINES OF CODE BY USAGE TYPE

--- Aggregate ---
Usage Type           |    Commits |  Additions |  Deletions
-------------------------------------------------------
none                 |          0 |          0 |          0
draft                |          1 |         56 |          1
full                 |          1 |         34 |          0
-------------------------------------------------------
TOTAL                |          2 |         90 |          1

[gabe-l-hart]

@ngxson more cleanup changes coming, will ping when review ready

[gabe-l-hart]

@ngxson Ok, ready for review now. Sorry to miss some of those naming mismatches.