Skip to content

Validate and add Models to supported model list #116

Description

@ravi9

Verify the following GGUF models and add them to the supported model list.
For an unsupported model, create an issue for tracking.

  1. Granite 4.0 Micro : granite-4.0-micro-Q4_K_M.gguf
  2. Qwen 3 4B : Qwen3-4B-Q4_K_M.gguf
  3. GPT OSS 20B : gpt-oss-20b-Q4_K_M.gguf
  4. Google Gemma 3N E4B : gemma-3n-E4B-it-Q4_K_M.gguf
  5. Mistral Small 3.2 24B : Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf
  6. Phi-4 mini instruct Q4_K_M : Phi-4-mini-instruct-Q4_K_M.gguf
  7. Gemma3_4B it Q4_0 : gemma-3-4b-it-qat-q4_0-gguf
  8. Qwen3-VL 2B FP16 (mmproj) : mmproj-F16.gguf
  9. Qwen3-VL 2B Q4_K_M (model) : Qwen3-VL-2B-Instruct-Q4_K_M.gguf
  10. Qwen3.5 Q4_K_M
  11. Qwen3.5 Q4_0

Activity

  1. shahkarnav115-beep commented on Mar 27, 2026

    @shahkarnav115-beep

    Hey @ravi9, can I contribute to this issue? I've previously run the TinyLlama-1.1B-Chat-v1.0-f16.gguf during my earlier testing, so I already have the test environment set up and ready to go!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions