common : resolve <quant>-<sidecar> download tags and list cached sidecars by allozaur · Pull Request #28405 · ggml-org/llama.cpp

allozaur · 2026-09-04T19:43:59Z

Overview

llama_download_from_hf builds cache ids as <quant>-<sidecar> tags so sidecar files (mmproj, imatrix, draft variants) get their own cache entries, and GET /models now lists cached sidecar files alongside the main weights.

Additional information

Pure C++ change, no UI. Review together with the UI stack only for API shape; mergeable independently.

Requirements

I have read and agree with the contributing guidelines
AI usage disclosure: YES, llama-ui:unsloth/Qwen3.8-Flash-Next

Copilot

🟡 Changes recommended

There are correctness issues in sidecar matching/removal semantics (including an unreachable bare-sidecar removal path) and the new <quant>-<sidecar> behavior lacks test coverage in the existing resolution test suite.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR extends Hugging Face model resolution and cache enumeration to support <quant>-<sidecar> tags for sidecar-only downloads (e.g. Q4_0-mtp) and to expose cached sidecars in the cached-model listing used by GET /models.

Changes:

Add parsing for <quant>-<sidecar> tags and sidecar-only resolution when no primary model matches.
Include cached sidecar GGUFs in common_list_cached_models() under <quant>-<sidecar> tags.
Adjust delete-by-tag logic to handle sidecar-targeted removals.

File summaries

File	Description
common/download.cpp	Adds sidecar-tag parsing/resolution, updates cached model listing to include sidecars, and refines removal behavior for sidecar tags.

Review details

Files reviewed: 1/1 changed files
Comments generated: 4
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

angt · 2026-09-28T11:15:02Z

I’m not feeling confident about this PR 😅

$ ./build/bin/llama download -hf a-mo-yehia/Qwen3-VL-8B-Contracts-OCR-GGUF:F16
Downloading mmproj_f16.gguf ───────╴                                  18%

$ ./build/bin/llama download -hf pentacoxian-dev/Qwen3.8-Flash-Next-IQ3E-Q8D-MTP-GGUF
error: model download failed
0.00.057.669 E common_download_get_hf_plan: no GGUF files found in repository pentacoxian-dev/Qwen3.8-Flash-Next-IQ3E-Q8D-MTP-GGUF

angt · 2026-09-28T11:21:17Z

Also:

$ ./build/bin/llama download -hf ggml-org/gemma-4-E2B-it-GGUF:Q4_0-mtp
Downloading mtp-gemma-4-E2B-it-Q4_0.gguf ─────────────────────────── 100%
error: model download failed

angt · 2026-09-28T11:25:20Z

Not the same issue i belive:

$ ./build/bin/llama download -hf pentacoxian-dev/Qwen3.8-Flash-Next-IQ3E-Q8D-MTP-GGUF
error: model download failed
0.00.048.988 E common_download_get_hf_plan: no GGUF files found in repository pentacoxian-dev/Qwen3.8-Flash-Next-IQ3E-Q8D-MTP-GGUF

The grid keeps three columns below md and every context cell leaves with it. A phone cannot spare the 11rem the column holds. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The search takes a row of its own and the filters follow it. The search also grows to the default size there, so iOS does not zoom the page when it focuses. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A dropdown of 24px rows is not a touch target. Below md the same actions render in a bottom drawer, with the label of the trigger as its title, and the drawer closes before it runs an action that moves the focus. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The sheet of its own is gone: the manager already lists every model, searches them and holds their actions, so the chat trigger opens it on the model the conversation runs. The sheet ui component stays, the add menu still uses it. Assisted-by: pi (zai-org/GLM-5.3-Flash)

There is no room beside the table on a narrow screen, so the pane takes the whole dialog instead of narrowing it, and the dialog itself fills the viewport. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A 28px avatar, 14px id and no modality icons leave the id and its badges the width they need. The modality icons stay on the wider screens that can hold them. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The badge group keeps its width, so a tight row truncates the id around the labels instead of cutting the last one in half. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The models manager, mcp servers and settings dialogs take the whole screen with square corners below md and keep the tighter header padding, and a dialog whose body is flush puts its close back inside the corner. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The pane covers the screen there, so its header takes the padding the top of a screen wants instead of the padding a panel edge wants. Assisted-by: pi (zai-org/GLM-5.3-Flash)

Opening the models manager, the mcp servers or the settings from the expanded list on a phone left the list open behind them; it steps aside now. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A phone keeps the model id on a row of its own and puts its badges and icons together under it, and the actions under that take a little more room. Assisted-by: pi (zai-org/GLM-5.3-Flash)

One 40px target with a 16px cross, its cross on the 16px inset the titles use, shared by the dialogs, the model pane and the navigation rail. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The family row holds a placeholder for the context column, which a phone drops: it left the chevron with no column and pushed it onto a row of its own. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The dropdown of 24px rows is not a touch target, so a phone shows the same levels in a bottom drawer, with the hint on the icon's own title. Assisted-by: pi (zai-org/GLM-5.3-Flash)

Below md the grid is the model and its actions: the load control and the state it reports are the pane's job on a phone, where there is room for them. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The phone sheet becomes a drawer, like the row actions are. Its actions now match the desktop dropdown: one "Add files" row that leaves the type to the picker, "System Message" and the tools. Reasoning leaves the menu since the composer has its own control for it. The sheet component loses its last user, so it goes. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A 16rem card anchored above the dial has nowhere to sit on a phone, so the dial opens a drawer there with the usage, the load control and the details. The card stays for wider screens. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A panel moves focus to its first button on open, which is the corner close. On a phone the tap that opened the panel is already inside it, so that focus only draws a stray ring on the close. Wider screens keep the focus move. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A phone has no width for a column of its own, so the row's last cell floats at the row's end: the model id keeps the whole width and the kebab, or the fold control, stays where the thumb already is. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The bulb carries the state on its own there, and 12px reads small next to the other controls of the composer. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The composer trigger leads to the manager there, so picking a model and pointing this chat at it has to happen in the pane. "Use in this chat" selects the model the way the desktop dropdown does, loads it when a router server holds it cold, and hands the composer back. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A phone's rows sat tight against each other and against the edge of the pane. Below md the rows and the family rows take a wider gutter and a taller box, the section headers follow that gutter, and the toolbar leaves more air between the search and the filters. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The pane lists everything the model reports, so its actions ride at the bottom of the screen, where the thumb reaches them without scrolling. In an open chat the bar offers "Use in this chat" and not a second way to start a new one. Assisted-by: pi (zai-org/GLM-5.3-Flash)

At screen size the border only draws a line along the glass, on the three panels that fill the screen: settings, MCP servers and the models manager. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A tap leaves the trigger under a pointer that reports hover, so the ghost fill it would take stays behind. Touch pointers hold no fill; a mouse keeps its hover. Assisted-by: pi (zai-org/GLM-5.3-Flash)

A model whose capabilities are not known yet left an empty box in the model selector triggers, where it only took the gap its icons would have used. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The pill around the model name carried more air than its label needed there: the gap between icon and name narrows, and the padding drops below the sm breakpoint. Assisted-by: pi (zai-org/GLM-5.3-Flash)

The pane offers "Use in this chat" only when a chat is open, so without one the test looked for a button that was never there. Assisted-by: pi (zai-org/GLM-5.3-Flash)

…cars A Q4_0-mtp style tag now resolves the sidecar file when no model file matches it, so a solo draft or mmproj download actually pulls the file. Cached sidecar files list as their own entries so the state survives a restart, and removing such a tag deletes only the sidecar. Assisted-by: pi:zai-org/GLM-5.3

This was referenced Sep 4, 2026

server : fix deadlock when removing a finished download #28406

Closed

ui : Models Discover and Download (WIP) #28374

Closed

allozaur requested review from ServeurpersoCom and ngxson September 4, 2026 21:38

allozaur marked this pull request as ready for review September 4, 2026 21:38

allozaur requested a review from a team as a code owner September 4, 2026 21:38

Copilot AI lite review requested due to automatic review settings September 4, 2026 21:38

Copilot started reviewing on behalf of allozaur September 4, 2026 21:39

View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

Comment thread common/download.cpp Outdated

Comment thread common/download.cpp Outdated

Comment thread common/download.cpp Outdated

Comment thread common/download.cpp

allozaur force-pushed the allozaur/server-download-sidecar-tags branch from 4fa6000 to e4d84ce Compare September 4, 2026 22:49

allozaur requested a review from ggerganov as a code owner September 4, 2026 22:49

github-actions Bot added the testing Everything test related label Sep 4, 2026

allozaur force-pushed the allozaur/server-download-sidecar-tags branch 8 times, most recently from 36471ac to a56cfe1 Compare September 7, 2026 23:16

allozaur force-pushed the allozaur/server-download-sidecar-tags branch 2 times, most recently from 257fbf4 to 0949ace Compare September 16, 2026 11:14

allozaur force-pushed the allozaur/server-download-sidecar-tags branch from 0949ace to 95cf88d Compare September 28, 2026 10:15

allozaur force-pushed the allozaur/server-download-sidecar-tags branch from 95cf88d to 9654dd8 Compare September 28, 2026 14:34

allozaur requested a review from a team as a code owner September 28, 2026 14:34

allozaur removed this pull request from stack #28420 September 28, 2026 14:39

allozaur changed the base branch from master to allozaur/ui/models-manager September 28, 2026 14:42

allozaur added 29 commits October 6, 2026 09:21

ui : drop the context column on a phone

2cf7f84

The grid keeps three columns below md and every context cell leaves with it. A phone cannot spare the 11rem the column holds. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : stack the models toolbar on a phone

8bbb873

The search takes a row of its own and the filters follow it. The search also grows to the default size there, so iOS does not zoom the page when it focuses. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : cover the table with the model pane on a phone

ca223d7

There is no room beside the table on a narrow screen, so the pane takes the whole dialog instead of narrowing it, and the dialog itself fills the viewport. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : tighten the model row on a phone

04194e4

A 28px avatar, 14px id and no modality icons leave the id and its badges the width they need. The modality icons stay on the wider screens that can hold them. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : keep the model badges whole

dce6148

The badge group keeps its width, so a tight row truncates the id around the labels instead of cutting the last one in half. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : give the model pane header room on a phone

4e7bd7f

The pane covers the screen there, so its header takes the padding the top of a screen wants instead of the padding a panel edge wants. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : close the mobile navigation behind a dialog

01bb149

Opening the models manager, the mcp servers or the settings from the expanded list on a phone left the list open behind them; it steps aside now. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : give the pane header its own rows on a phone

c8cdfd0

A phone keeps the model id on a row of its own and puts its badges and icons together under it, and the actions under that take a little more room. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : size the close of every panel the same on a phone

d9dba7a

One 40px target with a 16px cross, its cross on the 16px inset the titles use, shared by the dialogs, the model pane and the navigation rail. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : keep the family chevron on its row on a phone

88545ff

The family row holds a placeholder for the context column, which a phone drops: it left the chevron with no column and pushed it onto a row of its own. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : pick the reasoning effort in a drawer on a phone

4853b6c

The dropdown of 24px rows is not a touch target, so a phone shows the same levels in a bottom drawer, with the hint on the icon's own title. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : keep the status column off the phone table

11d12ad

Below md the grid is the model and its actions: the load control and the state it reports are the pane's job on a phone, where there is room for them. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : open the context panel in a drawer on a phone

bd46b17

A 16rem card anchored above the dial has nowhere to sit on a phone, so the dial opens a drawer there with the usage, the load control and the details. The card stays for wider screens. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : grow the reasoning bulb on a phone

aac3667

The bulb carries the state on its own there, and 12px reads small next to the other controls of the composer. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : drop the border of the phone's full screen dialogs

c066835

At screen size the border only draws a line along the glass, on the three panels that fill the screen: settings, MCP servers and the models manager. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : keep the reasoning trigger transparent on a touch screen

615ed15

A tap leaves the trigger under a pointer that reports hover, so the ghost fill it would take stays behind. Touch pointers hold no fill; a mouse keeps its hover. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : draw no capability box when there is nothing in it

799da9d

A model whose capabilities are not known yet left an empty box in the model selector triggers, where it only took the gap its icons would have used. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ui : tighten the phone model trigger

f8eeddf

The pill around the model name carried more air than its label needed there: the gap between icon and name narrows, and the padding drops below the sm breakpoint. Assisted-by: pi (zai-org/GLM-5.3-Flash)

test : give the switch test an open chat

4ca5a6e

The pane offers "Use in this chat" only when a chat is open, so without one the test looked for a button that was never there. Assisted-by: pi (zai-org/GLM-5.3-Flash)

ServeurpersoCom mentioned this pull request Oct 13, 2026

ui : Discover & Download Models from HF Hub #29584

Merged

Navigation Menu

Repository navigation

common : resolve <quant>-<sidecar> download tags and list cached sidecars - #28405

common : resolve <quant>-<sidecar> download tags and list cached sidecars#28405
allozaur wants to merge 82 commits into
allozaur/ui/models-managerfrom
allozaur/server-download-sidecar-tags

allozaur commented Sep 4, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

angt commented Sep 28, 2026

Uh oh!

angt commented Sep 28, 2026 •

edited

Loading

Uh oh!

angt commented Sep 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

allozaur commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Additional information

Requirements

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

angt commented Sep 28, 2026

Uh oh!

angt commented Sep 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

angt commented Sep 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

allozaur commented Sep 4, 2026 •

edited

Loading

angt commented Sep 28, 2026 •

edited

Loading