Skip to content

server : allow model downloads at model limit fix issue #26809 - #28530

Merged
ngxson merged 1 commit into
ggml-org:masterfrom
chenmiaoming:bugfix-26809
Sep 12, 2026
Merged

ngxson merged 1 commit into
ggml-org:masterfrom
chenmiaoming:bugfix-26809

Conversation

@chenmiaoming

@chenmiaoming chenmiaoming commented Sep 7, 2026 •

Copy link
Copy Markdown
Contributor

Overview

  • Bug: When --models-max is already at capacity, POST /models fails with "model limit reached, try again later" (see Misc. bug: Downloading a new model should be possible even if it would exceed models-max #26809).
  • Root cause: The under-lock capacity recheck in load(), intended to prevent concurrent overcommit, also rejects the download child process.
  • Fix: Apply the capacity check only for SERVER_CHILD_MODE_NORMAL.
  • Safety: Download child instances stay in the DOWNLOADING state, while is_running() only counts LOADED, LOADING, and SLEEPING instances (tools/server/server-models.h). Therefore, downloads do not consume model slots, and the child instance is cleaned up after the download exits.
  • Test: Set models_max=1 and preload MODEL_B to occupy the only slot, then start a model download. Verify that the download succeeds and MODEL_B remains loaded.

Additional information

see #26809

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES I used GPT-5.6 Sol to help me with call-stack analysis ,I personally reviewed, understood, and tested the change

@chenmiaoming
chenmiaoming requested a review from a team as a code owner September 7, 2026 03:47
@github-actions github-actions Bot added the server label Sep 7, 2026
@ggml-gh-bot

ggml-gh-bot Bot commented Sep 7, 2026

Copy link
Copy Markdown

Hi @chenmiaoming, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

@ggml-gh-bot ggml-gh-bot Bot added the draft PR will be changed to draft by github-actions bot label Sep 7, 2026
@github-actions
github-actions Bot marked this pull request as draft September 7, 2026 03:51
@github-actions github-actions Bot removed the draft PR will be changed to draft by github-actions bot label Sep 7, 2026
@chenmiaoming
chenmiaoming marked this pull request as ready for review September 7, 2026 04:19
@ngxson

ngxson commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

/bot review

@ggml-gh-bot

ggml-gh-bot Bot commented Sep 7, 2026 •

Copy link
Copy Markdown

❌ Code review failed.

Error: `git` exited with code 128

@ngxson
ngxson merged commit 56381e4 into ggml-org:master Sep 12, 2026
28 of 31 checks passed
pl752 pushed a commit to pl752/llama.cpp that referenced this pull request Sep 15, 2026
fencerJP pushed a commit to fencerJP/llama-apu that referenced this pull request Sep 16, 2026
quimmedes pushed a commit to quimmedes/cafe-llama.cpp that referenced this pull request Sep 16, 2026
zsogitbe pushed a commit to zsogitbe/llama.cpp that referenced this pull request Sep 17, 2026
frostyautumnleaf pushed a commit to frostyautumnleaf/llama.cpp that referenced this pull request Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants