Repository navigation
server: add notice for upcoming default port change 8080 --> 9931 - #26508
Merged
Merged
Conversation
ggerganov
approved these changes
Aug 3, 2026
ngxson
changed the title
Agusx1211
pushed a commit
to Agusx1211/llama-cpp-ds4f-m2-ultra
that referenced
this pull request
Aug 3, 2026
Context: - Bring the latest ggml-org llama.cpp master change into the DeepSeek V4 Flash M2 Ultra fork without rebasing shared history. - The incoming upstream range is f2b52a8..0b14b87 and contains one commit, ggml-org#26508. - The change is limited to llama-server startup diagnostics and does not alter model execution, DeepSeek V4 graph construction, Metal kernels, target build defaults, or benchmark tooling. Changes: - Merge upstream/master at 0b14b87 with both histories preserved. - Emit an informational warning after llama-server binds an address ending in :8080, announcing the planned future default-port change to :9931 and linking the upstream pull request. - Retain the current 8080 default; this merge changes neither binding behavior nor command-line compatibility. Rationale: - Keeping this small upstream server compatibility notice preserves the fork close to the moving upstream base without weakening any M2 Ultra or DeepSeek V4 specialization. - The merge applied automatically, so no fork-owned resolution or adaptation was necessary. Validation: - Audited the complete one-commit incoming log and diff; only tools/server/server.cpp changes by seven inserted diagnostic lines. - cmake -S . -B build-upstream-merge -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=OFF -DGGML_METAL=OFF configured successfully with GNU 15.2.0 on Linux x86_64. - cmake --build build-upstream-merge --target llama-server -j4 compiled and linked llama-server successfully. - Reconfigured with GGML_AVX2=OFF, GGML_BMI2=OFF, and GGML_FMA=OFF for the local Ivy Bridge host, then rebuilt successfully; the initial default x86 binary could not run because the build enabled instructions absent from that host. - A five-second no-model router-mode smoke on 127.0.0.1:8080 started, logged the exact :9931 notice and ggml-org#26508 reference after listening, then handled SIGTERM cleanup normally. - git diff --cached --check passed and git ls-files -u reported no unresolved entries. - Apple M2 Ultra validation was not run in this isolated integration lane; the lead agent owns target serialization and post-integration testing. Performance: - Not performance-sensitive. The merged code adds two warning log calls once at server startup when the listening address ends in :8080; no inference path changes and no performance benchmark was run. Risks and tradeoffs: - The notice deliberately fires for any explicit or default address ending in :8080, including custom hosts, matching upstream behavior. - The warning is temporary upstream policy and should eventually be removed when the default-port transition is complete. - Exact target-server compilation and smoke on Apple M2 Ultra remain an integration-level gate. Follow-up: - Merge this lane into a clean integration branch, rebuild the exact resulting commit on the serialized M2 Ultra lane, and run the normal server/correctness smoke before pushing. Agent: Codex Model: GPT-5 Signed-off-by: Codex (GPT-5) <codex-gpt-5@agents.invalid>
1 task
erusev
added a commit
to ggml-org/Llama-macOS
that referenced
this pull request
Aug 10, 2026
llama.cpp is moving its own default from 8080 to 9931 and already warns about it on 8080 (ggml-org/llama.cpp#26508); moving ahead of that flip breaks the fewest users, since the install base only grows. 8080 is also the most-collided port on a dev machine, so the new default is an improvement on its own. Users with an explicit port override are unaffected.
satindergrewal
pushed a commit
to satindergrewal/llama.cpp
that referenced
this pull request
Aug 12, 2026
…ml-org#26508) * server: add notice for upcoming default port change 8080 --> 6631 * add link to PR * correct to 9931
junjieyuan
added a commit
to junjieyuan/k8s-apps
that referenced
this pull request
Aug 15, 2026
llama.cpp will change its default port to 9931 in a future release (ggml-org/llama.cpp#26508); pin it explicitly now so the next image bump does not silently break the Service/HTTPRoute wiring.
kotetsuy
added a commit
to kotetsuy/EarthTourGuide
that referenced
this pull request
Aug 16, 2026
ttllm を AIassistant と symlink で共有している (EarthTourGuide/ttllm ->
../AIassistant/ttllm) ため、AIassistant 側 3764957 で ttllm の
LLAMA_SERVER_URL 既定値が 8080 -> 9931 に変わったのが、そのままこちらに
効いてしまう。start_all.sh は LLAMA_PORT=8080 のままだったので、
llama-server は 8080 で正常に動いているのに ttllm は 9931 を見に行き、
"llama": { "url": "http://localhost:9931", "reachable": false }
となって LLM に到達できなくなっていた。STT (NeMo) は正常なので、
音声は認識されるが返事が返ってこないという出方をする。
9931 側に寄せて統一する。あわせてドキュメントの 8080 表記も直した
(README / TECHNICAL の構成図とサービス一覧、three-vrm の依存条件)。
上流 ggml-org/llama.cpp#26508 がポート既定値の変更を予告しており、
AIassistant 側はそれに先回りしている。
検証: ttllm /health が reachable: true、/chat が 0.77s で応答、
world ツアーが東京タワー -> 万里の長城と flying/narrating を正常遷移。
注意: AIassistant と同じ 9931 を使うことになるので、両者は同時起動できない
(start_all.sh のポート占有チェックは 8000-8003 のみで 9931 は見ていない。
VRAM 的にもともと同時起動は不可)。
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
brittlewis12
pushed a commit
to brittlewis12/llama.cpp
that referenced
this pull request
Aug 17, 2026
…ml-org#26508) * server: add notice for upcoming default port change 8080 --> 6631 * add link to PR * correct to 9931
This was referenced Aug 21, 2026
algitbot
pushed a commit
to alpinelinux/aports
that referenced
this pull request
Aug 24, 2026
TheBrigandier
pushed a commit
to TheBrigandier/llama-server
that referenced
this pull request
Aug 25, 2026
Adding a second model to this repo would have meant a systemd file per (model x memory budget) pair, on top of a 406-line launcher whose defaults, validation and sampling were all specific to one GGUF. This replaces that with llama-server's built-in router mode: two units, two INI catalogs, no launcher. The axis of configuration changes. A unit is now a MEMORY BUDGET, not a model configuration: llama-router-full 29G/30G full.ini multi-user.target llama-router-limited 23G/25G limited.ini graphical.target Neither unit names a model. Everything model-shaped - which GGUF, n-cpu-moe, ubatch-size, ctx-size, cache-ram, sampling - moved into the catalogs, together with the measurement tables that justify each value. Adding a model is a [section] in a text file; it will never again require a new unit. full-128k and full-256k stop being deployments and become two sections of full.ini, selected by name in the request. --models-max 1 is what makes one fixed pair of ceilings correct for a catalog of any size: unload_lru() blocks until the outgoing model reports UNLOADED before the next load begins (server-models.cpp:762), so two models never co-exist in the cgroup. The default is 4, which on this box is an immediate OOM. Nothing loads at startup, and --sleep-idle-seconds (1800 full / 900 limited) releases the model and KV cache after idle. From boot until first use the server holds ~130 MB and no VRAM - previously it held 17.5 GiB and 10.8 GiB respectively, permanently. Waking costs a full 22GB re-read under --no-mmap, which is why the windows are minutes not seconds. Client-facing names are aliases, not sections. The ten OpenCode personas differ only in sampling and client-side limits, so they attach to the two sections as `alias =` entries. As separate sections under --models-max 1, switching persona would evict and reload 22GB - and OpenCode fires its small_model for title generation concurrently with the first message of every session. Measured: 17.9s cold load, then 5.2s for a request against a different alias of the loaded model. Names now carry full identity (publisher/model-quant@variant-persona). Note '@' and not ':' for the variant: the router treats ':' as the HuggingFace quant-tag separator and upper-cases what follows, so a section written ':128k' is served as ':128K' and the lowercase form 404s. Port moves 8080 -> 9931 ahead of llama.cpp changing its default (ggml-org/llama.cpp#26508), so a future upgrade is a no-op for clients. Two guarantees weakened, deliberately and worth knowing: - --ctx-size is no longer bounded to [131072, 262144] by anything. The check lived in the launcher; the router builds child argv itself and there is nowhere for it to live. It is now a review responsibility. - the --parallel/draft-mtp guard moved from script validation to `parallel = 1` in each catalog's [*] section. secrets.env is retired - nothing sources shell any more. Auth is the router's job: it owns --api-key-file and strips LLAMA_API_KEY from every child environment, so key values never reach a model process. Verified end to end, not just parsed: rendered child argv matches the old configs byte-for-byte (--ctx-checkpoints normalises to its --swa-checkpoints alias); systemd-analyze verify clean; live cutover on limited holds VRAM 10818 MiB vs 10894 and shmem 17.49 GiB vs 17.49, with every memory.events counter at zero; and `opencode run` against a multi-slash alias name completes a real generation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PKCtSAqcqsgbsFaTtGTF2D
Seanmcd15
approved these changes
Aug 31, 2026
doctorjei
added a commit
to doctorjei/droste-ai-halo
that referenced
this pull request
Sep 5, 2026
llama-server warns on every start that binds :8080 that its default port will become :9931 in a future release (ggml-org/llama.cpp#26508, fired from a string_ends_with on the listening address in tools/server/server.cpp). Moving now means moving once, on our own schedule, rather than inheriting the change mid-pin and having to explain a port that appeared to move by itself. Three of the four places the number lives are pinned against each other by g1lab/servewire.sh: the build-spec's SERVE_PORT_DEFAULT, the value llama.cfg shows on its commented DROSTE_LLAMA_PORT line, and installer/contract.sh's table. The README's port table follows. Existing boxes keep their own llama.cfg, which is seeded if_missing, so this reaches new boxes only -- an existing box goes on binding 8080 until its owner edits the line.
thecodacus
pushed a commit
to thecodacus/llama.cpp
that referenced
this pull request
Sep 7, 2026
…ml-org#26508) * server: add notice for upcoming default port change 8080 --> 6631 * add link to PR * correct to 9931
zbrad
pushed a commit
to zbrad/llama.cpp
that referenced
this pull request
Sep 10, 2026
…ml-org#26508) * server: add notice for upcoming default port change 8080 --> 6631 * add link to PR * correct to 9931
Merged
1 task
frostyautumnleaf
pushed a commit
to frostyautumnleaf/llama.cpp
that referenced
this pull request
Oct 5, 2026
…ml-org#26508) * server: add notice for upcoming default port change 8080 --> 6631 * add link to PR * correct to 9931
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Part of #26116 - transitioning llama-server from an on-demand command to a daemon process
Port gonna change from 8080 to 9931 (leetspeak of GGML)
Requirements