Repository navigation
CPU variants: CUDA-graphs default, Windows x86-64 clang build in CI, attach-mode routes lifetime, per-thread log-worker attach - #502
Merged
Conversation
…lama-extras.txt on Windows The JLLAMA_CPU_VARIANTS path adds ggml before llama.cpp and therefore skips the block in which llama.cpp changes two ggml defaults; it repeated only GGML_LLAMAFILE_DEFAULT. ggml's own default for GGML_CUDA_GRAPHS is OFF, and a CUDA backend built as a module in this path ran without CUDA graphs and lost two thirds of its token generation (265.6 -> 88.8 tg128 on an RTX 3070 at b11538, docs/handover/local-agent-report-b11538-windows.md, addendum 3). Both defaults are repeated now, and the comment states the rule: every *_DEFAULT llama.cpp sets before its own add_subdirectory(ggml) is repeated here, re-checked with `grep -n _DEFAULT CMakeLists.txt` on every bump. Verified without a CUDA toolkit (none in this environment): ggml declares the option unconditionally, and the variants build's CMakeCache now reads GGML_CUDA_GRAPHS:BOOL=ON like the static build's; a CUDA build itself was not possible here. On Windows the variants build now writes jllama-extras.txt (ggml-base.dll, ggml.dll -- the two libraries jllama.dll links, left out of jllama-files.txt on purpose), byte-identical to what merge-native-artifacts.sh derives in CI: without it LlamaLoader extracted jllama.dll alone, Windows resolved its ggml.dll import from java.exe's directory or PATH, and 57 model-backed tests of a local variants build failed on the load. Both file lists are written with LF (file(GENERATE) wrote CRLF on Windows, measured), and the merge script strips a CR when it reads jllama-files.txt: with the CRLF list the previous script failed with "names 'ggml-cpu-x64.dll', which the build did not produce" (reproduced over a simulated 27-artifact tree), and a CR that slipped through would also have put every CPU module into jllama-extras.txt. A Linux x86-64 variants build configures and builds with the change (18 libraries + jllama-files.txt, no extras file), and the merge script accepts the simulated tree with a pre-existing extras file, rewriting it identically. CLAUDE.md "CPU variants" states the defaults rule; TODO's stage-3 item carries the module-beside-the-CPU-set findings (GPU module built but not copied, the graphs default now repeated, not yet re-measured with CUDA). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…efresh from the local agent's report net.ladenthin.llama.text.model, .draft.model, .reasoning.model and .rerank.model override TestConstants.MODEL_PATH, DRAFT_MODEL_PATH, REASONING_MODEL_PATH and RERANKING_MODEL_PATH, the way every capability-specific model already had one, so a developer can run the suite against models already on the machine. The defaults stay the CI model set: TestConstantsTest now pins that over the DEFAULT_* constants (an override must not red it) and checks each resolved constant against the resolved property, which holds whether or not it is set. CLAUDE.md and README name the JDK requirement the local agent lost a deliverable to: javac must accept -XDaddTypeAnnotationsToSymbol (NullAway's JSpecify mode), which Oracle JDK 21 does not and Temurin 21 does. The executed-test band includes the Windows measurement (1872), the local test plan no longer lists the four paths as the ones without an override, and the handover prompt docs/handover/local-agent-pr-a.md goes -- its deliverables are answered in docs/handover/local-agent-report-b11538-windows.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…add the clang C++ test job
build-windows-x86_64 now builds -DJLLAMA_CPU_VARIANTS=ON with plain clang
(-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++, the command line the
local agent verified at b11538: 18 DLLs, verify-native-deps.py clean, the Java
suite green against the artifact once jllama-extras.txt exists), the way the
two Linux CPU jars ship since 5.2.0: 14 ggml-cpu-<level>.dll modules of which
ggml loads the best for the running CPU. Plain clang because only the GNU
driver produces all 14 variants (CMake sets MSVC for clang-cl too, and ggml's
`if (NOT MSVC)` drops five); the CMakeLists refuses cl.exe and clang-cl. The
job drops -DBUILD_TESTING (the option refuses it: jllama_test links the CPU and
RPC backends, modules here) and the sccache install (upstream's unconditional
`-Xclang -fno-pch-timestamp` makes every clang TU non-cacheable -- 719 of 720
requests, 0.00% hits, measured -- so it only cost time), and prints
`clang --version` first, since the LLVM on PATH decides whether patches/0017's
compiler half is a warning (clang 20) or the error it fixes (22+).
test-cpp-windows-x86_64 runs the C++ suite in a static plain-clang build with
the tests (the split test-cpp-linux-x86_64 / -aarch64 have), so Windows x86-64
has the suite under both toolchains -- the two MSVC-classifier jobs keep running
it with cl.exe -- and patches/0017's compiler half is reachable in CI at all.
It gates both publish jobs through package. test-java-windows-x86_64 is
unchanged in substance and now runs inference through the variants artifact
(the loader pre-loads ggml-base.dll and ggml.dll from the artifact's
jllama-extras.txt).
Not verified here: the first run of both Windows jobs (no Windows machine in
this session; the variants build itself and the Java suite against it are the
agent's measurements, the static clang build with gtest and
verify-patches-applied.sh under Git Bash are not measured). CLAUDE.md ("CPU
variants", "Where C++ tests run": 6 build+ctest / 5 test-only, "Windows
natives"), README's classifier rows, TODO item 1 (done, the measurements stand)
and the CHANGELOG follow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…nt toggles become ModelParameters llama_server_attach (patches/0007) built a server_routes on the attach worker's stack frame. server_routes' constructor registers a sleeping-state callback on the model's task queue, and server_queue::on_sleeping_state only appends -- there is no unregister -- so after NativeServer.close() the frame was gone and the callback stayed: the model's next idle sleep (--sleep-idle-seconds) ran update_cached_responses() on a destroyed object (a lock on freed memory, a dangling meta pointer). Found at the b11361 bump, TODO.md "NativeServer attach mode leaves a sleep callback behind". The attach entry now takes the caller's server_routes -- llama_server_attach(argc, argv, server_context &, server_routes &) -- and native_server.cpp passes jllama_context::routes, the object the model load builds before load_model() (as upstream's llama_server() does) and keeps for the model's lifetime; a vocab-only model, which has none, is refused with a LlamaException. The declaration in the patch and native_server_bridge.h change accordingly; the hunk counts were recounted, check-patches.py is clean, and all 11 patches apply on a pristine b11538 checkout. Consequence, documented in NativeServer's Javadoc, the README and the CHANGELOG: the route handlers read the model's params, so route-level settings -- the endpoint toggles --metrics / --props / --slots, --slot-save-path, the slot count, the sampling defaults -- come from the model's ModelParameters, and such a flag in the attach argv is parsed and ignored; the argv keeps the HTTP side (--host, --port, --api-key, --ssl-*, --threads-http, ...). Because no ModelParameters setter existed for the toggles, ModelFlag gains METRICS, PROPS, SLOTS and NO_SLOTS (all registered by llama.cpp's server parser, which test_model_flags.cpp checks) with enableMetricsEndpoint(), enablePropsEndpoint(), enableSlotsEndpoint() and disableSlotsEndpoint(); ModelFlagTest and ModelParametersExtendedTest cover them. NativeServerAttachSleepIntegrationTest (model-backed: the draft model with --sleep-idle-seconds 1 and --metrics) attaches a server, closes it, lets the model sleep and wake, attaches a second server, scrapes /metrics through it, and sleeps and wakes once more. Not run here (no model in this environment, Hugging Face unreachable), so the defect was not shown red first; CI runs it on every Java job. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…ot once per log line LlamaModel.setLogger's trampoline runs on common_log's worker thread, a plain std::thread llama.cpp creates and re-creates on every pause/resume, which has never seen the JVM. Until now get_jni_env_attaching did AttachCurrentThread + DetachCurrentThread for every log line: leak-free, but each attach creates a java.lang.Thread object and fires the JVMTI thread events (TODO.md measured 13 lines of a failed load arriving on 13 distinct Thread objects), noticeable at --verbose volumes. A thread_local guard now records the attach and detaches in its destructor when the worker thread exits -- common_log::pause() joins the worker, a normal exit on which C++ TLS destructors run, also for a dlopen'd library (glibc keeps it mapped while TLS destructors are registered), and JNI_OnUnload pauses the worker through common_log_set_callback(nullptr) before anything else. The thread is attached as a daemon under the name jllama-log-worker (visible in thread dumps), so DestroyJavaVM never waits for the singleton logger's worker; a thread the JVM already owns is used as it is and never detached. Each delivery runs inside PushLocalFrame/PopLocalFrame, since a thread that stays attached and has no Java frame would otherwise accumulate local references until it exits. LlamaLoggerTest pins it (model-free, needs libjllama): every delivering thread is named jllama-log-worker, and N > 1 lines arrive on fewer than N distinct Thread objects. The per-line time cost was not measured: the model-free fixture logs too few lines for a number, and no model is available here. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
bernardladenthin
had a problem deploying
to
startgate
October 10, 2026 11:03 — with 
GitHub Actions
Error
GitHub Actions
Error
bernardladenthin
had a problem deploying
to
maven-central
October 10, 2026 11:03 — with
GitHub Actions
Failure
GitHub Actions
Failure
bernardladenthin
had a problem deploying
to
maven-central
October 10, 2026 11:03 — with
GitHub Actions
Failure
GitHub Actions
Failure
|
8 of 9 tasks
vaiju1981
pushed a commit
to vaiju1981/java-llama.cpp
that referenced
this pull request
Oct 12, 2026
…olicy Finished and therefore deleted (completed work is not recorded here): the Windows x86-64 variants build (shipped in bernardladenthin#502), "smoke tests must run inference through the loaded module" (the fat-jar smokes and Java jobs do), the jbang one-file example (examples/jbang/Chat.java), the macOS packaged- artifact gate's "not yet observed green" (green since run 37985786772), the struck-through dead-flags entry, the acquire/release context-guard unit tests (test_jni_helpers.cpp has them), the OSInfo alias test (OSInfoTest has it) and the Java8CompatibilityHelper entry (the class is gone). The SonarCloud and License-Compliance sections described PRs bernardladenthin#248/bernardladenthin#298, both merged in June/July; main's HEAD carries a green SonarCloud check and no license-scanner status, so they are replaced by one line: a dependency-license gate is the owner's decision. Upstream: one section, one policy -- nothing is filed before the release is out, not even drafts -- with the inventory of the eleven patches (nine submittable as they are, 0003 permanent, 0016 a temporary carry) and the two upstream-only defects worth an issue on that day; the scattered "file X upstream" bullets under RPC and the logging sink go. The two loader items (reuse the extraction across runs; per-process isolation) are merged, since both need the same Windows locked-file co-design. The unsigned-DLL item records the answer: the OpenPGP release key cannot sign them (Authenticode needs an X.509 code-signing certificate), the README names the way out, the certificate is the owner's decision. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
add_subdirectory(ggml)--GGML_CUDA_GRAPHS_DEFAULTwas missing, and a CUDA backend built as a module there ran without CUDA graphs (-66.6% tg128, measured by the local agent at b11538). On Windows the build writesjllama-extras.txtitself (byte-identical to the merge step's derivation), so a local variants build is loadable; both file lists are LF, andmerge-native-artifacts.shstrips a CR (a CRLFjllama-files.txtmade the previous script fail, reproduced over a simulated 27-artifact tree).cpu-windows-x86-64is the CPU-variants build with plain clang (build-windows-x86_64, the agent's verified command line; no sccache -- clang TUs do not cache because of upstream's-Xclang -fno-pch-timestamp; no ctest -- the option refuses it), plus the newtest-cpp-windows-x86_64(static clang build with the C++ suite, gating both publish jobs). The MSVC jobs keep the suite undercl.exe.NativeServerattach mode serves the model's ownserver_routes(llama_server_attachtakes it;patches/0007,native_server.cpp) instead of a frame-local one whose sleeping-state callback stayed registered on the model's queue afterclose()and ran on freed memory at the next idle sleep. Behaviour change: route-level settings (--metrics/--props/--slots,--slot-save-path, slot count, sampling defaults) are the model'sModelParametersnow --enableMetricsEndpoint(),enablePropsEndpoint(),enableSlotsEndpoint()/disableSlotsEndpoint()added. New model-backedNativeServerAttachSleepIntegrationTest.thread_localguard, daemon, namedjllama-log-worker,PushLocalFrame/PopLocalFrameper delivery) instead of once per log line;LlamaLoggerTestpins it (13 lines of a failed load now arrive on 1Threadobject, previously 13).-Doverrides for the four shipped model paths (net.ladenthin.llama.text.model,.draft.model,.reasoning.model,.rerank.model); the Temurin-not-Oracle JDK note (-XDaddTypeAnnotationsToSymbol); CLAUDE.md/README/TODO/CHANGELOG refreshed from the agent's report; the handover prompt removed.Test plan
jllama-files.txt);GGML_CUDA_GRAPHS:BOOL=ONin its cache (no CUDA toolkit here, so no CUDA build)merge-native-artifacts.shover a simulated 27-artifact tree with a CRLF Windows variants artifact and a pre-existingjllama-extras.txt: the old script fails, the new one passes and rewrites the extras file byte-identicallycheck-patches.py0 problems; llama.cpp fetched fresh through the applier (verify-patches-applied.sh: 11 applied)-Dnet.ladenthin.llama.test.ngl=0) against the static build: 1901 run, 0 failures, 0 errors, 286 skipped; loader smoke subset (NativeLibraryLoadSmokeTest,BackendLoadTest,LlamaLoggerTest,RpcServerTest) against the rebuilt variants library: 24/24spotless:apply,spotbugs:check,javadoc:jar, the five Python checks, buildcheck unit tests,reuse lintNativeServerAttachSleepIntegrationTestare unverified until thenRelated issues / PRs
Refs #500, #501 (the local agent's report this acts on), #499 (PR C).
Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.md🤖 Generated with Claude Code
https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2