Skip to content

llama.cpp b11538, InvalidRequestException (400 instead of 500), agent follow-ups closed - #499

Merged
bernardladenthin merged 4 commits into
mainfrom
claude/hopeful-pascal-9jlbqb
Oct 9, 2026
Merged

bernardladenthin merged 4 commits into
mainfrom
claude/hopeful-pascal-9jlbqb

Conversation

@bernardladenthin

Copy link
Copy Markdown
Owner

Summary

  • llama.cpp b11534 → b11538 (one step, 45 KiB): #30229 (CUDA round fix), #30160 (graph reorder for get_rows), #30228 (UI Models Manager: GET /models gains context_length; common_get_decision_type replaced by common_get_gguf_info(), which no project C++ called), #28331 (llama-bench -fitc). All 11 patches apply unchanged, the drop-checks are unchanged, the two history rows are written. RouterModel.getContextLength() exposes the new router field (0 from an older server or a GGUF without the key); the value.* PIT gate at 100 % is covered by RouterModelTest / RouterModelsResponseParserTest.
  • C2: InvalidRequestException (a LlamaException) from the JNI layer, and OpenAiCompatServer answers 400 instead of 500 for a request the native layer rejects. Upstream's server has answered these with 400 since b11337 (#29060); the Java transport reported server_error, because the JNI layer threw the same LlamaException for a rejected request and a failed inference. Native side: c_llama_invalid_request next to c_llama_error, chosen by three helpers with upstream's own rules — throw_invalid_request for request parsing (also behind parse_json_params and require_json_field), throw_by_exception_type (std::invalid_argument / common_json_error → 400, the rule of upstream's ex_wrapper; tokenization, format_prompt_infill, the /v1/systemone handler), and error_class_for for a failed task result via the new json_helpers get_result_error_code (400 for ERROR_TYPE_INVALID_REQUEST and ERROR_TYPE_EXCEED_CONTEXT_SIZE). The message of a rejected request is the bare reason now, no longer the JSON object the old throw_invalid_request wrapped it in (CHANGELOG "Changed"). Java side: the server catches the typed exception together with the IllegalArgumentException of its own validation at the ten request sites, on every surface (OpenAI, Ollama, Anthropic, Responses; an invalid_request_error SSE event once a stream has started); a plain LlamaException stays 500. The Server layer may now see the Exception layer (LlamaArchitectureTest).
  • D1/D2 (agent follow-ups) were already done; the TODO entries were stale. The approval gate exists (ApprovalMode, ConsoleApprovalStrategy, AgentRunner.approval with Atmosphere's ToolApprovalPolicy, --auto; pinned by ApprovalWireTest) — entry removed. Tool rounds across REPL turns ride as AgentSession.toolNote in front of the next user message, and the asked-for form (full ChatMessage list incl. tool_calls) is impossible on atmosphere-ai 4.0.72: AbstractAgentRuntime.assembleMessages rebuilds every history entry as new ChatMessage(role, content) (checked in the bytecode) — entry rewritten with that reason and the reopen condition. docs/handover/local-agent-pr-a.md stays with a status note, because deliverable 6 carries the local agent's B2 measurements and the four OPEN items the Windows CPU-variants job still needs; TODO.md "CPU variants" points at it.

Test plan

  • Affected unit / integration tests pass locally — C++ 607/607 (ctest, +4 for get_result_error_code); Java suite model-free 1845 run / 0 failures / 285 skipped (-Dnet.ladenthin.llama.test.ngl=0; +6 model-free HTTP tests: typed rejection is 400 for chat, embeddings, completions, Ollama and Anthropic, an SSE error event when streaming, a plain LlamaException 500); ErrorHandlingTest tightened to the typed exception plus an empty input array and a malformed body; model-backed 400 cases in OpenAiServerEmbeddingsIntegrationTest ("input":[], unknown encoding_format) and OpenAiServerCompletionIntegrationTest (no prompt) — these need CI's models; spotless:check, compile spotbugs:check (0), javadoc:jar (0 errors, 0 warnings), check-natives.py, check-patches.py, check-release-gate.py, check-run-scripts.py, check-shared-files.py, the 113 buildcheck unit tests, reuse lint all green; clang-format 23.1.3 applied.
  • CI is green on this branch
  • Docs / CHANGELOG updated where applicable — CHANGELOG (Added / Changed / Fixed), CLAUDE.md (pin, "The JNI exception boundary", the json_helpers function list, the test table, the FindClass list), TODO.md, docs/history/llama-cpp-breaking-changes.md, Javadoc on every LlamaModel entry point that can throw the new exception.

Related issues / PRs

Follows #496, #497 and #498. Closes the TODO.md entries "400 vs. 500 for an invalid request body", "Approval for destructive tools" and (rewritten) "Tool rounds are not carried across REPL turns".

Checklist

  • I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • My commits follow Conventional Commits
  • No security-sensitive changes (if there are, I have notified the maintainer privately per SECURITY.md)

🤖 Generated with Claude Code

https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2


Generated by Claude Code

claude added 4 commits October 9, 2026 22:10
All eleven patches apply unchanged (verified in order against the pristine
tag); see docs/history/llama-cpp-breaking-changes.md for the range.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…p b11538)

Upstream #30228 makes GET /models carry the trained context length per
model, read offline from the GGUF. The parser reads it, 0 when absent (a
server before b11538, or a file without the key); value tests cover the
getter, the constructor delegation and equals/hashCode. CHANGELOG entries
for the bump and the getter.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…ODO and handover updated

Two entries of the Atmosphere agent's follow-up list in TODO.md were stale:

- "Approval for destructive tools": the gate has shipped (ApprovalMode,
  ConsoleApprovalStrategy, AgentRunner.approval with Atmosphere's
  ToolApprovalPolicy, --auto for unattended runs; ApprovalWireTest and
  ConsoleApprovalStrategyTest pin it). Removed.
- "Tool rounds are not carried across REPL turns": rewritten with the
  measured reason the asked-for form is impossible on atmosphere-ai 4.0.72
  (AbstractAgentRuntime.assembleMessages rebuilds every history entry as
  new ChatMessage(role, content), checked in the bytecode), what ships
  instead (AgentSession.toolNote in front of the next user message, pinned
  by LocalAgentTest), and when to reopen it.

docs/handover/local-agent-pr-a.md said it is deleted once PR A is merged;
PR A and the agent's corrections are, but deliverable 6 carries the
agent's measurements and the four OPEN items the Windows CPU-variants job
(B2) still needs, so the file stays with a status note and TODO.md's
"CPU variants" entry points at it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
…AiCompatServer answers 400

llama.cpp's own HTTP server has answered a malformed or empty request with
400 (invalid_request_error) since b11337 (#29060); the Java transport
answered 500, because the JNI layer threw the same LlamaException for a
rejected request and a failed inference and the server could not tell
them apart without matching on message text (TODO.md, "400 vs. 500").

Native side: a second exception class, exception.InvalidRequestException
(a LlamaException), cached as c_llama_invalid_request next to
c_llama_error. Three helpers decide which one a failure gets, each with
upstream's rule: throw_invalid_request for request parsing (an invalid
request whatever the parser threw; now also behind parse_json_params and
require_json_field), throw_by_exception_type for upstream code whose
exception type carries the verdict (std::invalid_argument and
common_json_error are 400, as in upstream's ex_wrapper; tokenization,
format_prompt_infill, the /v1/systemone handler), and error_class_for for
a failed task result, from the new json_helpers get_result_error_code
(400 for ERROR_TYPE_INVALID_REQUEST and ERROR_TYPE_EXCEED_CONTEXT_SIZE).
The empty-input and pooling-type checks of handleEmbeddings and a 400 from
the systemone route are typed the same way. The message of a rejected
request is the bare reason now, no longer the JSON object the old
throw_invalid_request wrapped it in; the type is the class.

Java side: OpenAiCompatServer catches InvalidRequestException together
with the IllegalArgumentException of its own validation at the ten
request sites (400, or the invalid_request_error SSE event once a stream
has started), on every surface. A plain LlamaException stays 500. The
Server layer may now see the Exception layer (LlamaArchitectureTest).

Tests: four C++ tests for get_result_error_code (607 total); six
model-free HTTP tests (a backend throwing the typed exception is 400 for
chat, embeddings, completions, Ollama and Anthropic, an SSE error event
when streaming, a plain LlamaException 500); ErrorHandlingTest tightened
to the typed exception plus an empty input array and a malformed body;
model-backed 400 cases in the embeddings and completion server
integration tests. Javadoc on every entry point that can throw it;
CHANGELOG, CLAUDE.md (the JNI exception boundary, the helper and test
tables) and TODO.md updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AytmJF9faEiQEVt6eetQS2
@sonarqubecloud

sonarqubecloud Bot commented Oct 9, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
61.9% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@bernardladenthin
bernardladenthin merged commit ed9b4f1 into main Oct 9, 2026
11 of 17 checks passed
@bernardladenthin
bernardladenthin deleted the claude/hopeful-pascal-9jlbqb branch October 9, 2026 22:38

This branch had an error being deployed

1 failed deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants