feat: add managed local AI with llama-server - #1178
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. |
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
|
Codex review: needs maintainer review before merge. Reviewed August 21, 2026, 6:12 AM ET / 10:12 UTC. ClawSweeper reviewWhat this changesThe branch adds opt-in, hardware-qualified Local AI onboarding that installs a verified llama-server and model, connects it to the managed WSL gateway, and exposes tray lifecycle controls. Merge readinessKeep open. This remains an unmerged, substantial Local AI feature with credible current-head proof; its rollout scope and long-term core ownership need explicit maintainer sponsorship before merge. Priority: P2 Review scores
Verification
How this fits togetherWindows onboarding checks hardware and WSL, then optionally provisions a local inference runtime before configuring the app-owned gateway to use it. The tray app later owns the runtime lifecycle and displays its connection state. flowchart LR
A[Windows hardware and WSL] --> B[Setup wizard]
B --> C{Local AI opted in and qualified}
C -->|Yes| D[Verified runtime and model]
D --> E[Local llama-server]
E --> F[Managed WSL gateway]
F --> G[Tray status and controls]
C -->|No| H[Normal gateway setup]
Decision needed
Why: The patch establishes a large new core-owned artifact, process, WSL, and provider lifecycle. Tests and proof support the implementation, but core ownership and rollout scope are product decisions. Before merge
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Land only with explicit maintainer sponsorship for the opt-in NVIDIA-only Local AI surface, retaining disabled-by-default configuration, loopback-only routing, and the tested rollback and recovery boundaries. Do we have a high-confidence way to reproduce the issue? Not applicable: this PR adds a new Local AI capability rather than repairing an established failing behavior. Is this the best way to solve the issue? Unclear: the implementation is coherent and proof-backed, but a maintainer must decide whether this large managed native-runtime surface belongs in core as one rollout. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against d3ed6c78bf14. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (16 earlier review cycles; latest 8 shown)
|
jacobtomlinson
added
status: 🚢 actively landing
|
This is an impressive, thoughtfully structured feature, especially the immutable artifact pins, SHA-256 checks, path containment, redirect allowlist, process ownership, and rollback compare-and-swap behavior. I validated the exact head on Windows: the full build and Shared, Tray, and SetupEngine unit suites pass. I also verified the llama.cpp and Hugging Face revisions, filenames, sizes, and hashes against upstream. It is not merge-ready yet. Two independent reviews agree on these blockers:
The proof gap is also material: this adds 9,089 changed production lines and 127 runtime types with zero new focused Before merge, please fix the five blockers, add focused regression coverage, and provide redacted current-head proof of a verified download, first inference, WSL reachability, restart, rollback, and interrupted-install recovery. The direction is excellent. The remaining work is about making this large native execution surface supportable and safe to ship. |
clawsweeper
Bot
added
rating: 🧂 unranked krab
6e8fd0b to
411493c
Compare
|
Current-head E2E behavior proof is now recorded in the PR body for
The PR body also states the remaining proof boundary honestly: this run reused already hash-verified artifacts, and destructive interruption/rollback were covered by focused tests rather than repeated as live destructive demonstrations. |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
Sanitized current-head onboarding and Local AI behavior proof for The recording shows the full maximized setup flow, successful native inference before WSL provisioning, the connected dashboard, and a live Local AI restart returning to |
411493c to
db8fd59
Compare
clawsweeper
Bot
added
proof: 🎥 video
a102f79 to
d241b7c
Compare
clawsweeper
Bot
added
feature: ✨ showcase
Move physical-memory discovery into one shared implementation. Give setup and inference code a consistent RAM capacity value. Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Accept full CUDA offload when either NVML memory movement or the parsed CUDA model buffer proves placement. Reject startup when neither signal accounts for the selected model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
The consent InfoBar content sat flush against the bottom edge of the warning, leaving the message and checkbox visually cramped. Add the same bottom content margin the sandbox InfoBar already uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L8LRNTZ5fBgjGwmtBCrSp3 Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Wait for the direct child process independently of redirected-pipe EOF, then bound the final output drain. Existing-distro inspection now fails closed when WSL times out or returns an unknown error. Show progress during the Welcome-page inspection so a cold WSL service does not make setup look frozen. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
After an elevated WSL install fails, run a best-effort GitHub quota diagnostic and distinguish likely quota exhaustion from other download failures. Diagnostic timeouts never replace the original failure. Offer Store, winget, and elevated PowerShell recovery routes, and allow the platform step to be retried. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
6f8dd82 to
31dffdb
Compare
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
|
Current-head onboarding and Local AI behavior proof for Shows WSL readiness before downloads, successful setup to the connected dashboard, and Local AI Stop/Start/Restart returning to Running and Connected. openclaw-onboarding-timelapse.mp4 |
|
Exact-head WSL install-failure recovery proof for The compiled production SetupEngine diagnostic path was exercised with an induced installer exit code 5. It preserved the original failure, exposed all supported recovery routes, and confirmed the setup step remains retryable: The exact-head fresh path is covered by the attached onboarding video and the green GitHub setup-connect E2E. The recovery proof was induced without uninstalling this machine's working WSL platform. |
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
clawsweeper
Bot
added
proof: sufficient
| }; | ||
| } | ||
|
|
||
| public async Task<LlamaServerRouterProbeResult> ProbeRouterAsync( |
There was a problem hiding this comment.
Why name it "Router"? can we rename to ProbeManagedModelAsync?
There was a problem hiding this comment.
Discussed offline, will address in subsequent PR.
| Uri endpoint, | ||
| string modelAlias, | ||
| CancellationToken cancellationToken = default) | ||
| { |
There was a problem hiding this comment.
would be good to add a comment explaning what the function does
There was a problem hiding this comment.
Discussed offline, will address in subsequent PR.
7 participants
What Problem This Solves
Resolves a problem where Windows users who wanted local inference had no guided,
hardware-aware setup path and could be left with stale runtime, model, process,
port, or gateway state after an interruption. Users whose machines were not
ready for Local AI also lacked a concise explanation of the missing dependency.
The setup flow now qualifies compatible NVIDIA hardware and inspects WSL before
mutation, offers Local AI only as an explicit opt-in, verifies the WSL platform
before large downloads, proves native inference, and then provisions the
app-owned WSL gateway.
Why This Change Was Made
Local AI qualification is based on runtime capability rather than a hard-coded
GPU SKU or CPU pairing. A trusted NVML probe records NVIDIA UUID, driver, CUDA,
total memory, and free memory facts; unambiguous DXGI association can add usable
shared memory. Automatic selection chooses the largest pinned model whose
weights, exact primary FP16 KV cache, and 8 GiB runtime workspace reserve fit
both total and currently free capacity.
After WSL platform readiness, the native llama.cpp runtime and GGUF model are
downloaded from pinned catalog entries, verified, installed under app-owned
paths, started, and checked for a real inference response and GPU placement
before app-owned gateway provisioning. The companion then resolves the single
setup-managed WSL distribution from registry state and owns its router/provider
lifecycle.
The implementation also adds exact-manifest reuse, strict partial GGUF resume,
catalog-owned orphan reconciliation, reparse-safe cleanup, fresh-process
uninstall, cancellation rollback, listener ownership checks, wildcard rejection,
and explicit 127.0.0.1 provider publication with drift-safe compensation.
Boundaries: this change qualifies NVIDIA devices exposed by NVML, not other GPU
vendors; recommends only fitting entries from the pinned model catalog; binds
the managed inference endpoint to local IPv4 loopback; and leaves Windows
session-container inference and additional model catalogs for later work. This
code-focused branch does not add documentation, screenshots, reports, or video.
User Impact
Users with a qualifying NVIDIA GPU can opt into Local AI during onboarding,
choose from models that fit their detected capacity, and see native inference
working before app-owned gateway provisioning. Setup can safely recover from an
interrupted download or install and can uninstall owned state from a fresh
process without removing user-modified configuration.
Users who cannot run Local AI see a small unavailable notice and can open See why
for hardware, driver, CUDA, WSL, and networking diagnostics. The tray exposes
Local AI status and lifecycle controls after setup.
Evidence
Current-head validation for
31dffdb0dbd9eebc8b98ce35cf13ab60762410a9:The isolated Dev onboarding run completed in 143.9 seconds. It reused exact
hash-verified artifacts, generated 32 inference tokens, verified 34/34 GPU
layers and 20,816,330,752 bytes of GPU load growth, provisioned only
OpenClawGateway-Dev, and ended with the dashboard and Local AI both connected.Stop, Start, and Restart returned the managed router to Running/Connected.
The 120.834-second H.264 video is 3270x2180 and has SHA-256
1CCD721CD55A4EDE13E37C4537C29FA4574B1093FA65CAC9C7D7327642D6265E.No older-head runtime value or media is used as proof for this revision.
Change Type
Scope
Validation
GitHub exact-head integration validation:
The local 17-case setup/connect attempt was interrupted by a host reboot before
TRX output. Its exact disposable distro/temp roots were removed and the three
pre-existing distros were verified intact. To avoid stressing the machine again,
the exact-head fresh integration flow was proven through the isolated Dev run
and GitHub's off-host E2E.
Repository check:
Real Behavior Proof
OpenClawGateway-Devdistribution.31dffdb0dbd9eebc8b98ce35cf13ab60762410a9.Security Impact
New permissions or capabilities? Yes
Secrets or tokens handling changed? No
New or changed network calls? Yes
Command or tool execution surface changed? Yes
Data access scope changed? Yes
If any answer is Yes, explain the risk and mitigation:
Setup can download pinned runtime/model artifacts, invoke the native inference
process and scoped WSL commands, read NVIDIA telemetry, and write app-owned
runtime, model, manifest, provider, and consented WSL configuration state.
Downloads require expected size and digest verification before promotion.
NVML loads only from trusted locations. Cleanup validates app-owned paths and
rejects reparse traversal. The router rejects wildcard binds, validates
listener ownership by PID and process start time, and publishes an explicit
127.0.0.1 endpoint. Configuration rollback and uninstall preserve concurrent
or user-drifted state.
Compatibility and Migration
Backward compatible? Yes
Config or environment changes? Yes
Migration needed? No
If yes, list the exact upgrade steps:
No manual upgrade step is required. Legacy Local AI manifest/profile data is
read compatibly. New installations create app-owned manifests and provider
state. Mirrored WSL networking is changed only after explicit user consent,
with byte-preserving restore and rollback behavior.
Review Conversations
These boxes should be completed only after the rewritten head is pushed, CI
finishes, and the current-head proof comment receives re-review.