Repository navigation
nemoclaw onboard fails on Ubuntu 24.04/Zorin OS 18: k3s cannot pull images from Docker Hub (nested container networking) #455
Description
Description
Disclaimer: I don't really know what is going on, this is an AI's assessment of my issues:
Environment:
OS: Zorin OS 18 (Ubuntu 24.04 Noble base)
Docker: 29.3.0
OpenShell: 0.0.10
NemoClaw: latest main
GPU: None (cloud inference)
What happened:
nemoclaw onboard always fails at the "Initializing environment" step with K8s namespace not ready. The k3s instance inside the OpenShell gateway container cannot pull images from Docker Hub or registry.k8s.io — all requests time out with i/o timeout.
Root cause identified:
The k3s containerd inside the gateway container cannot reach external registries. The following images fail to pull:
rancher/mirrored-pause:3.6
rancher/klipper-helm:v0.9.14-build20260210
rancher/local-path-provisioner:v0.0.34
registry.k8s.io/agent-sandbox/agent-sandbox-controller:v0.1.0
Manually importing these via docker save | ctr images import fixes the issue temporarily, but onboard.js always calls openshell gateway destroy before starting, wiping the imported images every run.
Workaround attempted:
Pre-seeding images into k3s containerd works IF done fast enough after gateway start, but the onboard.js code destroys and recreates the gateway every run making this impossible to automate reliably.
Suggested fix:
Either (1) bundle required images inside the gateway container image, or (2) add a pre-pull hook that seeds images before k3s initialization begins, or (3) don't unconditionally destroy the gateway if it's already healthy.
Reproduction Steps
Steps to Reproduce:
Fresh install of Zorin OS 18 (Ubuntu 24.04 base) with no GPU
Install Docker Engine following official Ubuntu instructions
Clone NemoClaw and run installer:
git clone https://github.com/NVIDIA/NemoClaw.git
cd NemoClaw
./install.sh
Install OpenShell:
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh
Run nemoclaw setup-spark to fix Ubuntu 24.04 cgroup v2 issues (required or Docker networking breaks)
Run nemoclaw onboard
Expected result:
Onboarding completes successfully through all 7 steps.
Actual result:
Fails at step 2/7 "Initializing environment" with:
× K8s namespace not ready
╰─▶ timed out waiting for namespace 'openshell' to exist
To confirm root cause, run after gateway container starts:
docker logs openshell-cluster-nemoclaw 2>&1 | grep "i/o timeout"
You will see repeated failures pulling rancher/mirrored-pause:3.6 from registry-1.docker.io with TCP timeout to 174.129.222.113:443, even though the container itself has internet access (ping 8.8.8.8 works).
Environment
OS: Zorin OS 18 (Ubuntu 24.04 Noble base)
Docker: 29.3.0
OpenShell: 0.0.10
NemoClaw: latest main
GPU: None (cloud inference)
Logs
Checklist
- I confirmed this bug is reproducible
- I searched existing issues and this is not a duplicate
Activity
wscurran commentedon Mar 25, 2026
✨ Thanks for submitting this issue about the onboard process failure on Ubuntu 24.04/Zorin OS 18, I'll investigate the k3s and Docker Hub issue to understand what's causing the problem with the K8s and OpenShell setup.
Possibly related open PRs:
wscurran commentedon Apr 13, 2026
The gateway-destroy-on-retry behavior that made the image pre-seeding workaround impossible was addressed in #961 (onboard resiliency hardening, merged March 27). The nested container networking timeout from Docker Hub is an environment-specific issue that may or may not persist on current versions — OpenShell has moved significantly past 0.0.10. If you're still hitting k3s image pull timeouts on the current version, please reopen with
docker logs openshell-cluster-nemoclawoutput and we'll investigate the specific registry routing.