Repository navigation
Description
Summary
Leash sidecar fails to start in quest pods on Docker Desktop Kubernetes (macOS, Apple Silicon). The cgroup-based control plane isolation cannot be applied because Docker Desktop's container runtime doesn't expose cgroup paths in the format leash expects for iptables -m cgroup --path matching.
Environment
- Host: macOS (Darwin 25.3.0, Apple Silicon)
- Kubernetes: Docker Desktop with Kubernetes enabled
- Leash image:
public.ecr.aws/s5i7k8t3/strongdm/leash:latest - Coder image:
989005111931.dkr.ecr.us-west-1.amazonaws.com/ng-coder:latest(linux/amd64 via Rosetta)
Error
no pod cgroup found (pod UID=a0eb5456-0384-431b-add8-ea3536ca7bfa)
2026/02/26 18:26:26 Warning: adjusted LEASH_PRIVATE_DIR permissions from 777 to 0700
2026/02/26 18:26:26 leashd_mounts public=/leash private=/leash-private
Loaded 1 file open policy rules
Default open policy result: ALLOW (root path '/' is allowed)
Loaded 1 exec policy rules
Default exec policy result: ALLOW (root path '/' is allowed)
2026/02/26 18:26:26 event=ca.generate public_dir=/leash private_dir=/leash-private
Loaded 3 connect IP rules (skipped 0 wildcard, 1 unresolved)
Default connect policy result: DENY (configured override)
2026/02/26 18:26:26 event=policy.restore source=file lsm_open=1 lsm_exec=1 lsm_connect=4 http_rewrites=0
2026/02/26 18:26:26 event=bootstrap.wait path=/leash/bootstrap.ready timeout=2m0s
2026/02/26 18:26:26 event=bootstrap.ready path=/leash/bootstrap.ready source=coder-container cgroup=/ mtime=2026-02-26T18:26:26.140889007Z
leash: applying network interception rules
2026/02/26 18:26:26 event=frontend.start addr=:18080
iptables v1.8.9 (nf_tables): RULE_APPEND failed (Invalid argument): rule in chain OUTPUT
leash: FATAL: could not apply cgroup-based control plane isolation
leash: This security control is required to prevent target container from accessing leashd API
2026/02/26 18:26:26 exit status 1
Root Cause
-
The
discover-cgroupinit container cannot find the pod's cgroup path under/sys/fs/cgroupon Docker Desktop — it writes empty strings to/leash/cgroup-pathand/leash/cgroup-hier. -
The coder container reports its cgroup as
/(from/proc/1/cgroup), which is not a valid cgroup2 hierarchy path for iptables matching. -
Leash then fails when trying to apply
iptables -m cgroup --path /(or similar) —RULE_APPEND failed (Invalid argument).
Expected Behavior
Leash should either:
- Gracefully degrade when cgroup-based isolation is unavailable (e.g., skip iptables isolation with a warning), or
- Support an alternative isolation mechanism for environments where cgroup paths aren't discoverable (Docker Desktop, kind, minikube, etc.)
Reproduction
- Set up ai-agent-local on Docker Desktop Kubernetes
- Send a message that triggers a quest with SDM access
- Observe the quest pod's leash container crash
Workaround
For local development, removing the leash sidecar from the pod spec in ai-agent-brain/internal/k8s/pod.go allows quest pods to run (without network isolation).
Activity
sunkadshreyas commentedon Feb 26, 2026
Root Cause Analysis (Updated)
After deeper investigation, there are three distinct bugs causing leash to fail on Docker Desktop Kubernetes:
Bug 1: nftables comment quoting (affects ALL environments)
The
ensure_rulefunction inapply-nftables.shpasses comment strings likeleash:return-mitmwithout nft-level quoting. The:character is a special token in nft's parser, causing all nftables rules to fail — not just the cgroup rule.Fix:
comment "$comment"→comment "\"$comment\""inensure_rule().Bug 2: nftables cgroup path quoting
The
socket cgroupv2 level 1 $TARGET_CGROUPrule passes the cgroup path without nft-level quotes. Paths starting with/cause a syntax error.Fix:
"$TARGET_CGROUP"→"\"$TARGET_CGROUP\""in the cgroup rule.Bug 3: Docker Desktop kernel lacks cgroup matching support
Docker Desktop's LinuxKit kernel (6.10.14-linuxkit) does not support:
iptables -m cgroup --path(xt_cgroup module)nftables socket cgroupv2 level 1(nft socket match)Both fail with "No such file or directory" / "Invalid argument". The kernel simply doesn't have these features compiled in.
Fix: Add
LEASH_CGROUP_ISOLATIONenv var (requiredby default,optionalfor Docker Desktop) so leash can start without cgroup-based control plane isolation where the kernel doesn't support it.Additional finding: nftables not installed in runtime image
The
Dockerfile.leashruntime-base stage installsiptablesbut notnftables. Since Debian bookworm's iptables uses the nf_tables backend, thenftbinary is needed for the preferred nftables code path.Fix: Add
nftablespackage to runtime-base in Dockerfile.leash.Fix
PR incoming.
Fix nftables comment quoting and add cgroup isolation toggle
navanchauhan commentedon Mar 4, 2026
Fixed by #66 — leash now falls back to a blanket port block when cgroup-based iptables matching is unavailable (same root cause as #60: Docker Desktop LinuxKit kernels lack xt_cgroup support).