[mashkovd]

Summary

• After PR fix(admins-openclaw): scrub stale runtime-mirror lock on every container start #96 fixed the runtime-mirror lock wedge, a different failure surfaced: bundled-runtime install (~30 extensions, 3-5s npm install each) blocks the Node event loop long enough for /healthz and /readyz to time out at 5s/3s.
• admins-openclaw 2026-05-02: container ran 28 min serving live WS traffic (device.pair, mctl.connect, codex.connect) before SIGKILL (exitCode 137) when 5 consecutive liveness probes failed.
• Bumps tolerance: readiness 30s → 50s, liveness 150s → 5 min.

Trade-off

A truly hung pod takes 5 min instead of 2.5 min to be killed. Acceptable for single-replica tenant openclaw where ms-level recovery isn't needed.

Test plan

• ArgoCD sync clean
• Pod stays Running 3/3 for 30+ min (cold-start plugin-install completes without probe-kill)
• Subsequent restarts within the same pod stay clean (wrapper from PR fix(admins-openclaw): scrub stale runtime-mirror lock on every container start #96 still clearing locks)

[mashkovd]

@codex review

[mashkovd]

@copilot review

[Copilot]

@copilot review

Code Review — fix(admins-openclaw): tolerate Node event-loop stalls in probes

One file changed: platform-gitops/services/admins/openclaw/values.yaml

---

✅ What's correct

Root-cause diagnosis is sound.
The startup probe (budget 5 + 90×5 = 455 s) only gates liveness/readiness during initial bring-up. Once the startup probe passes, liveness and readiness own ongoing health. Plugin install (~30 extensions × 3–5 s/each = 90–150 s) falls after startup-probe graduation, so the 5-min initial-delay on the startup probe doesn't help here.

Budget arithmetic is correct.

Probe	Before	After	Tolerance
Readiness	timeout=3, threshold=3	timeout=10, threshold=5	30 s → 50 s
Liveness	timeout=5, threshold=5	timeout=15, threshold=10	150 s → 300 s

periodSeconds (10 / 30) is unchanged, so wall-clock tolerances match the commit message.

Readiness comment is clear and specific (cites beta.2, exact extension count observation, outcome).

---

⚠️ Observations / minor notes

1. timeoutSeconds: 15 is half of periodSeconds: 30 for liveness. Each probe attempt may block the kubelet connection for up to 15 s before being counted as a failure. That's acceptable given the trade-off, but it means under sustained stalls the pod will appear slower to kubelet than a non-blocking endpoint would. No action required — just worth keeping in mind if you tune periodSeconds down in the future.

2. initialDelaySeconds: 60 on liveness doesn't cover the full install window. With 30 extensions at 3–5 s each, install can run 90–150 s, which exceeds the 60 s initial delay. The failureThreshold=10 / timeout=15 budget (300 s of tolerance) does cover it, but any future increase in extension count could erode this. The real fix is making /healthz non-blocking during install (off-thread async check), which would eliminate the need for these wide budgets entirely.

3. Rollout template gap (pre-existing, unrelated to this PR). templates/rollout.yaml doesn't render timeoutSeconds or failureThreshold for liveness/readiness probes (lines 76–90), so those values would silently fall back to k8s defaults (1 s / 3) for any Argo Rollout-based deployment. Not introduced by this PR — strategy.type: Recreate means this service uses the Deployment template — but worth a follow-up issue.

---

Verdict

The change is correct and safe for the declared constraints (single-replica tenant, no ms-level recovery requirement). The trade-off (5 min kill time for a truly hung pod) is explicitly documented in both the commit message and inline comments. No blockers.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".