Nightly E2E: triage & fix plan for 3 consistent failures on main #2104

The nightly E2E workflow has been failing on main every night for the last 5+ consecutive runs. Auto-created notifier #1992 reports the event; this issue is the fix plan.

Failing jobs

1. cloud-experimental-e2e — 04-landlock-readonly.sh checks 1–3 fail: /sandbox, .bashrc, .profile are writable. Root cause tracked in [All Platform][Security]OpenShell 0.0.26 does not enforce Landlock filesystem policy — /sandbox writable on all platforms #1739 (OpenShell 0.0.26 does not apply Landlock ruleset to sandbox process). Policy fix in progress upstream — out of scope for this issue, will clear once the OpenShell version pin in nemoclaw-blueprint/blueprint.yaml and scripts/install-openshell.sh is bumped to the fixed release.

2. token-rotation-e2e — hits the 15min job timeout (exit 124) during Phase 2 re-onboard. Silent hang — output is buffered in a $(…) capture with no per-step timeout, so the real error isn't visible.

3. snapshot-commands-e2e — Phase 7 snapshot restore fails due to an ambiguous timestamp prefix-match when two snapshots are created within the same second.

Fix plan (in scope for this issue)

• Snapshot: add --format json to nemoclaw <sandbox> snapshot list, use exact-ID match in the test, and fix the silent-exit pattern so future failures aren't invisible.
• Token-rotation: add streaming + bounded timeout around the Phase 2 onboard call so the real error surfaces. Quarantine the job with continue-on-error: true while we diagnose, then remove once the root cause is fixed.

Landlock work is handed off — see #1739.

Links: #1992 (noise tracker) · #1739 (Landlock regression, upstream) · workflow run 24642489117