Repository navigation
[SECURITY] No per-task lifecycle tracking or graceful abort — stuck tasks require sandbox destruction #801
Description
Problem Statement
NemoClaw tracks state at the sandbox level (nemoclaw.json with lastRunId, lastAction), but has no per-task lifecycle tracking. There is no way to:
- Know if a specific task is running, completed, failed, or timed out
- Abort a specific task without destroying the entire sandbox
- Detect orphaned tasks after a crash
- Resume or recover from a failed task
The only "abort" mechanism is action_rollback(), which stops and removes the entire sandbox.
Impact
- No graceful degradation: A stuck task requires full sandbox destruction, losing all state
- No observability: Operators cannot monitor individual task progress
- No crash recovery: If the process crashes mid-task, the task remains in an unknown state
- No external control: External systems cannot abort a specific agent action
Proposed Design
Implement a per-task state machine:
queued → running → completed
→ failed
→ timed_out
→ aborted
With:
- State persistence: Store task state with TTL (Redis, SQLite, or filesystem)
- Abort channel: Allow external callers to abort specific tasks without destroying the sandbox
- Crash recovery: On startup, scan for tasks left in
runningstate and mark asfailed - Timeout enforcement: Configurable per-task timeout with automatic state transition
References
- NeMo Agent Toolkit issue OpenAI/ChatGPT OAuth is not a viable NemoClaw provider path without manual host-side OpenClaw gateway bridging #1793 proposes this pattern for NAT workflows
- Standard practice in distributed task schedulers (Celery, Airflow, Kubernetes Jobs)
Alternatives Considered
No response
Category
enhancement: feature
Checklist
- I searched existing issues and this is not a duplicate
- This is a design proposal, not a "please build this" request
Activity
wscurran commentedon Mar 25, 2026
✨ Thanks for submitting this suggestion with a detailed explanation, it addresses a security concern that could improve the reliability and security of NemoClaw, which could make it more robust.
Possibly related open PRs:
#871 security: enforce process limit to prevent fork bombs in sandbox,
#699 fix(security): require explicit opt-in for Telegram bridge open chat access,
#611 security: remove telegram and discord from baseline sandbox policy
liuyingxuvka commentedon Apr 30, 2026
I have run into similar lifecycle problems where the system only has coarse environment-level state, so a stuck task eventually forces a full teardown. I built a small free and open-source tool called Flow-Guard to model task/workflow lifecycles before implementation.
Flow-Guard uses a finite-state mathematical model. It is not prompt-only and not an LLM wrapper; the AI agent is mainly a helper for installing it and running the first model.
For this design, I would model:
The counterexamples I would look for are a task stuck forever, an abort that leaves work still running, a crash that leaves the task in unknown state, or recovery that duplicates side effects instead of resuming/marking failure cleanly.
Repo: https://github.com/liuyingxuvka/FlowGuard
Feedback: liuyingxuvka/FlowGuard#1
This seems like a good candidate for a small design-time model before the task lifecycle is wired into storage and control APIs.
ericksoa commentedon May 1, 2026
Thanks for filing this and for spelling out the lifecycle states and failure modes.
After checking this against the current architecture, we are closing this as not planned for NemoClaw. Per-task lifecycle tracking, task-specific abort, timeout state transitions, and task resume/recovery are agent/runtime concerns rather than NemoClaw orchestration concerns. NemoClaw creates, configures, rebuilds, and connects to sandboxes; it does not own execution of individual agent tasks inside OpenClaw, Hermes, NAT, or future agents.
NemoClaw has since added lifecycle tracking for the work it does own: onboarding/rebuild session state, step-level progress, resume, and failure handling. Relevant PRs:
onboard --resume.If an upstream runtime later exposes a stable task/job lifecycle API, NemoClaw can surface that state and provide UX around it. It should not invent a parallel task scheduler for work it does not execute.
h-network commentedon May 1, 2026
Thank you for the response.
A clarification on the architectural framing: PR #2326 (one of the four cited) adds the NEMOCLAW_AGENT_TIMEOUT build arg, which bakes into agents.defaults.timeoutSeconds in openclaw.json. That is configurable per-agent-task inference timeout, set by NemoClaw at build time, controlling agent runtime behavior. The agents.defaults namespace is for agent task configuration, not onboarding session configuration.
Configurable per-task timeout was one of the explicit proposals in this issue ("Configurable per-task timeout with automatic state transition"). PR #2326 implements that primitive in NemoClaw — under a different name and at build time rather than runtime, but in the same configuration surface that controls agent execution.
The closure rationale states that "timeout state transitions... are agent/runtime concerns rather than NemoClaw orchestration concerns." The cited PR indicates that at minimum the timeout-configuration piece of agent task lifecycle is owned by NemoClaw via agents.defaults.
For the record, since this issue is being closed as wont-fix and not planned.