fix(cli): recover live sandboxes in list when local registry is empty - #5786
Merged
Merged
Conversation
`nemoclaw list` printed "No sandboxes registered" while the live gateway and container were healthy and `nemoclaw <name> status` reported Ready, because the list recovery path required a seed (existing registry entry, onboard session, or requested name) before probing the gateway. After a local registry loss with no seed, it trusted the empty registry. Widen recovery so an empty registry always attempts a bounded, read-only live-gateway inspection — no gateway select/start — whenever OpenShell is connected to a NemoClaw-managed gateway (`nemoclaw` or `nemoclaw-<port>`), never a foreign one. Recovered sandboxes are surfaced display-only and are NOT persisted: the global `openshell sandbox list` exposes only NAME/CREATED/PHASE, so a persisted entry would default agent to "openclaw" everywhere downstream and permanently misclassify a Deep Agents/Hermes sandbox. The renderer shows agent/model/provider/GPU as "unknown" for these rows rather than inventing OpenClaw/CPU defaults. Gateway probes are non-fatal (ignoreProbeErrors) so a hung gateway falls back to the empty registry, and recovery is gated on a clean (status 0) `sandbox list` so error text is never parsed as sandbox names. Signed-off-by: Yimo Jiang <yimoj@nvidia.com>
Add TSDoc summaries to the functions introduced/changed for #5714 (registry recovery, gateway lifecycle classification, OpenShell capture, and inventory row projection) to satisfy the CodeRabbit docstring-coverage pre-merge check. No behavior change. Signed-off-by: Yimo Jiang <yimoj@nvidia.com>
Add TSDoc to the OpenShell runtime adapter helpers and the gateway status-parsing helpers in the files touched by #5714 to clear the CodeRabbit docstring-coverage threshold (80%). No behavior change. Signed-off-by: Yimo Jiang <yimoj@nvidia.com>
6 tasks done
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in the Show a code coverage summary of the most covered files.
TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most covered files.
Updated |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThe PR adds live gateway recovery for empty registry listings, preserves gateway-recovered markers outside disk storage, and updates inventory output to show unknown agent, GPU, and phase values for recovered sandboxes. ChangesGateway recovery and inventory display
Sequence Diagram(s)sequenceDiagram
participant recoverRegistryEntries
participant recoverRegistryFromLiveGateway
participant getNamedGatewayLifecycleState
participant captureOpenshell
participant parseLiveSandboxEntries
recoverRegistryEntries->>recoverRegistryFromLiveGateway: readOnly: !hasRecoverySeed
recoverRegistryFromLiveGateway->>getNamedGatewayLifecycleState: ignoreProbeErrors: true
getNamedGatewayLifecycleState->>captureOpenshell: includeStderr: true
captureOpenshell-->>getNamedGatewayLifecycleState: stderr with Status:/Gateway:
getNamedGatewayLifecycleState-->>recoverRegistryFromLiveGateway: lifecycle state
recoverRegistryFromLiveGateway->>parseLiveSandboxEntries: sandbox list output
parseLiveSandboxEntries-->>recoverRegistryFromLiveGateway: {name, phase}[]
recoverRegistryFromLiveGateway-->>recoverRegistryEntries: recoveredFromGateway + ephemeralSandboxes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Contributor
Vitest E2E Scenario RecommendationRequired Vitest E2E scenarios: Dispatch required Vitest E2E scenarios:
Full Vitest E2E advisor summaryVitest E2E Scenario AdvisorBase: Required Vitest E2E scenarios
Optional Vitest E2E scenarios
Relevant changed files
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/lib/gateway-runtime-action.test.ts (1)
81-103: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAssert the stderr-capture contract in these probe tests.
These cases only verify
ignoreError. The production path ingetNamedGatewayLifecycleState()also depends onincludeStderrstaying in lockstep withignoreProbeErrors; otherwise the lifecycle parser loses theStatus:/Gateway:lines and these tests would still pass.Suggested assertion update
it("keeps probes fatal (no ignoreError) by default", () => { captureSpy.mockReturnValue({ status: 0, output: "Status: Connected\nGateway: nemoclaw\n" }); gatewayRuntime.getNamedGatewayLifecycleState("nemoclaw"); for (const [, opts] of captureSpy.mock.calls) { expect(opts?.ignoreError).not.toBe(true); + expect(opts?.includeStderr).not.toBe(true); } }); it("makes probes non-fatal when ignoreProbeErrors is set (`#5714`)", () => { @@ expect(captureSpy.mock.calls.length).toBeGreaterThanOrEqual(2); for (const [, opts] of captureSpy.mock.calls) { expect(opts?.ignoreError).toBe(true); + expect(opts?.includeStderr).toBe(true); } });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/gateway-runtime-action.test.ts` around lines 81 - 103, Update the probe tests for getNamedGatewayLifecycleState so they also assert the stderr-capture behavior, not just ignoreError. In the default case, verify the capture calls keep includeStderr enabled alongside the existing no-ignoreError expectation, and in the ignoreProbeErrors path verify every capture call has both ignoreError true and includeStderr true. Use the existing gatewayRuntime.getNamedGatewayLifecycleState and captureSpy assertions to keep the probe contract locked in.src/lib/registry-recovery-action.test.ts (1)
245-261: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAssert the transient recovery marker on the returned sandbox.
This test proves the sandbox is surfaced, but it does not verify that
recoverRegistryEntries()setsrecoveredFromGateway: true. That marker is what drives the downstream inventory renderer away from theopenclaw/CPU sandboxdefaults, and the current inventory tests synthesize it manually.Suggested assertion update
const result = await recoverRegistryEntries(); expect(result.recoveredFromGateway).toBe(1); const recovered = result.sandboxes.find((s) => s.name === "dcode-station"); expect(recovered).toBeDefined(); + expect(recovered?.recoveredFromGateway).toBe(true); // Minimal safe entry: no invented agent/model/provider metadata. expect(recovered?.model).toBeNull(); expect(recovered?.provider).toBeNull(); expect(recovered?.agent).toBeUndefined();🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/registry-recovery-action.test.ts` around lines 245 - 261, The `recoverRegistryEntries()` test covers rediscovery of the live sandbox, but it does not assert the transient recovery marker that downstream rendering depends on. Update the test around `recoverRegistryEntries`, `result.sandboxes`, and the recovered `"dcode-station"` entry to verify the returned sandbox is marked as recovered from the gateway (for example, by asserting the recovery flag is true on that sandbox or its equivalent field in the recovered entry). Keep the existing assertions for minimal metadata and add the missing check using the same symbols so the behavior is validated end to end.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/lib/gateway-runtime-action.test.ts`:
- Around line 81-103: Update the probe tests for getNamedGatewayLifecycleState
so they also assert the stderr-capture behavior, not just ignoreError. In the
default case, verify the capture calls keep includeStderr enabled alongside the
existing no-ignoreError expectation, and in the ignoreProbeErrors path verify
every capture call has both ignoreError true and includeStderr true. Use the
existing gatewayRuntime.getNamedGatewayLifecycleState and captureSpy assertions
to keep the probe contract locked in.
In `@src/lib/registry-recovery-action.test.ts`:
- Around line 245-261: The `recoverRegistryEntries()` test covers rediscovery of
the live sandbox, but it does not assert the transient recovery marker that
downstream rendering depends on. Update the test around
`recoverRegistryEntries`, `result.sandboxes`, and the recovered
`"dcode-station"` entry to verify the returned sandbox is marked as recovered
from the gateway (for example, by asserting the recovery flag is true on that
sandbox or its equivalent field in the recovered entry). Keep the existing
assertions for minimal metadata and add the missing check using the same symbols
so the behavior is validated end to end.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 103a3188-3c48-450a-a244-62363abf922b
📒 Files selected for processing (8)
src/lib/adapters/openshell/runtime.tssrc/lib/gateway-runtime-action.test.tssrc/lib/gateway-runtime-action.tssrc/lib/inventory/index.test.tssrc/lib/inventory/index.tssrc/lib/registry-recovery-action.test.tssrc/lib/registry-recovery-action.tssrc/lib/state/registry.ts
Contributor
PR Review Advisor — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
Review findings by urgency: 0 required fixes, 1 item to resolve/justify, 0 in-scope improvements
|
Address PR review feedback on #5714 registry recovery: - PRA-2: an incomplete (phantom) onboard session no longer counts as a recovery seed. `hasRecoverySeed` now requires a *confirmed* session (isSessionSandboxConfirmed && sandboxName), so an empty registry plus a phantom session stays in the read-only/display-only path instead of the mutating, persisting seeded path. - Restrict unseeded read-only recovery to `healthy_named` only (drop the `connected_other` branch): recover solely from the gateway this process resolves/targets, so `nemoclaw list` and a follow-up `nemoclaw <name> status` stay consistent and never advertise a sandbox the next command would resolve to a different gateway. Tests: phantom-session stays read-only/non-persisted; connected_other no longer recovers; assert the includeStderr<->ignoreProbeErrors lockstep in the probe tests; assert the recoveredFromGateway marker on recovered rows. Signed-off-by: Yimo Jiang <yimoj@nvidia.com>
Collaborator
Author
Review feedback addressed (head
|
5 tasks
wscurran
added
area: cli
Contributor
Resolve OpenShell runtime adapter overlap: keep both the upstream includeStreams option and this branch's includeStderr option on RunnerOptions/captureOpenshell. Signed-off-by: Yimo Jiang <yimoj@nvidia.com> # Conflicts: # src/lib/adapters/openshell/runtime.ts
…type) Address PR #5786 PR Review Advisor round 2 for #5714: - PRA-3 (Required): carry the trusted live PHASE from `openshell sandbox list` into recovered list rows so `list` shows e.g. Ready, consistent with `nemoclaw <name> status`. Agent stays "unknown" (documented): the gateway list is not an authoritative agent source; the real agent is reconciled by the follow-up named command. New phase-layout-robust parser `parseLiveSandboxEntries` uses a broadened phase vocabulary so terminal/transient phases (Failed, CrashLoopBackOff, Creating, …) are preserved, not just Ready/Running. - PRA-4: keep the transient display markers (`recoveredFromGateway`, `livePhase`) out of the durable SandboxEntry type — use a narrow return-only `RecoveredSandboxEntry` for ephemeral rows and strip both keys in serializeSandboxEntryForDisk so a force-passed marker can never reach sandboxes.json. - PRA-5 (and PRA-2): apply isSessionSandboxConfirmed in shouldRecoverRegistryEntries too, so an incomplete (phantom) session with existing registry entries no longer flips recovery into the mutating gateway path. PRA-1 is resolved in code by the PRA-4 narrow type. Tests: live-phase parsing across column layouts and terminal phases; recovered row renders phase + unknown agent/GPU; phantom session with existing registry does not trigger mutating recovery; serialization strips the transient markers. Signed-off-by: Yimo Jiang <yimoj@nvidia.com>
Collaborator
Author
PR Review Advisor round 2 — addressed (head
|
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/lib/runtime-recovery.test.ts (1)
55-58: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd the protobuf-schema-mismatch assertion here too.
parseLiveSandboxEntries()sharesisNonSandboxRow()with the other recovery helpers, but this new suite only checks genericError:text. A dedicated assertion for the schema-mismatch sentinel would lock in the “don’t invent fake recovered sandboxes from decode errors” behavior.➕ Suggested test addition
it("skips headers and error lines when parsing live entries", () => { expect(parseLiveSandboxEntries("No sandboxes found.")).toEqual([]); expect(parseLiveSandboxEntries("Error: boom")).toEqual([]); + expect( + parseLiveSandboxEntries( + 'Error: × status: Internal, message: "Sandbox.metadata: SandboxResponse.sandbox: invalid wire type value: 6"', + ), + ).toEqual([]); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/runtime-recovery.test.ts` around lines 55 - 58, Add a dedicated assertion in the parseLiveSandboxEntries test to cover the protobuf-schema-mismatch sentinel alongside the existing header and Error: cases. Update the runtime-recovery test around parseLiveSandboxEntries and its shared isNonSandboxRow path so it explicitly expects an empty result for the schema-mismatch decode-error text, locking in that recovery helpers do not fabricate sandbox entries from protobuf decode failures.src/lib/registry-recovery-action.ts (1)
104-128: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExtract the recovery-seed rules into one shared helper.
shouldRecoverRegistryEntries()now embeds the same confirmed-session / recovery-seed semantics thatrecoverRegistryEntries()also recomputes later. Keeping those checks separate makes it easy for the probe gate and the read-only-vs-seeded path to drift apart again on a future edit.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/registry-recovery-action.ts` around lines 104 - 128, Extract the confirmed-session and recovery-seed decision logic into a shared helper so both shouldRecoverRegistryEntries and recoverRegistryEntries use the exact same rules. Centralize the checks for isSessionSandboxConfirmed, sessionSandboxName, requestedSandboxName, hasRecoverySeed, and the empty-registry case in a single function, then call it from both paths to avoid drift between the probe gate and the read-only/seeded recovery flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/lib/registry-recovery-action.ts`:
- Around line 104-128: Extract the confirmed-session and recovery-seed decision
logic into a shared helper so both shouldRecoverRegistryEntries and
recoverRegistryEntries use the exact same rules. Centralize the checks for
isSessionSandboxConfirmed, sessionSandboxName, requestedSandboxName,
hasRecoverySeed, and the empty-registry case in a single function, then call it
from both paths to avoid drift between the probe gate and the read-only/seeded
recovery flow.
In `@src/lib/runtime-recovery.test.ts`:
- Around line 55-58: Add a dedicated assertion in the parseLiveSandboxEntries
test to cover the protobuf-schema-mismatch sentinel alongside the existing
header and Error: cases. Update the runtime-recovery test around
parseLiveSandboxEntries and its shared isNonSandboxRow path so it explicitly
expects an empty result for the schema-mismatch decode-error text, locking in
that recovery helpers do not fabricate sandbox entries from protobuf decode
failures.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: b2afe2e6-b464-4c53-a72b-c4278a17f442
📒 Files selected for processing (8)
src/lib/inventory/index.test.tssrc/lib/inventory/index.tssrc/lib/registry-recovery-action.test.tssrc/lib/registry-recovery-action.tssrc/lib/runtime-recovery.test.tssrc/lib/runtime-recovery.tssrc/lib/state/registry.tstest/registry.test.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- src/lib/inventory/index.test.ts
- src/lib/inventory/index.ts
- src/lib/registry-recovery-action.test.ts
Restore CodeRabbit docstring coverage above the 80% threshold after the #5714 round-2 changes brought runtime-recovery.ts into the diff: add TSDoc to the sandbox-list parsing helpers and serializeSandboxEntryForDisk. No behavior change. Signed-off-by: Yimo Jiang <yimoj@nvidia.com>
Collaborator
Author
Maintainer scope acceptance — recovered-row agent is
|
Collaborator
Author
|
@coderabbitai resume |
Contributor
✅ Action performedReviews resumed. |
cv
approved these changes
Jun 26, 2026
This was referenced Jun 26, 2026
8 of 21 tasks
cv
pushed a commit
that referenced
this pull request
Jun 29, 2026
## Summary Adds the v0.0.69 release notes to the published release-notes page so users can see the shipped sandbox recovery, Deep Agents Code, Hermes, inference, policy, and release-validation changes. The section is based on the v0.0.69 announcement and links each user-facing theme to the deeper docs pages that already cover the behavior. ## Changes - Added a new `v0.0.69` section to `docs/about/release-notes.mdx`. - Linked release-note themes to lifecycle, backup, troubleshooting, Deep Agents Code, commands, workspace, messaging, Hermes, inference, security, monitoring, and network-policy docs. Source summary: - #5455 -> `docs/about/release-notes.mdx`: Summarized persistent workspace and state cleanup during sandbox destroy. - #5738 -> `docs/about/release-notes.mdx`: Summarized nonzero exit status preservation for failed hosted endpoint validation. - #5786 -> `docs/about/release-notes.mdx`: Summarized live sandbox rediscovery when local registry state is missing. - #5881 -> `docs/about/release-notes.mdx`: Summarized the `nemo-deepagents` alias command surface. - #5594 -> `docs/about/release-notes.mdx`: Summarized the Hermes Agent 2026.6.19 update. - #5777 -> `docs/about/release-notes.mdx`: Summarized manifest-derived messaging channel support. - #5825 -> `docs/about/release-notes.mdx`: Summarized DeepSeek V4 Flash managed-vLLM defaults for DGX Station. - #5877 -> `docs/about/release-notes.mdx`: Summarized provider switch metadata preservation. - #5932 -> `docs/about/release-notes.mdx`: Summarized transient inference smoke retry behavior. - #5934 -> `docs/about/release-notes.mdx`: Summarized constrained inference smoke retry boundaries. - #5681 -> `docs/about/release-notes.mdx`: Summarized Shields config-hash sealing during auto-restore. - #5682 -> `docs/about/release-notes.mdx`: Summarized sandbox connect process-limit enforcement. - #5683 -> `docs/about/release-notes.mdx`: Summarized JSON agent failure provenance warnings. - #5711 -> `docs/about/release-notes.mdx`: Summarized sparse-source log breadcrumbs. - #5838 -> `docs/about/release-notes.mdx`: Summarized host-authoritative Shields status. - #5880 -> `docs/about/release-notes.mdx`: Summarized policy round-trip documentation updates. - #5886 -> `docs/about/release-notes.mdx`: Summarized network request approval-flow documentation updates. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: doc-only release-notes prose; no runtime behavior changed. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [ ] Targeted tests pass for changed behavior - [ ] Full `npm test` passes (broad runtime changes only) - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) `npm run docs` passed with 0 errors and the existing Fern light-mode accent contrast warning. `fern check --warnings` reported the same accent-color warning. --- Signed-off-by: Miyoung Choi <miyoungc@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added release notes for **v0.0.69**, covering improved sandbox lifecycle recovery (state preservation across destroy/recreate/rebuild/recovery/validation failures), clearer Deep Agents Code terminal/CLI behavior, and safer Hermes messaging/provider switching with manifest-driven channels. * Improved inference setup validation guidance, including handling of local/compatible endpoints and redaction of sensitive validation errors. * Refreshed release-gate documentation with clearer approval examples and validation behavior for NVIDIA API keys vs hosted inference keys. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
…NVIDIA#5786) ## Summary `nemoclaw list` printed "No sandboxes registered" while the live OpenShell gateway and container were healthy and `nemoclaw <name> status` reported `Phase: Ready` (reported on DGX Station with a Deep Agents sandbox). The `list` recovery path required a *seed* — an existing registry entry, an onboard session, or an explicit requested name — before it would probe the gateway. After a local registry loss with no seed, it trusted the empty registry and reported nothing, even though the named-status path could find and reconcile the same sandbox. ## Related Issue Fixes NVIDIA#5714 ## Changes - **Widen list recovery for an empty registry.** `recoverRegistryEntries` now attempts recovery whenever the local registry is empty, even with no session/requested-name seed. - **Bounded, read-only, non-mutating gateway inspection for the unseeded case.** Plain `nemoclaw list` never selects or starts a gateway. It inspects the live sandbox list only when OpenShell is connected to a **NemoClaw-managed** gateway (`nemoclaw` or a per-port `nemoclaw-<port>`) — never a foreign OpenShell gateway. Gateway probes are non-fatal (`ignoreProbeErrors`), so a hung/timed-out gateway falls back to the empty registry instead of exiting the process, and recovery is gated on a clean (`status === 0`) `sandbox list` so error text is never parsed as a sandbox name. - **Display-only recovery — recovered entries are NOT persisted.** The global `openshell sandbox list` exposes only NAME/CREATED/PHASE, not the agent or gateway binding. Persisting a recovered entry would default `agent` to `openclaw` everywhere downstream (state dirs, connect, rebuild, doctor) and permanently misclassify a Deep Agents/Hermes sandbox. Recovered sandboxes are surfaced for the current `list` only; follow-up named commands reconcile the real agent via the gateway. - **Honest rendering.** Recovered rows show `agent`/`model`/`provider`/`GPU` as `unknown` rather than inventing OpenClaw/CPU defaults. - `getNamedGatewayLifecycleState` gains an opt-in `ignoreProbeErrors` (default keeps existing fatal behavior); `captureOpenshell` forwards `includeStderr` so non-fatal probes still capture the gateway status text. ## Type of Change - [x] Code change (feature, bug fix, or refactor) ## Verification Verified end-to-end through the real worktree CLI (`node ./bin/nemoclaw.js …`) against a live OpenShell gateway. The local registry was emptied and the onboard session removed to simulate the reporter's registry loss; the on-disk registry is checked after each run. **1. Pre-fix (reporter mismatch) — `list` blind while a Ready sandbox is live:** ```console $ node ./bin/nemoclaw.js list No sandboxes registered. Run `nemoclaw onboard` to get started. $ node ./bin/nemoclaw.js sbox-4778 status Sandbox: sbox-4778 Model: nvidia/nemotron-3-super-120b-a12b Provider: nvidia-prod ... Phase: Ready # docker ps -a → openshell-sbox-4778-… Up (container healthy, gateway Connected) ``` **2. Post-fix — `list` rediscovers the live sandboxes (display-only, registry stays empty):** ```console $ node ./bin/nemoclaw.js list Recovered 5 sandbox entries from the live OpenShell gateway. Sandboxes: probe3014x agent: unknown model: unknown provider: unknown GPU: unknown policies: none e2e-5468-ollama agent: unknown model: unknown provider: unknown GPU: unknown policies: none issue4538-fix agent: unknown model: unknown provider: unknown GPU: unknown policies: none sbox-4778 agent: unknown model: unknown provider: unknown GPU: unknown policies: none dcode5744 agent: unknown model: unknown provider: unknown GPU: unknown policies: none $ node -e 'console.log(Object.keys(require(process.env.HOME+"/.nemoclaw/sandboxes.json").sandboxes))' [] # recovered for display only — NOT persisted (agent is unknowable from the gateway list) ``` (Recovery was also re-verified while OpenShell was connected to a NemoClaw per-port gateway `nemoclaw-8092` — `connected_other` to a NemoClaw-managed name still recovers; a foreign gateway name does not.) **3. Post-fix graceful degradation — gateway unreachable, empty registry → no crash, no bogus names:** ```console $ node ./bin/nemoclaw.js list # gateway down: Connection refused on :8080 # `openshell sandbox list` → transport error No sandboxes registered. Run `nemoclaw onboard` to get started. $ echo $? 0 ``` Targeted + adjacent unit tests pass (registry-recovery, gateway-runtime, inventory, openshell adapter, repro-2666, gateway/connect drift; ~290 tests), `npm run typecheck:cli` clean, Biome clean; `nemoclaw-start` (126) passes with the `nemoclaw/` subproject deps installed. - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [x] Targeted tests pass for changed behavior - [x] Tests added or updated for new or changed behavior - [x] No secrets, API keys, or credentials committed Signed-off-by: Yimo Jiang <yimoj@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Improved recovery for empty registries with unseeded sessions via bounded, read-only ephemeral results. * Inventory rendering now supports gateway-recovered sandboxes (uses `unknown` agent/GPU and shows `phase` when available). * Added live sandbox phase parsing with `parseLiveSandboxEntries`. * **Bug Fixes** * Added stderr retention for lifecycle/probe classification (`includeStderr`) when probe errors are ignored. * Prevented transient recovery/display markers (`recoveredFromGateway`, `livePhase`) from persisting to disk. * **Tests** * Added cases for unseeded vs seeded recovery, probe option behavior, and phase parsing variations. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --- Supersedes NVIDIA#5771 (reopened from NVIDIA/NemoClaw branch so trusted advisor workflows can run). --------- Signed-off-by: Yimo Jiang <yimoj@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
## Summary Adds the v0.0.69 release notes to the published release-notes page so users can see the shipped sandbox recovery, Deep Agents Code, Hermes, inference, policy, and release-validation changes. The section is based on the v0.0.69 announcement and links each user-facing theme to the deeper docs pages that already cover the behavior. ## Changes - Added a new `v0.0.69` section to `docs/about/release-notes.mdx`. - Linked release-note themes to lifecycle, backup, troubleshooting, Deep Agents Code, commands, workspace, messaging, Hermes, inference, security, monitoring, and network-policy docs. Source summary: - NVIDIA#5455 -> `docs/about/release-notes.mdx`: Summarized persistent workspace and state cleanup during sandbox destroy. - NVIDIA#5738 -> `docs/about/release-notes.mdx`: Summarized nonzero exit status preservation for failed hosted endpoint validation. - NVIDIA#5786 -> `docs/about/release-notes.mdx`: Summarized live sandbox rediscovery when local registry state is missing. - NVIDIA#5881 -> `docs/about/release-notes.mdx`: Summarized the `nemo-deepagents` alias command surface. - NVIDIA#5594 -> `docs/about/release-notes.mdx`: Summarized the Hermes Agent 2026.6.19 update. - NVIDIA#5777 -> `docs/about/release-notes.mdx`: Summarized manifest-derived messaging channel support. - NVIDIA#5825 -> `docs/about/release-notes.mdx`: Summarized DeepSeek V4 Flash managed-vLLM defaults for DGX Station. - NVIDIA#5877 -> `docs/about/release-notes.mdx`: Summarized provider switch metadata preservation. - NVIDIA#5932 -> `docs/about/release-notes.mdx`: Summarized transient inference smoke retry behavior. - NVIDIA#5934 -> `docs/about/release-notes.mdx`: Summarized constrained inference smoke retry boundaries. - NVIDIA#5681 -> `docs/about/release-notes.mdx`: Summarized Shields config-hash sealing during auto-restore. - NVIDIA#5682 -> `docs/about/release-notes.mdx`: Summarized sandbox connect process-limit enforcement. - NVIDIA#5683 -> `docs/about/release-notes.mdx`: Summarized JSON agent failure provenance warnings. - NVIDIA#5711 -> `docs/about/release-notes.mdx`: Summarized sparse-source log breadcrumbs. - NVIDIA#5838 -> `docs/about/release-notes.mdx`: Summarized host-authoritative Shields status. - NVIDIA#5880 -> `docs/about/release-notes.mdx`: Summarized policy round-trip documentation updates. - NVIDIA#5886 -> `docs/about/release-notes.mdx`: Summarized network request approval-flow documentation updates. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [x] Tests not applicable — justification: doc-only release-notes prose; no runtime behavior changed. - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Git hooks passed during commit and push, or `npx prek run --from-ref main --to-ref HEAD` passes - [ ] Targeted tests pass for changed behavior - [ ] Full `npm test` passes (broad runtime changes only) - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) `npm run docs` passed with 0 errors and the existing Fern light-mode accent contrast warning. `fern check --warnings` reported the same accent-color warning. --- Signed-off-by: Miyoung Choi <miyoungc@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added release notes for **v0.0.69**, covering improved sandbox lifecycle recovery (state preservation across destroy/recreate/rebuild/recovery/validation failures), clearer Deep Agents Code terminal/CLI behavior, and safer Hermes messaging/provider switching with manifest-driven channels. * Improved inference setup validation guidance, including handling of local/compatible endpoints and redaction of sensitive validation errors. * Refreshed release-gate documentation with clearer approval examples and validation behavior for NVIDIA API keys vs hosted inference keys. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This was referenced Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nemoclaw listprinted "No sandboxes registered" while the live OpenShell gateway and container were healthy andnemoclaw <name> statusreportedPhase: Ready(reported on DGX Station with a Deep Agents sandbox). Thelistrecovery path required a seed — an existing registry entry, an onboard session, or an explicit requested name — before it would probe the gateway. After a local registry loss with no seed, it trusted the empty registry and reported nothing, even though the named-status path could find and reconcile the same sandbox.Related Issue
Fixes #5714
Changes
recoverRegistryEntriesnow attempts recovery whenever the local registry is empty, even with no session/requested-name seed.nemoclaw listnever selects or starts a gateway. It inspects the live sandbox list only when OpenShell is connected to a NemoClaw-managed gateway (nemoclawor a per-portnemoclaw-<port>) — never a foreign OpenShell gateway. Gateway probes are non-fatal (ignoreProbeErrors), so a hung/timed-out gateway falls back to the empty registry instead of exiting the process, and recovery is gated on a clean (status === 0)sandbox listso error text is never parsed as a sandbox name.openshell sandbox listexposes only NAME/CREATED/PHASE, not the agent or gateway binding. Persisting a recovered entry would defaultagenttoopenclaweverywhere downstream (state dirs, connect, rebuild, doctor) and permanently misclassify a Deep Agents/Hermes sandbox. Recovered sandboxes are surfaced for the currentlistonly; follow-up named commands reconcile the real agent via the gateway.agent/model/provider/GPUasunknownrather than inventing OpenClaw/CPU defaults.getNamedGatewayLifecycleStategains an opt-inignoreProbeErrors(default keeps existing fatal behavior);captureOpenshellforwardsincludeStderrso non-fatal probes still capture the gateway status text.Type of Change
Verification
Verified end-to-end through the real worktree CLI (
node ./bin/nemoclaw.js …) against a live OpenShell gateway. The local registry was emptied and the onboard session removed to simulate the reporter's registry loss; the on-disk registry is checked after each run.1. Pre-fix (reporter mismatch) —
listblind while a Ready sandbox is live:2. Post-fix —
listrediscovers the live sandboxes (display-only, registry stays empty):(Recovery was also re-verified while OpenShell was connected to a NemoClaw per-port gateway
nemoclaw-8092—connected_otherto a NemoClaw-managed name still recovers; a foreign gateway name does not.)3. Post-fix graceful degradation — gateway unreachable, empty registry → no crash, no bogus names:
Targeted + adjacent unit tests pass (registry-recovery, gateway-runtime, inventory, openshell adapter, repro-2666, gateway/connect drift; ~290 tests),
npm run typecheck:cliclean, Biome clean;nemoclaw-start(126) passes with thenemoclaw/subproject deps installed.Verifiedin GitHubnpx prek run --from-ref main --to-ref HEADpassesSigned-off-by: Yimo Jiang yimoj@nvidia.com
Summary by CodeRabbit
unknownagent/GPU and showsphasewhen available).parseLiveSandboxEntries.includeStderr) when probe errors are ignored.recoveredFromGateway,livePhase) from persisting to disk.Supersedes #5771 (reopened from NVIDIA/NemoClaw branch so trusted advisor workflows can run).