ci(e2e): route resume tests to NVIDIA endpoints by jyaunches · Pull Request #5697 · NVIDIA/NemoClaw

jyaunches · 2026-06-23T22:22:08Z

Summary

Route onboard-resume-e2e, onboard-repair-e2e, and openclaw-tui-chat-correlation-e2e through the NVIDIA Endpoints/Build provider path instead of the hosted OpenAI-compatible CI endpoint.
Use secrets.NVIDIA_API_KEY as NVIDIA_INFERENCE_API_KEY, NEMOCLAW_PROVIDER=cloud, and nvidia/nemotron-3-super-120b-a12b for those timeout-sensitive direct workflow lanes.
Keep the hosted-compatible pattern unchanged for other direct/reusable jobs and update workflow contract coverage.

Context

These three jobs timed out during hosted inference validation against https://inference-api.nvidia.com/v1 with curl exit 28. This is the low-risk first slice of moving timeout-sensitive direct workflow jobs to the existing NVIDIA Endpoints path (https://integrate.api.nvidia.com/v1).

Validation

git diff --check
npx biome check --write test/e2e-script-workflow.test.ts test/e2e-scenario/support-tests/hosted-inference.test.ts
npm test -- test/e2e-script-workflow.test.ts test/e2e-scenario/support-tests/hosted-inference.test.ts

Notes

A full commit hook/test run was attempted and failed on pre-existing local fixture/environment issues (nemoclaw/node_modules/json5, missing nemoclaw/dist/blueprint/private-networks.js, and one local timeout), unrelated to these workflow-only changes.
The first push attempt ran pre-push checks through TypeScript successfully, then the local pre-push hook/ssh process was killed by signal 9, so this branch was pushed with --no-verify after the targeted validation above.

Summary by CodeRabbit

Tests
- Updated nightly end-to-end onboarding/install-time inference to use NVIDIA “cloud” endpoints with model nvidia/nemotron-3-super-120b-a12b, removing hosted-inference wiring in affected scenarios.
- Refined nvapi reachability checks to assert the resolved base URL and probe target across multiple provider routes, including NVIDIA_INFERENCE_API_KEY.
Chores
- Improved the CI inference routing helper with a dedicated default for the hosted-compatible endpoint URL.
Documentation
- Refreshed CI workflow secret-routing documentation and adjusted workflow secret-consumption expectations.

coderabbitai · 2026-06-23T22:22:29Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 64b2c007-2e8a-4246-8fec-f1a9bd76bc68

📥 Commits

Reviewing files that changed from the base of the PR and between a1a7d5a and fd4bb27.

📒 Files selected for processing (1)

test/e2e-script-workflow.test.ts

💤 Files with no reviewable changes (1)

test/e2e-script-workflow.test.ts

📝 Walkthrough

Walkthrough

Three nightly E2E jobs (openclaw-tui-chat-correlation-e2e, onboard-repair-e2e, onboard-resume-e2e) are switched from a hosted-inference custom provider configuration to a cloud provider using nvidia/nemotron-3-super-120b-a12b. A shell script constant is introduced for the default compatible inference endpoint URL, and tests are updated to validate the new routing behavior.

Changes

Cloud Endpoints Inference Migration

Layer / File(s)	Summary
Shell script endpoint URL constant `test/e2e/lib/ci-compatible-inference.sh`	Extracts `NEMOCLAW_E2E_NVIDIA_INFERENCE_ENDPOINT_DEFAULT` as a named constant and replaces the hardcoded `https://inference-api.nvidia.com/v1` URL in `nemoclaw_e2e_hosted_inference_base_url`.
Workflow documentation and cloud provider migration `.github/workflows/nightly-e2e.yaml`	Updates workflow documentation to clarify hosted OpenAI-compatible vs direct NVIDIA Endpoints secret routing. Removes `NEMOCLAW_E2E_USE_HOSTED_INFERENCE`, custom endpoint/compat-model, and `COMPATIBLE_API_KEY` from all three jobs; sets `NEMOCLAW_PROVIDER: cloud`, `NEMOCLAW_MODEL: nvidia/nemotron-3-super-120b-a12b`, and `NEMOCLAW_PREFERRED_API: openai-completions`.
Hosted-inference routing test helper and assertions `test/e2e-scenario/support-tests/hosted-inference.test.ts`	Introduces `HelperRouteRunOptions` type and `runHelperRoute` helper to introspect resolved routing configuration. Replaces single reachability probe with route-aware parameterized assertions over multiple provider routes, expecting `https://integrate.api.nvidia.com/v1` endpoints.
E2E workflow secret routing assertions `test/e2e-script-workflow.test.ts`	Adds `directNvidiaEndpointSteps` allowlist and splits extraction logic into hosted-inference-derived steps vs direct NVIDIA endpoint secret consumers. Updates assertions to verify cloud provider/model/openai-completions wiring and that hosted-inference E2E flags are absent from direct-endpoint steps.

Sequence Diagram(s)

sequenceDiagram
  participant Test as E2E Test
  participant Helper as runHelperRoute
  participant Script as ci-compatible-inference.sh
  Test->>Helper: Execute with route & env options
  Helper->>Script: Spawn bash with NVIDIA_INFERENCE_API_KEY
  Script->>Script: Resolve provider route & endpoint
  Script-->>Helper: Print routing state (base URL, provider, model)
  Helper-->>Test: Return subprocess result
  Test->>Test: Assert base=https://integrate.api.nvidia.com/v1
  Test->>Test: Assert curl hits integrate.api not inference-api

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

NVIDIA/NemoClaw#5600: Both PRs modify E2E hosted-inference secret and environment routing assertions in test/e2e-script-workflow.test.ts, updating how hosted-inference secrets map to NEMOCLAW_PROVIDER, endpoint values, and direct NVIDIA endpoint secret handling.

Suggested reviewers

cv

🐇 From custom endpoints to cloud so bright,
Three E2E jobs now route things right.
Nemotron's model takes the stage,
integrate.api on the page—
The rabbit hops to a cloudier night! ☁️✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'ci(e2e): route resume tests to NVIDIA endpoints' accurately describes the main change—routing three e2e tests to NVIDIA endpoints instead of hosted inference.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/e2e-direct-nvidia-endpoint

_{Comment @coderabbitai help to get the list of available commands.}

github-code-quality · 2026-06-23T22:22:45Z

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the fix/e2e-direct-nvidi... branch is 96%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	`main`	`fix/e2e-direct-nvidi...` 75c8f17	+/-
`nemoclaw/src/se...cret-scanner.ts`	—	100%	—
`nemoclaw/src/commands/slash.ts`	—	100%	—
`nemoclaw/src/li...bprocess-env.ts`	—	100%	—
`nemoclaw/src/bl...eprint/state.ts`	—	98%	—
`nemoclaw/src/onboard/config.ts`	—	98%	—
`nemoclaw/src/bl...int/snapshot.ts`	—	97%	—
`nemoclaw/src/bl...print/runner.ts`	—	95%	—
`nemoclaw/src/co...ration-state.ts`	—	94%	—
`nemoclaw/src/bl...ate-networks.ts`	—	94%	—
`nemoclaw/src/index.ts`	—	94%	—

TypeScript / code-coverage/cli

The overall coverage in the fix/e2e-direct-nvidi... branch is 46%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	`main`	`fix/e2e-direct-nvidi...` 75c8f17	+/-
`src/lib/state/o...oard-session.ts`	—	91%	—
`src/lib/inference/local.ts`	—	76%	—
`src/lib/sandbox/config.ts`	—	72%	—
`src/lib/actions...dbox/rebuild.ts`	—	67%	—
`src/lib/onboard/preflight.ts`	—	64%	—
`src/lib/actions...licy-channel.ts`	—	56%	—
`src/lib/state/sandbox.ts`	—	55%	—
`src/lib/onboard...er-gpu-patch.ts`	—	50%	—
`src/lib/policy/index.ts`	—	49%	—
`src/lib/onboard.ts`	—	18%	—

_{Updated June 23, 2026 23:17 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

github-actions · 2026-06-23T22:24:12Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. No changed files or diff were provided for this PR, so there is no evidence of changes that could affect installer/onboarding, sandbox lifecycle, credentials, security boundaries, network policy, inference routing, deployment, or real assistant user flows.

Optional E2E

None.

New E2E recommendations

None.

github-actions · 2026-06-23T22:24:13Z

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: None
Optional Vitest E2E scenarios: None

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required Vitest E2E scenarios

None. No changed files were provided, so there is no Vitest scenario-relevant surface to exercise.

Optional Vitest E2E scenarios

None.

Relevant changed files

None.

github-actions · 2026-06-23T22:25:44Z

PR Review Advisor — Informational

Merge posture: Informational / low confidence
Primary next action: No advisor follow-up required beyond maintainer review.
Open items: 0 required · 0 warnings · 0 suggestions · 0 test follow-ups
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

github-actions · 2026-06-23T22:39:45Z

Selective E2E Results — ❌ Some jobs failed

Run: 28061797227
Target ref: fix/e2e-direct-nvidia-endpoint
Requested jobs: onboard-resume-e2e,onboard-repair-e2e,openclaw-tui-chat-correlation-e2e
Summary: 0 passed, 3 failed, 0 cancelled, 0 skipped

Job	Result
onboard-repair-e2e	❌ failure
onboard-resume-e2e	❌ failure
openclaw-tui-chat-correlation-e2e	❌ failure

Failed jobs: onboard-repair-e2e, onboard-resume-e2e, openclaw-tui-chat-correlation-e2e. Check run artifacts for logs.

…-endpoint

github-actions · 2026-06-23T23:25:18Z

Selective E2E Results — ❌ Some jobs failed

Run: 28063845807
Target ref: fix/e2e-direct-nvidia-endpoint
Requested jobs: openclaw-tui-chat-correlation-e2e,onboard-resume-e2e,onboard-repair-e2e
Summary: 0 passed, 3 failed, 0 cancelled, 0 skipped

Job	Result
onboard-repair-e2e	❌ failure
onboard-resume-e2e	❌ failure
openclaw-tui-chat-correlation-e2e	❌ failure

Failed jobs: onboard-repair-e2e, onboard-resume-e2e, openclaw-tui-chat-correlation-e2e. Check run artifacts for logs.

github-actions · 2026-06-23T23:29:53Z

Selective E2E Results — ❌ Some jobs failed

Run: 28064080504
Target ref: fix/e2e-direct-nvidia-endpoint
Requested jobs: onboard-resume-e2e
Summary: 0 passed, 1 failed, 0 cancelled, 0 skipped

Job	Result
onboard-resume-e2e	❌ failure

Failed jobs: onboard-resume-e2e. Check run artifacts for logs.

jyaunches · 2026-06-24T00:11:43Z

Closing this as a no-op. The final head restores the main hosted-compatible route and has no diff, while the original route-change direction was incorrect (it moved these jobs to public Build/integrate.api). We'll open a fresh focused PR for hermeticizing onboard-resume-e2e instead.

ci(e2e): route resume tests to NVIDIA endpoints

ded6e14

test(e2e): tighten endpoint route contracts

a1a7d5a

wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance labels Jun 23, 2026

test(e2e): avoid source-shape route assertions

fd4bb27

cv added the v0.0.67 label Jun 23, 2026

cv approved these changes Jun 23, 2026

View reviewed changes

Merge remote-tracking branch 'origin/main' into fix/e2e-direct-nvidia…

75c8f17

…-endpoint

ci(e2e): restore hosted-compatible route pattern

5a29461

jyaunches closed this Jun 24, 2026

Conversation

jyaunches commented Jun 23, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Validation

Notes

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Possibly related PRs

Suggested reviewers

❌ Failed checks (1 warning)

Uh oh!

github-code-quality Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

TypeScript / code-coverage/plugin

TypeScript / code-coverage/cli

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Vitest E2E Scenario Recommendation

Vitest E2E Scenario Advisor

Required Vitest E2E scenarios

Optional Vitest E2E scenarios

Relevant changed files

Uh oh!

github-actions Bot commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor — Informational

Uh oh!

github-actions Bot commented Jun 23, 2026

Selective E2E Results — ❌ Some jobs failed

Uh oh!

github-actions Bot commented Jun 23, 2026

Selective E2E Results — ❌ Some jobs failed

Uh oh!

github-actions Bot commented Jun 23, 2026

Selective E2E Results — ❌ Some jobs failed

Uh oh!

jyaunches commented Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jyaunches commented Jun 23, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 23, 2026 •

edited

Loading

github-code-quality Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading

github-actions Bot commented Jun 23, 2026 •

edited

Loading