test(vitest): improve failure and developer feedback by cv · Pull Request #6701 · NVIDIA/NemoClaw

cv · 2026-07-12T04:03:22Z

Summary

CI now suppresses passed-test logs while preserving failure logs and GitHub annotations, and ordinary local runs let Vitest select its environment-aware reporter. Focused changed/watch commands and opt-in seeded-shuffle and async-leak diagnostics shorten developer feedback without changing deterministic coverage lanes. This is PR 6 in the #6692 stack and is based on #6700.

Related Issue

Part of #6692

Changes

Remove the explicit root reporter override so Vitest selects the local reporter and adds its GitHub Actions reporter, while resolveVitestFeedback keeps CI on silent: "passed-only" and preserves the existing skipped-test suppression.
Add focused test:changed and test:watch commands for the cli, plugin, and e2e-support projects.
Add opt-in test:shuffle and test:diagnose:leaks commands with coverage disabled; shuffle accepts Vitest's printed seed for replay, and leak diagnostics retain both the default and hanging-process reporters.
Register exact root watch triggers for current YAML, Python, shell, generated, manifest, and workflow inputs that are read outside Vite's import graph. test/helpers/vitest-watch-triggers.ts is required by the focused watch command, and test/vitest-watch-triggers.test.ts protects every concrete mapping, all 31 current shared-workflow consumers, path existence, Windows normalization, and unrelated-file exclusions.
Update AGENTS.md and CONTRIBUTING.md with the focused commands, seed replay, leak exit-code caveat, and local/CI reporter behavior.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Quality Gates

Tests added or updated for changed behavior
Existing tests cover changed behavior — justification:
Tests not applicable — justification:
Docs updated for user-facing behavior changes
Docs not applicable — justification: user-facing NemoClaw behavior is unchanged; the contributor-facing Vitest commands and feedback contract are documented in AGENTS.md and CONTRIBUTING.md.
Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification:
Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 9 focused contract tests passed; real changed, watch, seeded-shuffle, and async-leak command smokes passed; CLI type-checking and the source-shape/title/test-size gates passed.
Applicable broad gate passed — npm test: 1,442 files passed and 3 expected files skipped; 16,473 tests passed and 40 expected opt-in/platform tests skipped. npm run test:fast also passed 927 files and 9,876 tests with 1 expected skip.
Quality Gates section completed with required justifications or waivers
No secrets, API keys, or credentials committed
npm run docs builds without warnings (doc changes only) — passed with zero errors; Fern reported two unrelated baseline warnings.
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

New Features
- Added focused test commands for changed tests, watch mode, shuffled execution, and async leak diagnosis.
- Added watch-mode mappings to automatically run related tests when supported files change.
- Added reproducible shuffle diagnostics and improved CI test output behavior.
Documentation
- Expanded contributor guidance with examples and recommendations for using the new testing workflows.
Tests
- Added coverage validating command options, watch triggers, path handling, and CI feedback behavior.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

coderabbitai · 2026-07-12T04:03:27Z

📝 Walkthrough

Walkthrough

Added focused Vitest npm scripts for changed tests, watch mode, deterministic shuffling, and async-leak diagnostics. Centralized CI feedback behavior, added opaque-input watch triggers, tested the workflows, and documented their use.

Changes

Vitest developer feedback

Layer / File(s)	Summary
Feedback workflows and configuration `package.json`, `test/helpers/vitest-feedback.ts`, `test/helpers/vitest-npm-script.ts`, `vitest.config.ts`, `test/vitest-developer-feedback.test.ts`	Adds focused Vitest scripts, CI-aware feedback settings, command interception helpers, configuration wiring, and assertions for command flags and reporters.
Opaque-input watch mapping `test/helpers/vitest-watch-triggers.ts`, `vitest.config.ts`, `test/vitest-watch-triggers.test.ts`	Maps opaque file changes to concrete tests, expands workflow contracts, normalizes paths, and validates trigger outputs.
Developer workflow guidance `AGENTS.md`, `CONTRIBUTING.md`	Documents focused testing, watch mode, shuffle seed replay, and async-leak diagnostics.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested labels: feature

Suggested reviewers: ericksoa, jyaunches

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: improved Vitest failure handling and developer feedback.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch codex/vitest-failure-feedback-diagnostics

_{Comment @coderabbitai help to get the list of available commands.}

github-code-quality · 2026-07-12T04:03:57Z

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the codex/vitest-failure... branch remains at 79%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.

File	`main` d683b6a	`codex/vitest-failure...` daeba61	+/-
`src/lib/messagi...ate-resolver.ts`	78%	56%	-22%
`src/lib/messagi...ate-resolver.ts`	88%	76%	-12%
`src/lib/messagi...parser-utils.ts`	90%	86%	-4%
`src/lib/inferen...board-probes.ts`	94%	92%	-2%
`src/lib/security/redact-url.ts`	100%	98%	-2%
`src/lib/state/registry.ts`	83%	82%	-1%
`src/lib/agent/m...fest-readers.ts`	95%	95%	0%
`src/lib/agent/s...store-reader.ts`	88%	88%	0%
`src/lib/state/config-io.ts`	95%	96%	+1%
`src/lib/sandbox...vileged-exec.ts`	75%	87%	+12%

_{Updated July 12, 2026 07:05 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

github-actions · 2026-07-12T04:04:09Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. This PR changes test-runner feedback and local developer tooling only. It does not modify installer/onboarding behavior, sandbox lifecycle, credentials, security or network boundaries, inference routing, deployment, or assistant user flows. The watch-trigger mappings are covered by the added deterministic tests and do not change live E2E target behavior.

Optional E2E

None.

New E2E recommendations

None.

github-actions · 2026-07-12T04:05:14Z

PR Review Advisor — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: No advisor follow-up required beyond maintainer review.
Findings: 0 required · 0 warnings · 0 optional suggestions
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Workflow run details

This is an automated review. Required findings need action before merge. Warnings and optional suggestions do not require a response or follow-up. A human maintainer makes the final merge decision.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv · 2026-07-12T04:07:31Z

Addressed the codebase-growth guard in signed commit 096dba6. The two non-asserting fixture branches moved unchanged into test/helpers/vitest-npm-script.ts; test/vitest-developer-feedback.test.ts now has zero if statements. All 9 focused contracts, CLI type-checking, source-shape/title/conditional scans, and hooks pass. This was an additive fast-forward with no history rewrite.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

Signed-off-by: Carlos Villela <cvillela@nvidia.com> # Conflicts: # AGENTS.md # CONTRIBUTING.md # vitest.config.ts

github-actions · 2026-07-12T06:59:22Z

🌿 Preview your docs: https://nvidia-preview-pr-6701.docs.buildwithfern.com/nemoclaw

coderabbitai

🧹 Nitpick comments (1)

test/vitest-developer-feedback.test.ts (1)
18-37: 🎯 Functional Correctness | 🔵 Trivial | 🏗️ Heavy lift

Exercise reporter behavior through an observable Vitest run.

This test only inspects configuration shape; it never verifies reporter selection, GitHub annotations, or retained failure output. Add a minimal failing Vitest subprocess assertion under GITHUB_ACTIONS=true, or narrow the test title and assertions to the resolver/config contract.

As per path instructions, tests should verify behavioral outcomes rather than implementation or configuration shape.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/vitest-developer-feedback.test.ts` around lines 18 - 37, Update the
“Vitest developer feedback” test to verify observable behavior through a minimal
failing Vitest subprocess under GITHUB_ACTIONS=true, including reporter
selection, GitHub annotations, and retained failure output; otherwise narrow the
test title and assertions to the resolver/config contract and remove
configuration-shape checks such as testOptions.reporters and testOptions.silent.
Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/vitest-developer-feedback.test.ts`:
- Around line 18-37: Update the “Vitest developer feedback” test to verify
observable behavior through a minimal failing Vitest subprocess under
GITHUB_ACTIONS=true, including reporter selection, GitHub annotations, and
retained failure output; otherwise narrow the test title and assertions to the
resolver/config contract and remove configuration-shape checks such as
testOptions.reporters and testOptions.silent.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c1542fbb-695a-4cf1-a102-e54ca6b72eaf

📥 Commits

Reviewing files that changed from the base of the PR and between d683b6a and daeba61.

📒 Files selected for processing (9)

AGENTS.md
CONTRIBUTING.md
package.json
test/helpers/vitest-feedback.ts
test/helpers/vitest-npm-script.ts
test/helpers/vitest-watch-triggers.ts
test/vitest-developer-feedback.test.ts
test/vitest-watch-triggers.test.ts
vitest.config.ts

## Summary Four file-granular integration-test outliers are split along behavioral boundaries while preserving every title and assertion. Five serial local samples reduce the slowest median from 51.44s to 24.11s, with every replacement file below 30s and no family-runtime regression; pool and scheduling behavior remain unchanged. This is PR 7 in the #6692 stack and is based on #6701. ## Related Issue Part of #6692 ## Changes - Split the Hermes restart seal, local credential helper, onboarding, and OpenClaw integrity suites into focused integration-test files. - Keep repeated setup in non-test registration or fixture modules so each importing test file owns its hooks and mutable resources. - Keep the credential-helper execution-root ownership assertions in the same session shard as tests that create those roots, avoiding cross-file races under normal file parallelism. - Bind ten relocated Hermes filesystem checks to one descriptor or read, and route the eleven deliberate current-path assertions through a descriptor-backed snapshot helper. - Preserve all 257 previously listed titles plus the same two platform-gated Hermes cases; the focused 4-worker run collected 265 definitions and passed with only those two expected skips. - Regenerate all 93 timing hints from CI run `29180354863` at head `92439a702f1a8c7cca8609f1259304115a10931d`, including seven replacement shards above the 5-second fallback, and update the sequencer contract to separate the two slowest replacements. - Update the OpenClaw dependency review so each security boundary points to the new base, contract, or plugin-install suite. - Record five-sample local slowest-shard medians: credential helper 24.11s (was 51.44s), Hermes 23.37s (was 50.25s), onboard 8.73s (was 35.41s), and OpenClaw integrity 12.22s (was 33.85s). - Reweighted CI reduced the slowest CLI shard job from 378s to 326s (52s / 13.8%) and narrowed shard spread from 95s to 62s. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: product behavior is unchanged; the security review was updated only to keep regression-evidence paths resolvable after the split. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent static review verified exact title/assertion preservation and identified the credential-root race and stale security mappings; both were fixed before commit, and the full canonical suite passed. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — the 15-file 4-worker run passed 263 tests with 2 expected platform skips; project-membership, title-style, conditional, source-shape, test-size, CLI type-check, and five-sample timing checks passed. - [x] Applicable broad gate passed — `npm test`: 1,452 files passed and 3 expected files skipped; 16,473 tests passed and 40 expected tests skipped. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — passed with zero errors; Fern reported two unrelated baseline warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Carlos Villela <cvillela@nvidia.com> --------- Signed-off-by: Carlos Villela <cvillela@nvidia.com>

## Summary Vitest project validation now compares every filesystem test candidate with Vitest's complete eight-project roster, rejecting omissions, overlaps, wrong routing, unexpected files, and unexpected project names. The shared credential-free E2E workflow now applies Vitest's native e2e/credential-free tag filter in addition to the existing safe matrix scanner. The dashboard-port regression fixture also uses a coherent fake OpenShell 0.0.72 component set and an OS-assigned gateway port so the repo-wide gate stays hermetic. This is the final entry in the ordered #6692 stack after #6693, #6696, #6697, #6699, #6700, #6701, #6702, and #6705 landed. ## Related Issue Closes #6692 ## Changes - Discover broad test/spec JavaScript and TypeScript candidates under src, test, and nemoclaw/src, including nested dist, coverage, and .venv paths that Vitest would see. - Compare candidate routing with Vitest file listings and the complete roster from list-tags JSON, including empty and opt-in projects. - Produce sorted diagnostics for zero membership, overlap, wrong project, unsupported candidates, unexpected listed files, and missing or extra projects. - Invoke Vitest portably through process.execPath and node_modules/vitest/vitest.mjs. - Require the exact native credential-free tag filter in the shared E2E workflow contract while retaining the TypeScript scanner's path, project, ID, and declaration checks. - Keep the dashboard-port exhaustion regression independent of port 8080, developer OpenShell overrides, channel selection, and network installation. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: these are contributor/test-harness safeguards; AGENTS.md now states the exhaustive project-membership contract, the E2E README already documents the native module tag, and the required documentation-writer review found no user-facing change. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent review covered the shared credential-free workflow and onboarding regression fixture; it confirmed the filter preserves the existing scanner and the fixture's child-only environment cannot reach a real OpenShell install. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub - [x] Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 46 integration tests passed across the boundary and dashboard-port files; 4 E2E-support workflow tests passed; project membership is exact for 1,537 candidates across 8 projects; CLI type-check, title style, Biome, and diff checks passed. The fixture also passed with hostile inherited OpenShell paths and dev-channel selection. - [x] Applicable broad gate passed — npm test passed 1,456 files / 16,515 tests before the final focused hardening; npm run check passed the repo-wide structural, CLI coverage, and plugin coverage gates, followed by focused reruns and normal hooks after the final child-environment pin. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] npm run docs builds without warnings (doc changes only) - [ ] Doc pages follow the style guide (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Carlos Villela <cvillela@nvidia.com>  ## Summary by CodeRabbit - **Tests** - Credential-free end-to-end test runs now execute only tests matching the credential-free tag. - Vitest project validation now checks for missing, overlapping, unsupported, wrong, zero-membership, and unexpected test-to-project assignments, plus roster mismatches. - Expanded boundary tests cover test candidate discovery, expected project mapping, Vitest invocation resolution, and listing/roster parsing. - Improved onboarding “dashboard port exhaustion” assertions for more hermetic, reliable verification. - **Documentation** - Updated testing guidance to require Vitest project globs be disjoint and exhaustive, enforced by the projects check.  --------- Signed-off-by: Carlos Villela <cvillela@nvidia.com>

## Summary CI now suppresses passed-test logs while preserving failure logs and GitHub annotations, and ordinary local runs let Vitest select its environment-aware reporter. Focused changed/watch commands and opt-in seeded-shuffle and async-leak diagnostics shorten developer feedback without changing deterministic coverage lanes. This is PR 6 in the NVIDIA#6692 stack and is based on NVIDIA#6700. ## Related Issue Part of NVIDIA#6692 ## Changes - Remove the explicit root reporter override so Vitest selects the local reporter and adds its GitHub Actions reporter, while `resolveVitestFeedback` keeps CI on `silent: "passed-only"` and preserves the existing skipped-test suppression. - Add focused `test:changed` and `test:watch` commands for the `cli`, `plugin`, and `e2e-support` projects. - Add opt-in `test:shuffle` and `test:diagnose:leaks` commands with coverage disabled; shuffle accepts Vitest's printed seed for replay, and leak diagnostics retain both the default and hanging-process reporters. - Register exact root watch triggers for current YAML, Python, shell, generated, manifest, and workflow inputs that are read outside Vite's import graph. `test/helpers/vitest-watch-triggers.ts` is required by the focused watch command, and `test/vitest-watch-triggers.test.ts` protects every concrete mapping, all 31 current shared-workflow consumers, path existence, Windows normalization, and unrelated-file exclusions. - Update `AGENTS.md` and `CONTRIBUTING.md` with the focused commands, seed replay, leak exit-code caveat, and local/CI reporter behavior. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: user-facing NemoClaw behavior is unchanged; the contributor-facing Vitest commands and feedback contract are documented in `AGENTS.md` and `CONTRIBUTING.md`. - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 9 focused contract tests passed; real changed, watch, seeded-shuffle, and async-leak command smokes passed; CLI type-checking and the source-shape/title/test-size gates passed. - [x] Applicable broad gate passed — `npm test`: 1,442 files passed and 3 expected files skipped; 16,473 tests passed and 40 expected opt-in/platform tests skipped. `npm run test:fast` also passed 927 files and 9,876 tests with 1 expected skip. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — passed with zero errors; Fern reported two unrelated baseline warnings. - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Carlos Villela <cvillela@nvidia.com>  ## Summary by CodeRabbit * **New Features** * Added focused test commands for changed tests, watch mode, shuffled execution, and async leak diagnosis. * Added watch-mode mappings to automatically run related tests when supported files change. * Added reproducible shuffle diagnostics and improved CI test output behavior. * **Documentation** * Expanded contributor guidance with examples and recommendations for using the new testing workflows. * **Tests** * Added coverage validating command options, watch triggers, path handling, and CI feedback behavior.  --------- Signed-off-by: Carlos Villela <cvillela@nvidia.com>

## Summary Four file-granular integration-test outliers are split along behavioral boundaries while preserving every title and assertion. Five serial local samples reduce the slowest median from 51.44s to 24.11s, with every replacement file below 30s and no family-runtime regression; pool and scheduling behavior remain unchanged. This is PR 7 in the NVIDIA#6692 stack and is based on NVIDIA#6701. ## Related Issue Part of NVIDIA#6692 ## Changes - Split the Hermes restart seal, local credential helper, onboarding, and OpenClaw integrity suites into focused integration-test files. - Keep repeated setup in non-test registration or fixture modules so each importing test file owns its hooks and mutable resources. - Keep the credential-helper execution-root ownership assertions in the same session shard as tests that create those roots, avoiding cross-file races under normal file parallelism. - Bind ten relocated Hermes filesystem checks to one descriptor or read, and route the eleven deliberate current-path assertions through a descriptor-backed snapshot helper. - Preserve all 257 previously listed titles plus the same two platform-gated Hermes cases; the focused 4-worker run collected 265 definitions and passed with only those two expected skips. - Regenerate all 93 timing hints from CI run `29180354863` at head `92439a702f1a8c7cca8609f1259304115a10931d`, including seven replacement shards above the 5-second fallback, and update the sequencer contract to separate the two slowest replacements. - Update the OpenClaw dependency review so each security boundary points to the new base, contract, or plugin-install suite. - Record five-sample local slowest-shard medians: credential helper 24.11s (was 51.44s), Hermes 23.37s (was 50.25s), onboard 8.73s (was 35.41s), and OpenClaw integrity 12.22s (was 33.85s). - Reweighted CI reduced the slowest CLI shard job from 378s to 326s (52s / 13.8%) and narrowed shard spread from 95s to 62s. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: product behavior is unchanged; the security review was updated only to keep regression-evidence paths resolvable after the split. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent static review verified exact title/assertion preservation and identified the credential-root race and stale security mappings; both were fixed before commit, and the full canonical suite passed. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — the 15-file 4-worker run passed 263 tests with 2 expected platform skips; project-membership, title-style, conditional, source-shape, test-size, CLI type-check, and five-sample timing checks passed. - [x] Applicable broad gate passed — `npm test`: 1,452 files passed and 3 expected files skipped; 16,473 tests passed and 40 expected tests skipped. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — passed with zero errors; Fern reported two unrelated baseline warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Carlos Villela <cvillela@nvidia.com> --------- Signed-off-by: Carlos Villela <cvillela@nvidia.com>

…A#6707) ## Summary Vitest project validation now compares every filesystem test candidate with Vitest's complete eight-project roster, rejecting omissions, overlaps, wrong routing, unexpected files, and unexpected project names. The shared credential-free E2E workflow now applies Vitest's native e2e/credential-free tag filter in addition to the existing safe matrix scanner. The dashboard-port regression fixture also uses a coherent fake OpenShell 0.0.72 component set and an OS-assigned gateway port so the repo-wide gate stays hermetic. This is the final entry in the ordered NVIDIA#6692 stack after NVIDIA#6693, NVIDIA#6696, NVIDIA#6697, NVIDIA#6699, NVIDIA#6700, NVIDIA#6701, NVIDIA#6702, and NVIDIA#6705 landed. ## Related Issue Closes NVIDIA#6692 ## Changes - Discover broad test/spec JavaScript and TypeScript candidates under src, test, and nemoclaw/src, including nested dist, coverage, and .venv paths that Vitest would see. - Compare candidate routing with Vitest file listings and the complete roster from list-tags JSON, including empty and opt-in projects. - Produce sorted diagnostics for zero membership, overlap, wrong project, unsupported candidates, unexpected listed files, and missing or extra projects. - Invoke Vitest portably through process.execPath and node_modules/vitest/vitest.mjs. - Require the exact native credential-free tag filter in the shared E2E workflow contract while retaining the TypeScript scanner's path, project, ID, and declaration checks. - Keep the dashboard-port exhaustion regression independent of port 8080, developer OpenShell overrides, channel selection, and network installation. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: these are contributor/test-harness safeguards; AGENTS.md now states the exhaustive project-membership contract, the E2E README already documents the native module tag, and the required documentation-writer review found no user-facing change. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent review covered the shared credential-free workflow and onboarding regression fixture; it confirmed the filter preserves the existing scanner and the fixture's child-only environment cannot reach a real OpenShell install. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub - [x] Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 46 integration tests passed across the boundary and dashboard-port files; 4 E2E-support workflow tests passed; project membership is exact for 1,537 candidates across 8 projects; CLI type-check, title style, Biome, and diff checks passed. The fixture also passed with hostile inherited OpenShell paths and dev-channel selection. - [x] Applicable broad gate passed — npm test passed 1,456 files / 16,515 tests before the final focused hardening; npm run check passed the repo-wide structural, CLI coverage, and plugin coverage gates, followed by focused reruns and normal hooks after the final child-environment pin. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] npm run docs builds without warnings (doc changes only) - [ ] Doc pages follow the style guide (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Carlos Villela <cvillela@nvidia.com>  ## Summary by CodeRabbit - **Tests** - Credential-free end-to-end test runs now execute only tests matching the credential-free tag. - Vitest project validation now checks for missing, overlapping, unsupported, wrong, zero-membership, and unexpected test-to-project assignments, plus roster mismatches. - Expanded boundary tests cover test candidate discovery, expected project mapping, Vitest invocation resolution, and listing/roster parsing. - Improved onboarding “dashboard port exhaustion” assertions for more hermetic, reliable verification. - **Documentation** - Updated testing guidance to require Vitest project globs be disjoint and exhaustive, enforced by the projects check.  --------- Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv added 9 commits July 11, 2026 18:58

test(vitest): add owned resource fixtures

177bb4a

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

test(vitest): preserve proxy startup failures

d3ff019

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

test(vitest): keep cleanup branches out of test bodies

8725d80

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

test(vitest): isolate mock and environment state

63ece97

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

test(vitest): close failed readiness probes

752dee2

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

test(vitest): improve failure and developer feedback

ea3aede

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

test(vitest): discard failed proxy owners

4b6f585

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

merge(vitest): sync owned-resource base

25b1a32

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

merge(vitest): sync isolation base

225256a

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv self-assigned this Jul 12, 2026

test(vitest): keep command fixture branches out of tests

096dba6

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv mentioned this pull request Jul 12, 2026

test(vitest): split critical-path integration suites #6702

Merged

12 of 21 tasks

cv added 3 commits July 11, 2026 22:44

test(vitest): make debug specification grammatical

419639b

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

merge(vitest): sync grammatical debug specification

7e4a6dd

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

merge(vitest): sync grammatical debug specification

e56181f

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

Base automatically changed from codex/vitest-mock-env-isolation to main July 12, 2026 06:52

merge(main): sync merged Vitest stack base

daeba61

Signed-off-by: Carlos Villela <cvillela@nvidia.com> # Conflicts: # AGENTS.md # CONTRIBUTING.md # vitest.config.ts

coderabbitai Bot reviewed Jul 12, 2026

View reviewed changes

cv merged commit 6d288e6 into main Jul 12, 2026
57 checks passed

cv deleted the codex/vitest-failure-feedback-diagnostics branch July 12, 2026 07:08

github-actions Bot added the v0.0.81 label Jul 12, 2026

cv mentioned this pull request Jul 12, 2026

test(vitest): enforce exact discovery and credential-free tags #6707

Merged

11 of 21 tasks

wscurran added the area: ci CI workflows, checks, release automation, or GitHub Actions label Jul 30, 2026

Navigation Menu

Repository navigation

test(vitest): improve failure and developer feedback - #6701

test(vitest): improve failure and developer feedback#6701
cv merged 14 commits into
mainfrom
codex/vitest-failure-feedback-diagnostics

cv commented Jul 12, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 12, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

github-code-quality Bot commented Jul 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 12, 2026 •

edited

Loading

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Uh oh!

github-actions Bot commented Jul 12, 2026 •

edited

Loading

Uh oh!

cv commented Jul 12, 2026

Uh oh!

github-actions Bot commented Jul 12, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

cv commented Jul 12, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Type of Change

Quality Gates

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

github-code-quality Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

TypeScript / code-coverage/plugin

TypeScript / code-coverage/cli

Uh oh!

github-actions Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Uh oh!

github-actions Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor — No blocking findings

Uh oh!

cv commented Jul 12, 2026

Uh oh!

github-actions Bot commented Jul 12, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cv commented Jul 12, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 12, 2026 •

edited

Loading

github-code-quality Bot commented Jul 12, 2026 •

edited

Loading

github-actions Bot commented Jul 12, 2026 •

edited

Loading

github-actions Bot commented Jul 12, 2026 •

edited

Loading