Repository navigation
test(e2e): add setup_scenarios aliases for 13 layered test_plans (epic #3588 Phase 4) #4378
Description
Summary
Add setup_scenarios aliases in test/e2e-scenario/nemoclaw_scenarios/scenarios.yaml for the 13 layered test_plans that currently have no user-friendly scenario ID. This unblocks deletion of the legacy scenarios/run.ts TS framework and removes a hidden coupling where the only thing that knew these IDs was the deleted listScenarios() registry.
This is Phase 4 (Onboarding Matrix Expansion) follow-up work for epic #3588. The onboarding profiles and test_plans already exist; only the user-friendly aliases are missing.
Background
While cleaning up the obsolete test/e2e-scenario/scenarios/run.ts runner (replaced by test/e2e-scenario/runtime/run-scenario.sh), we found that the OLD TS registry (scenarios/scenarios/baseline.ts) defined 23 user-friendly scenario IDs, but the new nemoclaw_scenarios/scenarios.yaml only declares 10 under setup_scenarios. The remaining 13 exist only as test_plans with internal __-form IDs.
Several places still reference the missing user-friendly IDs and currently rely on the OLD registry to resolve them:
.github/workflows/e2e-scenarios.yaml—ROUTEStable maps these IDs to runner types.test/e2e-scenario-advisor.test.ts— advisor test assertsubuntu-repo-cloud-openclaw-telegramresolves.tools/e2e-advisor/scenarios.mts— enumerates scenarios for the advisor.
Missing user-friendly IDs (13)
Each maps 1:1 to an existing test_plans entry via alias_for_plan:
| Missing setup_scenarios id | Existing test_plans id |
|---|---|
ubuntu-repo-cloud-openclaw-brave |
ubuntu-repo-docker__cloud-nvidia-openclaw-brave |
ubuntu-repo-cloud-openclaw-telegram |
ubuntu-repo-docker__cloud-nvidia-openclaw-telegram |
ubuntu-repo-cloud-openclaw-discord |
ubuntu-repo-docker__cloud-nvidia-openclaw-discord |
ubuntu-repo-cloud-openclaw-slack |
ubuntu-repo-docker__cloud-nvidia-openclaw-slack |
ubuntu-repo-cloud-hermes-discord |
ubuntu-repo-docker__cloud-nvidia-hermes-discord |
ubuntu-repo-cloud-hermes-slack |
ubuntu-repo-docker__cloud-nvidia-hermes-slack |
ubuntu-repo-cloud-openclaw-resume |
ubuntu-repo-docker__cloud-nvidia-openclaw-resume-after-interrupt |
ubuntu-repo-cloud-openclaw-repair |
ubuntu-repo-docker__cloud-nvidia-openclaw-repair-existing-config |
ubuntu-repo-cloud-openclaw-double-same-provider |
ubuntu-repo-docker__cloud-nvidia-openclaw-double-same-provider |
ubuntu-repo-cloud-openclaw-double-provider-switch |
ubuntu-repo-docker__cloud-nvidia-openclaw-double-provider-switch |
ubuntu-repo-cloud-openclaw-token-rotation |
ubuntu-repo-docker__cloud-nvidia-openclaw-token-rotation |
ubuntu-repo-openai-compatible-openclaw |
ubuntu-repo-docker__openai-compatible-openclaw |
(That's 12 in the table — the 13th depends on whether ubuntu-repo-cloud-openclaw and the negative-path IDs are reconciled separately. Confirm during implementation.)
Acceptance criteria
- Each ID above exists as a
setup_scenarios:entry withalias_for_plan: <plan-id>. -
bash test/e2e-scenario/runtime/run-scenario.sh <each-id> --plan-onlysucceeds for all 13 IDs. -
.github/workflows/e2e-scenarios.yamlROUTEStable is unchanged (every ID still maps). -
tools/e2e-advisor/scenarios.mtsenumerates these IDs from YAML (no TS registry dependency). -
test/e2e-scenario-advisor.test.tspasses against the YAML resolver (currently fails onubuntu-repo-cloud-openclaw-telegram). - Framework-tests under
test/e2e-scenario/framework-tests/continue to pass.
Context — companion cleanup
This issue is the prerequisite for landing the cleanup PR that deletes the obsolete run.ts path:
- Deletes
test/e2e-scenario/scenarios/(entire OLD TS framework, 33 files) - Deletes
test/e2e-scenario/manifests/(only consumed by OLD framework) - Deletes
test/e2e-scenario/onboarding_assertions/(dead in NEW path) - Deletes 6 framework-tests that only test the OLD framework
- Switches
.github/workflows/e2e-scenarios.yamlfromnpx tsx ...run.tstobash ...run-scenario.sh
That cleanup PR can land now (with the advisor test deferred). Once this issue lands, the advisor test will pass against the YAML resolver and the migration is fully complete.
Parent
Epic #3588 (Phase 4: Onboarding Matrix Expansion).
Activity
ci(e2e-scenarios): align upload paths with run-scenario.sh outputs
refactor(e2e-advisor): align e2e scenario advisor to better follow e2…
cv commentedon Jun 2, 2026
Tracking update: opened PR #4649 to remove repo-local E2E migration state and keep mutable migration status in GitHub issues/PRs instead.
This issue remains the right place to track the setup-scenario alias cleanup and acceptance state; the repo docs will point maintainers here rather than carrying a duplicate alias/migration checklist.
docs(e2e): keep migration state in issues (#4649)
test(e2e): add scenario aliases for layered plans (#4651)