Gate the CLI starter validation jobs on the test selector - #19807
Conversation
Co-authored-by: radical <1472+radical@users.noreply.github.com>
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 19807Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 19807" |
There was a problem hiding this comment.
Pull request overview
Routes six cross-platform CLI starter validation jobs through selective CI.
Changes:
- Adds the
job:cli-starter-validationselector target and routing rules. - Gates workflow jobs and result checks on the selector output.
- Updates selector tests and documentation.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
.github/workflows/tests.yml |
Adds selector-based job gates. |
eng/github-ci/test-trigger-map.yml |
Defines starter-validation routing. |
tests/Infrastructure.Tests/TestTriggerMap/TestTriggerMapTests.cs |
Tests target and gate wiring. |
docs/ci/test-trigger-map.md |
Documents the new target. |
Suppressed comments (1)
eng/github-ci/test-trigger-map.yml:419
- The new curated project routing is not exercised by a real-map behavior test. The added tests cover the script path and YAML gate binding, while
EveryAffectedProjectRuleGlobMatchesASolutionProjectonly proves that each name matches some project; it would still pass if a required project were omitted or a leaf such asAspire.Hosting.Rediswere added, defeating this PR's selection contract. Add positive Layer 1 cases for representative listed projects and the documented negative Redis case.
- projects:
- Aspire.Cli
- Aspire.TypeSystem
- Aspire.Managed
- Aspire.AppHost.Sdk
This comment has been minimized.
This comment has been minimized.
Co-authored-by: radical <1472+radical@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (3)
Previously missed (1) — in code that hasn't changed since the last review.
eng/github-ci/test-trigger-map.yml:25
job:cli-starter-validationis not added to theCLI_BUNDLEgroup at lines 33-39. Consequently, aneng/Bundle.proj-only change selects the existing bundle consumers but leavesrun_cli_starter_validationfalse, even though these jobs install that bundle. Add this target toCLI_BUNDLEand covereng/Bundle.projwith a real-map assertion.
This issue also appears in the following locations of the same file:
- line 246
- line 415
# job:cli-starter-validation tests.yml cli_starter_validation_* (6 platforms)
eng/github-ci/test-trigger-map.yml:250
- This routing only covers the workflow and validation script, but not several loose inputs the job executes:
src/Aspire.ProjectTemplates/**,src/Aspire.Cli/Templating/Templates/ts-starter/**,eng/dashboardpack/**, andeng/dcppack/**. Their existing rules select Templates/Hosting SDK/TerminalHost or TypeScript SDK tests, not either CLI test keyed by the new derived rule, so those changes can leave all six starter jobs skipped. Route these paths tojob:cli-starter-validationand add exact real-map cases.
- paths:
- .github/workflows/cli-starter-validation.yml
- eng/scripts/cli-starter-validation.ps1
targets: [job:cli-starter-validation]
reason: the reusable workflow and the script it runs ARE the starter validation
eng/github-ci/test-trigger-map.yml:417
- Deriving the job only from selected CLI tests does not implement the required production-project triggers. For example,
Aspire.DashboardandAspire.Hosting.PostgreSQLdo not flow to either CLI test through ProjectReferences, so changes to the dashboard launched by every run or the PostgreSQL package added by the script can skip this validation. Add the intended exactaffected_project_rulesentry (including the other starter dependencies) and focused real-map coverage.
- tests: [test:Aspire.Cli.Tests, test:Aspire.Cli.EndToEnd.Tests]
targets: [job:cli-starter-validation]
reason: starter validation follows the same trigger set as the CLI and CLI end-to-end tests
This comment has been minimized.
This comment has been minimized.
CLI starter validation was derived only from CLI unit and end-to-end tests. Changes to the C# starter, AppHost/PostgreSQL path, packaged DCP/dashboard SDK, and CreateLayout could therefore skip all six platform jobs. Derive the job from stable consumer tests instead of a production-project allowlist. Include it in CLI_BUNDLE and route CreateLayout through that group. Add real-map tests for each consumer class while keeping unrelated leaf integrations excluded. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5289c147-1640-420e-9f36-10024c2af62e
This comment has been minimized.
This comment has been minimized.
Selected test projects are not reliable proxies for the starter scenarios. They omit some direct runtime inputs while also aggregating unrelated templates, integrations, client libraries, and test-only dependencies. Route starter validation through narrow affected-project rules for the direct production artifacts and exact path rules for package, template, and workflow inputs. Layer 1 still supplies transitive closure behind those stable boundaries. Add positive coverage for every direct scenario input and negative coverage for unrelated integrations, templates, clients, and test infrastructure. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5289c147-1640-420e-9f36-10024c2af62e
There was a problem hiding this comment.
Note
Copilot could not run the full agentic suite for this review because it was automatically requested on a bot-authored pull request. Request a review from Copilot under Reviewers to retry with the full agentic suite. Improved support for bot-authored pull requests is coming soon.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
This comment has been minimized.
This comment has been minimized.
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
The starter archive-path invariant scraped YAML and embedded JSON with format-sensitive regexes and deferred assertions. This could fail after harmless formatting changes or silently inspect only the first matching archive dependency or RID. Parse the workflow with YamlDotNet and the targets input with System.Text.Json, require exactly one archive dependency and target per starter job, and collect paths eagerly for clearer diagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5289c147-1640-420e-9f36-10024c2af62e
This comment has been minimized.
This comment has been minimized.
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
The final-results gate repeated the same selector output on every starter platform. Group the six skipped-result checks behind one selection guard while preserving the explicit platform list. Update the regression test to pin the complete grouped condition and require the selection guard to appear exactly once. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5289c147-1640-420e-9f36-10024c2af62e
Tests selectorSelects the full PR test matrix + all PR-gated jobs (ALL) — a rule matching '.github/workflows/tests.yml' selects ALL Advisory workflow impact (4)
Selection computed for commit |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
|
The CI build failed due to test failure(s) that appear unrelated to the PR changes. These may be flaky tests. Suspected flaky test(s): Suggested actions:
You can re-run the failed jobs from the workflow run page. |
[automated] ## Description
The six cross-platform CLI starter validation jobs previously ran on every pull request. They now participate in selective CI and run only when a changed consumer can affect the starter scenarios.
The selector exposes
job:cli-starter-validation, threads its boolean throughsetup_for_tests, gates all six platform jobs, and treats a skipped job as an error only when the selector requested it.The trigger follows stable direct artifact boundaries instead of selected test projects:
eng/Bundle.proj, and the validation workflow, scenario script, and PR installer.CreateLayoutandeng/Bundle.projroute throughCLI_BUNDLE, which now includes starter validation.Selected test projects are intentionally not used as proxies. Their dependency graphs also include unrelated templates, Azure/browser integrations, Npgsql client code, and test-only utilities or fakes, which would fan unrelated changes into all six platform jobs.
Layer 1 still supplies transitive dependency closure behind each direct production boundary, avoiding a granular list of implementation projects. Core Hosting changes therefore reach starter validation, while unrelated Redis, Npgsql client, Azure App Containers, xUnit template, npm-launcher, native-symbol packaging, and test-infrastructure changes remain excluded.
The real-map tests pin the direct runtime inputs, loose package and workflow inputs, executable CLI archive boundaries, bundle construction, and unrelated-change exclusions. A workflow invariant also requires every platform's skipped-result check to remain coupled to
run_cli_starter_validation.Checklist
<remarks />and<code />elements on your triple slash comments?