Skip to content

[automated] fix(ci): select installer jobs and loose CI dependencies correctly - #19687

Merged
Ankit Jain (radical) merged 3 commits into
mainfrom
ankj/conditional-test-selector-audit
Aug 28, 2026
Merged

Ankit Jain (radical) merged 3 commits into
mainfrom
ankj/conditional-test-selector-audit

Conversation

@radical

Copy link
Copy Markdown
Member

Description

[automated] Conditional-test audit results can currently miss gated installer jobs, force unnecessary full matrices, and present schedule/outerloop-only targets as PR-runnable work. For example, an edit to the reusable installer workflow selects Infrastructure.Tests but not the WinGet or Homebrew jobs it implements. Several loose CI inputs instead report:

run-all fallback: 'eng/generate-catalog.ps1' is neither Layer-1-owned nor matched by a Layer 2 rule

The curated map did not cover every runtime script/config dependency, and the audit renderer treated every selected token as work that PR enforcement could run.

This change routes confirmed loose inputs to their actual consumers, makes CI-wide inputs explicit ALL, and explicitly accounts for paths already exercised by baseline or dedicated workflows as well as paths with no PR-CI consumer. Audit comments and job summaries now separate PR-gated work from advisory workflow impact that is scheduled independently of the regular selector gates.

Repository-wide code-review guidance now prompts reviewers to check selector routing when PRs add test projects, CI jobs/workflows, or loose CI inputs, while avoiding manual mappings for ProjectGraph-owned .NET changes.

Native CLI archive verifier scripts remain in the main CI workflow but no longer select tests because the unconditional archive jobs already execute them. The Windows-only template catalog generator and schedule-only skills updater are classified outside PR CI. The cross-platform CLI starter validator is also outside selector targets because unconditional PR jobs already execute it. The bundle verifier remains covered by its dedicated PR workflow. Root VS Code settings and Pyright configuration skip the main matrix because they have no CI consumer; root VS Code launch and task configuration route to the extension unit job. Copilot canvas extension changes skip the main matrix and continue through their dedicated validation workflow.

Selector enforcement remains disabled (enforce: 'false'). Unowned source and shared-file changes still fail safe to ALL.

Validation:

dotnet test --project tests/Infrastructure.Tests/Infrastructure.Tests.csproj --no-launch-profile -- --filter-namespace "Infrastructure.Tests.TestTriggerMap" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"
Passed: 154, Failed: 0, Skipped: 0

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Copilot AI balanced review requested due to automatic review settings August 25, 2026 23:19
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 19687

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 19687"

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refines conditional CI selection so installer jobs and loose dependencies route correctly, while schedule/outerloop-only targets remain advisory.

Changes:

  • Adds precise routing and skip classifications for CI inputs.
  • Separates PR-runnable work from advisory workflow impact.
  • Expands selector regression tests and maintenance guidance.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tools/SelectTests/TestSelector.cs Updates selector terminology.
tools/SelectTests/Program.cs Separates PR and advisory targets in reports.
tests/Infrastructure.Tests/TestTriggerMap/TestTriggerMapTests.cs Verifies real-map routing and target vocabulary.
tests/Infrastructure.Tests/TestTriggerMap/SelectTestsCliTests.cs Tests advisory report rendering.
tests/Infrastructure.Tests/TestTriggerMap/SelectTestsAcceptanceTests.cs Tests new prefilter classifications.
eng/github-ci/test-trigger-map.yml Adds and corrects CI dependency routing.
eng/github-ci/ci-skip-entirely-patterns.txt Excludes independently validated or inert files.
docs/ci/test-trigger-selector-design.md Documents advisory-target behavior.
docs/ci/test-trigger-map.md Expands map maintenance guidance.
AGENTS.md Adds conditional-selection review guidance.
.agents/skills/code-review/SKILL.md Incorporates selector checks into reviews.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@radical
Ankit Jain (radical) force-pushed the ankj/conditional-test-selector-audit branch from a64e3be to 9477701 Compare August 25, 2026 23:35
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 11/11 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Copilot AI review requested due to automatic review settings August 26, 2026 23:55
@radical
Ankit Jain (radical) force-pushed the ankj/conditional-test-selector-audit branch from c84a5d9 to d7e514b Compare August 26, 2026 23:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Ankit Jain (radical) and others added 3 commits August 28, 2026 01:42
Loose CI inputs could fall through to ALL or miss gated jobs, and audit
output mixed regular PR work with independently scheduled targets. A
broad skip pattern could also suppress CI for root VS Code launch and
task changes consumed by extension tests.

Route Layer 2 inputs to exact consumers or explicit classifications,
separate advisory targets in comments and summaries, and add exact-set
regressions for under- and over-selection. Keep enforcement disabled.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0a785829-bac9-47ca-9fcf-34019bf2950e
The selector documentation had stale job tokens and ambiguous ALL and
advisory semantics, leaving future changes without a reliable procedure
for classifying loose CI inputs.

Document the executable target vocabulary, independently scheduled work,
the Layer 1 versus Layer 2 decision process, and the focused validation
needed when routing rules change.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0a785829-bac9-47ca-9fcf-34019bf2950e
Pull request reviews did not prompt reviewers to check selector routing
when changes add test projects, CI jobs, reusable workflows, or loose
inputs consumed outside the ProjectGraph.

Add concise repository-wide and review-skill guidance for tracing real
consumers, preserving run_* wiring, and requiring focused selector
regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0a785829-bac9-47ca-9fcf-34019bf2950e
Copilot AI review requested due to automatic review settings August 28, 2026 05:42
@radical
Ankit Jain (radical) force-pushed the ankj/conditional-test-selector-audit branch from d7e514b to 8ee71d0 Compare August 28, 2026 05:42
@github-actions

Copy link
Copy Markdown
Contributor

Tests selector (audit mode)

The regular PR test matrix and PR-gated jobs still run in audit mode. The PR tests and jobs below are what selective CI would run under enforcement. Advisory-only targets are reported here but are not scheduled through the regular PR matrix or job gates; independent workflows may still run them for a PR.

1 / 99 PR test projects · 0 PR jobs · 0 advisory-only targets, from 7 changed files.

Selected PR test projects (1 / 99)

Infrastructure.Tests

Selected PR jobs (0)

none

Advisory workflow impact (0)

none


How these were chosen — grouped by what changed

📄 eng/github-ci/ci-skip-entirely-patterns.txt (changed)
→ 1 directly: Infrastructure.Tests

📄 eng/github-ci/test-trigger-map.yml (changed)
→ 1 directly: Infrastructure.Tests

🧪 tests/Infrastructure.Tests/TestTriggerMap/SelectTestsAcceptanceTests.cs (changed test)
→ 1 directly: Infrastructure.Tests

🧪 tests/Infrastructure.Tests/TestTriggerMap/SelectTestsCliTests.cs (changed test)
→ 1 directly: Infrastructure.Tests

🧪 tests/Infrastructure.Tests/TestTriggerMap/TestTriggerMapTests.cs (changed test)
→ 1 directly: Infrastructure.Tests

📄 tools/SelectTests/Program.cs (changed)
→ 1 directly: Infrastructure.Tests

📄 tools/SelectTests/TestSelector.cs (changed)
→ 1 directly: Infrastructure.Tests

Job reasons

none


Selection computed for commit 8ee71d0.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

@radical
Ankit Jain (radical) merged commit 02d805c into main Aug 28, 2026
384 checks passed
@radical
Ankit Jain (radical) deleted the ankj/conditional-test-selector-audit branch August 28, 2026 05:54
@github-actions github-actions Bot added this to the 13.6 milestone Aug 28, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-engineering-systems infrastructure helix infra engineering repo stuff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants