Skip to content

ci: streamline test job fan-out - #19758

Closed
Ankit Jain (radical) wants to merge 2 commits into
microsoft:mainfrom
radical:radical-streamline-ci-workflows
Closed

Ankit Jain (radical) wants to merge 2 commits into
microsoft:mainfrom
radical:radical-streamline-ci-workflows

Conversation

@radical

@radical Ankit Jain (radical) commented Aug 27, 2026 •

Copy link
Copy Markdown
Member

[automated]

Description

PR CI now uses focused extension validation and five stable test-matrix lanes. This makes selector-driven skips and job names easier to interpret, while trusted extension release PRs avoid starting the full test workflow.

Root cause. Extension unit testing and VSIX packaging were embedded in tests.yml, so the release fast path had to thread an extensionReleaseOnly mode through the full workflow. The matrix splitter also maintained a sixth overflow lane even though the measured no-package matrix has 204 entries.

The fix. Extension unit testing and optional packaging now live in extension-unit-tests.yml, called directly by normal and release CI. Packaging still runs after unit-test failures when extension E2E needs the VSIX. The splitter emits five dependency lanes and fails explicitly if any lane exceeds GitHub's 256-job limit, rather than silently dropping tests. Static OS-qualified lane names remain distinct in CI timeline rendering, and selector routing covers both extension unit and E2E consumers.

Why the skipped extension-release job remains. It is the pre-existing, mutually exclusive fast path for trusted bot-authored extension release PRs. Ordinary PRs show it as skipped because GitHub renders conditional jobs in the check graph, but it consumes no runner time.

Coverage is unchanged. Normal PR CI still covers the rolling-build test set and adds the existing PR-only CLI E2E and starter-validation checks. The selector and enumeration criteria are unchanged; the passing run executed all 313 matrix entries exactly once plus all 34 extension E2E shards.

Validation.

  • PR CI run 33117080669 passed for commit cccd63256d.
  • The run executed all 313 matrix entries exactly once: 204 no-package, 12 Linux package, 12 Windows package, 0 macOS package, and 85 CLI-archive entries, with no overflow jobs.
  • Extension unit tests, both VSIX package assertions, artifact upload, and all 34 extension E2E shards passed; every E2E shard downloaded the VSIX successfully.
  • 71 focused workflow, selector, and matrix contracts, 10 focused extension production-gate tests, and 669 broader Infrastructure.Tests passed locally.
  • actionlint reported no new diagnostics in the changed workflows.

Fixes # (issue)

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Move VS Code extension unit testing and optional VSIX packaging into a
focused reusable workflow. This lets trusted extension release PRs avoid
the full tests workflow while preserving the artifact needed by
extension E2E jobs after unit-test failures.

Replace the overflow matrix lane with a hard 256-entry limit so tests can
never be silently omitted. Give package-dependent lanes stable
OS-qualified names and route changes to the new workflow through the test
selector.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 062c6fdf-bd7d-4ceb-926e-ebe8a10eecda
Copilot AI balanced review requested due to automatic review settings August 27, 2026 20:38
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 19758

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 19758"

@github-actions github-actions Bot added the area-engineering-systems infrastructure helix infra engineering repo stuff label Aug 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Streamlines CI by separating extension validation and consolidating test fan-out into five bounded lanes.

Changes:

  • Extracts extension unit tests and optional VSIX packaging into a reusable workflow.
  • Removes the overflow lane and fails matrices exceeding GitHub’s 256-job limit.
  • Updates routing, documentation, job names, and contract tests.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file
File Description
.github/workflows/ci.yml Calls focused extension validation for release PRs.
.github/workflows/tests.yml Uses five named test lanes and the extracted extension workflow.
.github/workflows/extension-unit-tests.yml Adds reusable extension unit testing and optional packaging.
eng/scripts/split-test-matrix-by-deps.ps1 Removes overflow splitting and enforces lane limits.
eng/scripts/expand-test-matrix-github.ps1 Updates matrix-splitting documentation.
eng/github-ci/test-trigger-map.yml Routes extension workflow changes to unit and E2E validation.
tests/Infrastructure.Tests/WorkflowScripts/ExtensionReleaseFastPathWorkflowTests.cs Verifies workflow extraction, callers, permissions, and lane names.
tests/Infrastructure.Tests/TestTriggerMap/TestTriggerMapTests.cs Verifies extension workflow routing.
tests/Infrastructure.Tests/PowerShellScripts/SplitTestMatrixByDepsTests.cs Tests five-lane output and limit failures.
tests/Infrastructure.Tests/PowerShellScripts/ExpandTestMatrixGitHubTests.cs Updates end-to-end matrix assertions.
tests/Infrastructure.Tests/Pipelines/ExtensionE2eWorkflowTests.cs Verifies VSIX production despite unit-test failures.
docs/ci/TestingOnCI.md Documents the five-lane model and explicit limit failure.
docs/ci/ci-pipeline-optimizations.md Removes the obsolete overflow lane.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

The extension production-gate tests still read
.github/workflows/tests.yml after VSIX packaging moved into the focused
extension workflow. This made CI fail even though packaging and artifact
upload completed successfully.

Point the assertions at extension-unit-tests.yml so they continue
guarding the workflow that owns the packaging steps.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 062c6fdf-bd7d-4ceb-926e-ebe8a10eecda
Copilot AI review requested due to automatic review settings August 27, 2026 21:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

@radical

Copy link
Copy Markdown
Member Author

[automated] Superseded by #19761, which is based on #19688 so the dedicated extension-release job can be removed in favor of enforced test selection. The replacement is registered as the top of stack #19689 and its CI passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-engineering-systems infrastructure helix infra engineering repo stuff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants