azure-pipelines: enable on tag push, port validation, and tune the new release pipeline by dscho · Pull Request #914 · microsoft/git

dscho · 2026-05-12T21:21:41Z

Switch microsoft/git releases over to the Azure Pipeline at .azure-pipelines/release.yml (added in #913) and step out of the way of the legacy build-git-installers GitHub Actions workflow, for which we no longer have a permissible code-signing certificate.

The series breaks into four blocks:

Make the Windows build honour the resolved `git_version`

The Windows mingw-w64-git build picked up the wrong artifact name on non-release tags, and on tags using a different vfs prefix it aborted inside GIT-VERSION-GEN's prefix check. Side-step both, and drop a separately built source tarball that nothing downstream consumes.

Port artifact validation from `build-git-installers`

The release stage's validate_* jobs were TODO-stubbed since the initial port. Fill them in to mirror the legacy workflow: silently install the artifact on each OS and assert git --version reports the tag, plus a universal-binary CPU check on macOS.

Flip the trigger on, demote the old workflow

With everything in place, switch the Azure Pipeline to fire on v[0-9]*vfs* tag pushes and turn ESRP and GitHub-release publishing on by default. Demote build-git-installers to workflow_dispatch only so the two pipelines do not race over the same draft release.

Performance and reliability fixes

Parallelise the I/O-bound Windows SDK sparse checkout and mingw-w64-git build; work around the dpkg lock that unattended-upgrades holds on freshly booted Linux agents; fix %PROGRAMFILES% mis-resolving under the ARM64 agent's emulated PowerShell; and fold each OS's validate job into the corresponding build job since the per-job 1ES overhead dwarfed the actual work.

mjcheetham

The commit message for the first commit (azure-pipelines: make the Windows mingw-w64-git build honour git_version) talks about historic versions of 'this commit', but that doesn't really help future readers since that previous version doesn't exist.

It also makes reference to "pipeline run #188" which is being tagged against a GitHub PR. Avoid linking to ADO runs as examples of "this failed here" since they are often culled.

mjcheetham · 2026-05-13T14:17:04Z

+                    # please.sh's `create-sdk-artifact` step does the
+                    # final sparse-checkout of the build-installers
+                    # SDK subset, which is I/O-bound (lots of small
+                    # writes), not CPU-bound. The default
+                    # checkout.workers=1 leaves the agent's I/O
+                    # subsystem mostly idle; bumping it well beyond
+                    # the core count gives a substantial speedup.
+                    GIT_CONFIG_PARAMETERS: "'checkout.workers=56'"


Why not just update the script? It's only called by Azure Pipelines in this build, and there's no downside to a parallel checkout here. Normal devs will not run the setup-git-sdk script.

I want to try and keep the YAML as simple as possible.

The problem is that the please.sh script is intended not only for automation, but also for human use. And it cannot really know whether an insane 56 checkout workers is really appropriate in every instance it is called.

I mean update the setup-git-sdk script to set that envar for please.. that is only called by automation :)

mjcheetham · 2026-05-13T14:24:32Z

+                # Validate the freshly built installer in-place: silently
+                # install Git-*.exe and assert that `git --version` reports
+                # the version we resolved at the prereqs stage. Folded into
+                # the build job so it runs on the same agent without the
+                # 1ES job-startup overhead a separate validate job carries.


Part of the reason for keeping validation jobs on a fresh agent was to replicate the minimal environment we expect this to work in. This build job has installed and the Git SDK and dependencies.. we want to make sure install works on a clean machine.

FWIW we still save lots of 1ES crud by not checking out the source repo.
There is also the 'validationJob' type that we can set for these to further reduce extra crud (but the 1ES PT team need to fix it first).

True. But then there is still the substantial overhead (the job, which you can access here, took 3m34s, where installing Git took 27s and validating it took 1s). I really hate that overhead. Also, I dislike how every validation has to wait for all of the build jobs to finish... Why can't the Linux validation run once the Linux build is done?

Why can't the Linux validation run once the Linux build is done?

We can split out the build steps into explicit (rather than matrix) jobs: build_windows, build_mac, build_linux.. then have explicit OS-validation jobs that depend on those too.

validation_linux <- build_linux

When building on Windows, the version comes from an annotated tag at HEAD, falling back to a `git describe + timestamp` string when no such tag exists. Real release runs already have the tag in place via the tag-push trigger; manual debug runs do not. Create the `v$(git_version)` tag ourselves so the artifact name matches the pipeline-resolved version. The tag alone is not enough: microsoft/git's GIT-VERSION-GEN refuses any version not based on `DEF_VER`, so debug tags using a different vfs prefix would still abort the build. GIT-VERSION-GEN also reads a `version` file at the source-tree root in preference to running `git describe`, and the prefix check only fires on the describe path, so planting that file side-steps both. The file has to live in the directory the build compiles from. Make that directory a worktree of the agent's checkout so refs and objects are shared without duplicating the source, and override `remote.origin.url` per-worktree (via `extensions.worktreeConfig`) so the build's unconditional `git fetch` stays on local disk rather than hitting the Azure-supplied remote. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

…e on Windows `--build-src-pkg` makes please.sh produce a source tarball alongside the binary mingw-w64-git packages. Nothing downstream consumes it: the tarball is not bundled into the installer, not signed, not published as an artifact, and not attached to the GitHub release. Drop the flag. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Fill in the `validate_windows_*` jobs in the release stage, still TODO-stubbed since the initial port of the build-git-installers workflow. Mirror the legacy workflow's check: silently install the .exe and assert that `git --version` matches the tag. Validation only consumes the pre-built artifact, so skip the implicit `checkout: self` to avoid cloning the entire Git history just to throw the working tree away. The macOS and Linux validate jobs are filled in by subsequent commits. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Continue the validation port started in the previous commit. Mirror the legacy build-git-installers workflow on macOS: install the .pkg and assert that `git --version` matches the tag. Two extras vs. the Windows port: on Apple Silicon agents, Homebrew's `/opt/homebrew/bin/git` appears earlier on PATH than the pkg-installed `/usr/local/bin/git`, so `brew uninstall git` before the install is needed for the subsequent `git --version` to actually exercise our artifact; and check that `git version --build-options` reports `cpu: $(uname -m)` so the universal binary runs natively rather than under Rosetta. The Linux validate jobs are filled in by the next commit. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Finish the validation port started two commits ago. Mirror the legacy build-git-installers workflow on Linux: install the .deb and assert that `git --version` matches the tag. Use `apt-get install -y` rather than the legacy workflow's `apt install`, which would otherwise pause for the y/N prompt; run `apt-get update` first so dependency resolution sees the agent's current package index. With this commit all three OS validate jobs run real artifact checks before the GitHub draft release publishing job downstream of them fires, completing the build-git-installers workflow migration. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

With build, signing, notarization, validation, and draft-release publishing all in place, the Azure Pipeline is ready to take over from the GitHub Actions build-git-installers workflow. Switch `trigger: none` to a tag-only trigger matching the same `v[0-9]*vfs*` pattern the GitHub workflow used (and that resolve-version.sh validates against), and explicitly exclude all branches so the pipeline does not fire on every topic-branch push. Flip the `esrp` and `github` parameter defaults from false to true. The GitHub release job still uses `isDraft: true`, so a maintainer inspects and publishes the release manually; manual runs in the ADO UI can still uncheck either box for a dry run. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

The Azure Pipeline at .azure-pipelines/release.yml now builds and signs the official microsoft/git installers on every `v[0-9]*vfs*` tag push. Two release pipelines firing on the same tag would race each other uploading assets to the same draft GitHub release, so this workflow has to step out of the way. We also no longer have access to a code-signing certificate this workflow could use, so even on its own it would attach unsigned (or improperly signed) installers, which we must not ship. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Initialising the Git for Windows SDK sparse-checks out thousands of small files (the MinGW toolchain plus the MSYS2 userland). The work is firmly I/O-bound, not CPU-bound, so the default of one checkout worker leaves most of the agent's I/O subsystem idle. Raise the worker count to 56, matching what git-for-windows/git-sdk-64's own CI uses for the same operation; deliberately well above the agent's CPU count for the I/O-bound reason above. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

The mingw-w64-git build invokes `make` without an explicit `-j`, leaving the bulk of the per-file work (perl scripts, doc tree, contrib targets, per-.c compilation) serial on a single core. As with the SDK sparse checkout in the previous commit, most of the wall-clock time is spent shuffling small files in and out of the SDK's pacman cache and the build tree rather than in the compiler itself; the work is predominantly I/O-bound, so the right factor is well above the agent's core count rather than `$(nproc)`. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

The 1ES Ubuntu agents come up with `unattended-upgrades` already running, which holds `/var/lib/dpkg/lock-frontend` for the first few minutes after boot. The Linux build and validate jobs' `apt-get` invocations therefore intermittently fail with E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process <pid> (unattended-upgr) on what is genuinely just a timing race. Tell apt to poll for the lock with a generous timeout rather than failing immediately, turning the race into a backoff and avoiding wrapping each apt invocation in an ad-hoc retry loop. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

The Windows validate job looked up the installed `git.exe` under `%PROGRAMFILES%\Git\cmd\git.exe`. On the ARM64 1ES agent that resolved to `C:\Program Files (x86)\Git\cmd\git.exe`, where the installer never puts anything, and the step failed. `%PROGRAMFILES%` is per-process: Windows remaps it to `C:\Program Files (x86)` for any 32-bit-emulated process, and the ARM64 hosted agent's classic PowerShell evidently runs under emulation. `%ProgramW6432%` is defined on every 64-bit Windows and always points at the native `C:\Program Files`, regardless of the calling process's bitness. The Inno Setup installer installs into `{pf}` -> `C:\Program Files\Git`, so the swap works on both x64 and ARM64. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

1ES pipeline templates add substantial fixed overhead per job (image acquisition, SDL gating, artifact-staging plumbing, job tear-down). For the Windows validate job that runs only a few seconds of PowerShell against the installer it just built, that overhead dwarfs the work itself. Unlike GitHub Actions, Azure Pipelines provides little benefit from re-running an individual failed job in isolation, so the per-OS separation between build and validate buys nothing here. Fold the install + version-check steps into the corresponding build job and drop the separate validate job. The macOS and Linux validate jobs are folded in subsequent commits. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

For the same reason as the Windows fold in the previous commit, the per-job 1ES overhead is not worth paying for the few seconds it takes to install the .pkg, check the reported version, and confirm the universal binary's CPU. Fold the validation steps into the macOS build job and drop the separate validate job. The Linux validate jobs are folded in the next commit. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

For the same reason as the Windows and macOS folds in the previous two commits, the per-job 1ES overhead is not worth paying for the few seconds it takes to install the .deb and check the reported version. Fold the validation steps into the Linux build job and drop the separate validate job; the fold reuses the dpkg-lock work-around the build-deps step already needs. The release stage's only remaining job after this is `github`, whose ordering is already provided by the release stage's own dependsOn entry on the build stage. Assisted-by: Claude Opus 4.7 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

dscho marked this pull request as draft May 13, 2026 13:46

mjcheetham reviewed May 13, 2026

View reviewed changes

dscho added 14 commits May 14, 2026 10:31

dscho force-pushed the turn-off-build-git-installers branch from 085afc2 to 967ec96 Compare May 14, 2026 09:46

dscho marked this pull request as ready for review May 14, 2026 16:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

azure-pipelines: enable on tag push, port validation, and tune the new release pipeline#914

azure-pipelines: enable on tag push, port validation, and tune the new release pipeline#914
dscho wants to merge 14 commits into
vfs-2.53.0from
turn-off-build-git-installers

dscho commented May 12, 2026 •

edited

Loading

Uh oh!

mjcheetham left a comment

Uh oh!

mjcheetham May 13, 2026

Uh oh!

dscho May 14, 2026

Uh oh!

mjcheetham May 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

mjcheetham May 13, 2026

Uh oh!

dscho May 14, 2026

Uh oh!

mjcheetham May 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dscho commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Make the Windows build honour the resolved git_version

Port artifact validation from build-git-installers

Flip the trigger on, demote the old workflow

Performance and reliability fixes

Uh oh!

mjcheetham left a comment

Choose a reason for hiding this comment

Uh oh!

mjcheetham May 13, 2026

Choose a reason for hiding this comment

Uh oh!

dscho May 14, 2026

Choose a reason for hiding this comment

Uh oh!

mjcheetham May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mjcheetham May 13, 2026

Choose a reason for hiding this comment

Uh oh!

dscho May 14, 2026

Choose a reason for hiding this comment

Uh oh!

mjcheetham May 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dscho commented May 12, 2026 •

edited

Loading

Make the Windows build honour the resolved `git_version`

Port artifact validation from `build-git-installers`

mjcheetham May 14, 2026 •

edited

Loading