Refresh the image registry cache in a workflow that cannot be cancelled

[potiuk]

v3-3-test builds its linux/amd64 CI images from scratch, ~20 minutes per Python version, because the registry has no cache for them:

#20 importing cache manifest from ghcr.io/apache/airflow/v3-3-test/ci/python3.11:cache-linux-amd64
#20 ERROR: failed to configure registry cache importer:
    ghcr.io/apache/airflow/v3-3-test/ci/python3.11:cache-linux-amd64: not found

The registry holds cache-linux-amd64 for Python 3.10 alone on that branch, while cache-linux-arm64 exists for all five.

Why

A branch built on push refreshed its cache from inside the CI run, which cancels in progress on the next push. With no cache the build runs cold, which widens the window the next push cancels in, which leaves the cache missing. Run 30973271401 is the loop in one screenshot:

success    04:09–04:46     Build CI linux/amd64 image 3.10 … 3.14   (all five built)
success    04:58–05:17:11  Push CI Early:3.10 image cache           ← the one tag that exists
cancelled  04:58–05:17:44  Push CI Early:3.11 image cache
cancelled  04:58–05:17:43  Push CI Early:3.13 image cache
cancelled  04:58–05:17:41  Push CI Early:3.12 image cache
cancelled  04:57–05:17:44  Push CI Early:3.14 image cache

The next push landed at 05:17:26 and killed four of the five, fifteen seconds after the first finished. Of the last 60 AMD runs on that branch, 45 were cancelled and 14 failed — so the Regular cache push in finalize-tests.yml, which sits behind the whole test matrix, never runs there either.

A job cannot opt out of its run being cancelled, so the refresh has to be a workflow of its own.

What this does

refresh-image-registry-cache.yml — refreshes the CI cache for both platforms, on workflow_dispatch and on push to release-prep / providers branches. Push runs are never cancelled (a newer one queues behind); manual runs are grouped separately, so a merge cannot cancel one, and re-firing it replaces the run in flight.

Every Python version, both paths. Selective checks narrows python-versions to the default one on a push where only text files changed (_should_run_all_tests_and_versions), which is a second, independent way a branch ends up with cache for one version. The workflow forces the all versions label; a new test in test_selective_checks.py pins that.

The early cache push is removed from the regular build. It is what this workflow replaces, so additional-ci-image-checks.yml loses that job and the seven workflow_call inputs only it consumed (removed from both wrappers identically, so check-ci-workflows-in-sync stays green). main is unaffected: it is not built on push, and its Push Image Cache job at the end of the scheduled canary runs to completion.

refresh-constraints.yml — the manual constraints refresh, split out of update-constraints-on-push.yml so it is not filtered by that workflow's paths: uv.lock and appears under its own name. It calls that workflow rather than copying it, and refreshes all three flavours — source-providers, no-providers and PyPI — since refreshing only PyPI leaves CI and Breeze on the older pins. release-constraints.yml is untouched.

Docs — both procedures in dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md now lead with the workflows; the local buildx / qemu / refresh_images.sh route is kept but demoted to a fallback. The automated-CI section no longer describes the early cache push this PR deletes.

Also fixed in dev/MANUALLY_BUILDING_IMAGES.md: the merge step was documented as docker release-management merge-prod-images in both the regular and slim variants — it is a breeze subcommand, so the documented command fails as written.

Follow-ups this does not do

• v3-3-test needs the workflow backported before its cache can be refreshed there — workflow_dispatch runs the file from the branch you select.
• dev/MANUALLY_BUILDING_IMAGES.md contradicts itself on the buildx version (a warning says "at least 0.13", the prose says 0.23 is needed for docker-container). Left alone rather than guessed at.
• The buildx setup instructions now exist in both dev docs and could be collapsed to one copy.

---

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

[potiuk]

OK Yolo :)

[github-actions]

Backport failed to create: v3-3-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-3-test

You can attempt to backport this manually by running:

cherry_picker e69c188 v3-3-test

This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.