Update build image to the latest `buildkit` change

[potiuk]

We have just merged the #20664 change that heavily simplified the way how our images are build and cached. We did not have any task implemented so far to cover those caching (I worked on this simplification in parallel) so the changes should affect Breeze2 only very slightly.

The nice thing is that we have 700 (!) less lines of bash by just this change so the Breeze/CI Python rewrite project suddenly became way simpler :).

Those are the changes that I think affect current breeze2 work and should be implemented:

• no --force-pull flag - the caching mechanism of buildkit is so good that we do not have to force pull image any more
• no -check-if-base-python-image-updated flag - we do not keep base python image any more in our registry and we do not need to refresh the base image. Base image will be refreshed automatically as we also warn the users that they should always rebase to latest branch HEAD to rebuild the image
• no push-image command - we do not need to push images any more. instead we have prepare-build-cache command which builds images locally and pushes cache for them.
• no more complex "check if image needs pull" logic is needed any more.

[potiuk]

cc: @edithturn

[edithturn]

Got it @potiuk, do you want me to take this task? it will be great :)

[potiuk]

cc: @Bowrna

[potiuk]

Ah no -it's @Bowrna's just wanted you to take a look

[Bowrna]

@potiuk do you think with buildkit changes, do we have to make changes in build-image in Breeze2? Could you explain to me more how this buildkit change will affect Breeze2?

airflow/dev/breeze/src/airflow_breeze/ci/build_params.py

Lines 33 to 36 in ff4939f

	build_id: int = 0
	# To construct docker cache ci directive
	docker_cache: str = "pulled"
	airflow_extras: str = "devel_ci"

[potiuk]

Just a little. Actually if anything, The "buildkit" change made it the "old" breeze build much closer to what the "new" breeze did (i.e. simplified it A LOT).

What we really need to do is just syncing the parameters that are used to prepare the "docker build" command (see the description of the issue above):

• --force-pull and --check-if-python-bae are gone
• build is always executed as one step for both PROD and CI image (previously PROD image was build in two steps) - so no change vs. what you've done already
• --all the code responsible for pulling images for cache is gone now (so you do not need to implement it).

So this one is really to make sure that we have the same parameters in build command than what we have in "old" breeze.

[potiuk]

One thing - but this should be a separate issue - is that we have this prepare-build-cache which slightly modifies build command (adds --cache-to and starts builder before) - but this is something we should add after and the whole part (it is guarded by PREPARE_BUILDX_CACHE variable) about this can be skipped.

[Bowrna]

ok @potiuk I think I have to go and dig deeper to understand the impact of buildkit change. Could you tell me which command in breeze I can go through to better understand on the parts you have mentioned?

[potiuk]

Just

• build-image
• prepare-image-cache

That's all. Those are the only two impacted "really", we call "build-image" methods from other commands (for example when we run ./breeze - build image is run when we determine if we should do it, but they use the same "_build_images.sh" libraries to call build image methods.)