[BUG] compose build does not abort build on SIGTERM

[fho]

Description

When a "docker compose build" process receives a TERM signal, it ignores it and continues running.
I expect "docker compose build" to terminate gracefully.
This works when pressing ctrl+c.

When sending the INT signal, "docker compose build" also does not terminate.
This might be the same bug then #10586.

Steps To Reproduce

git clone https://github.com/ndeloof/truc.git
cd truc
(export BUILDKIT_PROGRESS=plain; docker compose -f compose.yml build --no-cache >&2 & (sleep 0.2 && pkill -fe 'docker compose' && echo "KILLED IT"))

Compose Version

Docker Compose version v2.18.1 and 
Docker Compose version v2.17.3

Docker Environment

No response

Anything else?

No response

[ndeloof]

Tested with another approach: I'm running killall -2 docker-compose after I started a build:

docker compose build  --no-cache
[+] Building 1.0s (5/5) FINISHED                                                                                                             
 => [demo internal] load build definition from Dockerfile                                                                               0.0s
 => => transferring dockerfile: 456B                                                                                                    0.0s
 => [demo internal] load .dockerignore                                                                                                  0.0s
 => => transferring context: 2B                                                                                                         0.0s
 => [demo] resolve image config for docker.io/docker/dockerfile:1.4                                                                     0.5s
 => CACHED [demo] docker-image://docker.io/docker/dockerfile:1.4@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e84ced81e24ef1a0d  0.0s
 => CANCELED [demo internal] load metadata for docker.io/library/golang:1.18-alpine                                                     0.4s
canceled

Could you please confirm this works for you as well ?
AFAICT your script using pkill only sends a kill signal to the parent docker CLI command, which isn't propagated to CLI plugin. Would need to send kill signal to the process group (like Ctrl+C does in a terminal)

[ndeloof]

unrelated: what's the -e flag for ? I don't have this flag supported on my version of the pkill command and this doesn't seem to be a common one (https://fr.manpages.org/pkill)

[ndeloof]

I can confirm build get canceled if I use

kill -2 -$(ps -o pgid= $(pgrep -f 'docker compose'))

[fho]

@ndeloof you are right, the signal needs to be sent to the pgid, thanks for having a look

unrelated: what's the -e flag for ? I don't have this flag supported on my version of the pkill command and this doesn't seem to be a common one (https://fr.manpages.org/pkill)

   -e, --echo
          Display name and PID of the process being killed.  (pkill only.)

(https://manpages.debian.org/bullseye/procps/pkill.1.en.html)

[fho]

@ndeloof shouldn't the docker process forward the signals to the plugin processes that it started, when it receives a signal?

[ndeloof]

@fho AFAICT this is why process groups have been designed, so that your child process also receive SIGINT on user-requested interruption. I don't think forwarding signals to child processes is a common POSIX practice, but I might be wrong. Feel free to open an issue on docker/cli if you get any additional reference regarding this point.

[fho]

@ndeloof that it is the purpose of process groups makes sense.
On the other hand, for docker compose I don't know why it would make sense that the child processes continue to run when the parent receives SIGTERM. There is the PR_SET_PDEATHSIG attribute for processes, that when set will terminate the child processes if the parent dies.
This is another topic, also related to docker not compose.

Back to the root topic: :-)
When I start docker compose build, get the PGID of that process and send TERM to the PGID (kill -- -$PGID), all processes that docker compose build started should terminate.
This works in a simple docker compose project.

In our bigger internal docker compose project it doesn't.
When I sent 3x TERM to the PGID, one /usr/lib/docker/cli-plugins/docker-compose process does not terminate and becomes parent of pid 1. That process receives the SIGTERM process that was send to the PGID, I verified this with attaching strace before sending the signal.
When I send afterwards TERM signals to that remaining docker-compose process it also ignores it.
It continues to do write syscalls like [pid 23625] write(1, "\33[34m => => exporting layers "..., 241) = 241, so it is still doing the build.
The terminal in which I ran docker compose build broke, it does not show the build output of the remaining docker-compose process. The terminal accepts input and runs commands but overwrites all output immediately with the last output that was shown before the kill happened.
The timer [+] Building 606.8s (179/179) is still shown in the first line and continues to get updated.

This should not happen, that process should have been also terminated, shouldn't it?

So far I could not reproduce this with a simple compose file that I can share. I'll try to figure out with dlv what happens.

[fho]

I created #10648 after I understood the issue better.
I realized this morning that it is also quite simple to reproduce and added the information also to #10648.