So I was setting up my local dev, following the docs for openshift on libvirt. I downloaded the latest rhcos image and went through the install, but bootkube failed because it couldn't read a config file. I know this seems like an installer issue, but I think it's an rhcos issue, and you will see why in a second.
Essentially, it was mounting some volumes onto a container to render some assets (see https://github.com/openshift/installer/blob/master/modules/bootkube/resources/bootkube.sh#L7-L12 and https://github.com/openshift/installer/blob/master/modules/bootkube/resources/bootkube.service#L11) and the renderer needed /opt/tectonic/kco-config.yaml (mounted into the container at /assets/kco-config.yaml). However only the directory structure of /opt/tectonic was making it into the container (I used an alpine container to figure that out). In the course of debugging, I tried adding the :z suffix to the volume mount with the alpine container, and the problem went away. When I tried to repro the original issue after that, it was gone, even without the :z suffix.
Reading up on the :z volume mount suffix, it seems like it can have consequences on the host machine (https://stackoverflow.com/a/35222815), particularly, it seems to manipulate selinux labels in some way. My guess is that something changed in the base image kind of recently having to do with the labels on /opt/tectonic (or /var/opt/tectonic which afaict is hardlinked one way or the other), and now the docker command fails to mount the files in properly, and the :z suffix inadvertently fixes the issue.
I'm not sure what could've changed to cause this, but I figured I would throw it out there and see if anyone had some insight. It was definitely a thorn at any rate.
So I was setting up my local dev, following the docs for openshift on libvirt. I downloaded the latest rhcos image and went through the install, but bootkube failed because it couldn't read a config file. I know this seems like an installer issue, but I think it's an rhcos issue, and you will see why in a second.
Essentially, it was mounting some volumes onto a container to render some assets (see https://github.com/openshift/installer/blob/master/modules/bootkube/resources/bootkube.sh#L7-L12 and https://github.com/openshift/installer/blob/master/modules/bootkube/resources/bootkube.service#L11) and the renderer needed
/opt/tectonic/kco-config.yaml(mounted into the container at/assets/kco-config.yaml). However only the directory structure of/opt/tectonicwas making it into the container (I used an alpine container to figure that out). In the course of debugging, I tried adding the:zsuffix to the volume mount with the alpine container, and the problem went away. When I tried to repro the original issue after that, it was gone, even without the:zsuffix.Reading up on the
:zvolume mount suffix, it seems like it can have consequences on the host machine (https://stackoverflow.com/a/35222815), particularly, it seems to manipulate selinux labels in some way. My guess is that something changed in the base image kind of recently having to do with the labels on/opt/tectonic(or/var/opt/tectonicwhich afaict is hardlinked one way or the other), and now the docker command fails to mount the files in properly, and the:zsuffix inadvertently fixes the issue.I'm not sure what could've changed to cause this, but I figured I would throw it out there and see if anyone had some insight. It was definitely a thorn at any rate.