I decided to spend some time playing with bootc. Mostly I'm inspired by the following articles:
- CoreOS + native container
- Hand-on demo (the last video), build bootc and auto update from registry
- bootc desktop
- bootc for homelab
Day 1
To install bootc in a VM I need an image. bootc-image-builder requires root and I don't want to run this on the host. So I chose CoreOS as the inital system and installed it to QEMU.
I thought it is a great idea to share a folder from host to guest as podman container storage. However, it was not as smooth as I had expected:
- virtiofsd on Debian is too old, so I set up NFS.
- rootless podman doesn't work well with NFS.
- rootfull podman complains upstream fs of overlayfs missing features, the performance was terrible.
Day 2
I didn't find a way of resizing a qcow2 image online. On the other hand I figured maybe I don't need build a disk image after all. CoreOS is already based on ostree, maybe I can use `bootc switch`. This is essentially the same approach as in the first blog post.
`bootc switch` just works, it can reboot, but I cannot login (ssh or local). Fortunately (and quite nicely), I can rollback ostree even without logging in, because I can do that with grub.
I suspect it is because some files are overriding the files in the image.
Day 3
I learned that QEMU has builtin samba support, which is much easier to use than NFS.
Eventually I found that it was SELinux that has been messing up. With `restorecon -R` I could login from QEMU terminal, but not ssh. After logging in, `bootc status` threw an error about /boot, so I guess I needed `bootc install` afte rall.
So I just went back to the original solution, just resize the CoreOS image and build bootc image inside CoreOS. It worked this time.
Now I need to complete the loop, the new bootc OS should build itself and automatically update itself. And a few more things to fix:
- /etc/fstab does not work if modified when building the container, I need to create systemd mount files for mounts
- /etc/hostname does not work if modified when building the container, I need to set it after each boot
- Unlike `bootc install`, `bootc-image-builder` does not provide flags to override the bootc repository, so I need to set it after boot
- Transient etc sounds like a good idea, but I'll need to manually configure /boot. I'd like to enable it when the official doc explains more details.
Conclusion:
bootc works and its quite fun. But unfortunately I didn't find a way of actually using it in production yet:
- I could put it into a VM, but sharing files between host and VM is not pretty at the moment (on Debian stable)
- I don't trust it as my main server yet, and I don't have other machine to which I can install bootc bare-metal.
I think later I'll spend more time trying to tinker with it.
Comments