WangLu's Notes

I stumbled upon the Gentoo wiki page for systemd-nspawn , which in turn led me to nspawn.org , mkosi , and later systemd-sysupdate . mkosi quickly caught my eye because it's almost exactly what I wanted to build myself, as mentioned in a previous post . So, I decided to spend my "sysadmin fun quota" on it. Overview mkosi is similar to docker build or podman build , but it's designed for creating full OS images. It focuses on development and testing. For example, much like nix-shell , mkosi can quickly launch a sandboxed shell with a specific distribution and selected packages installed. The systemd project itself uses mkosi for testing across different distros. The re-introduction article  is a great read. Speed Note that this is by no means a rigid benchmark. My setup is an SSD with LUKS and an ext4 filesystem (without reflink support). Building Container Images mkosi is pretty fast. A simple mkosi command creates a fresh Debian image. I used the --incrementa...

Today, my pipeline regularly builds images for my disposable VMs . Here's the current process: A dedicated builder VM reads  Containerfile s for all VMs, including itself. The builder VM uses podman build to create container images for all VMs. The builder VM then uses bootc-image-builder to create disk images for all VMs. This process works well, but it has a significant issue: the disk images aren't built efficiently. Unlike container images, which benefit from reusable, cacheable layers, disk images are always built from scratch. This leads to long build times and limited opportunities for data deduplication. To address this, I've been exploring alternative options to improve the pipeline. Disk Image Formats and Deduplication My Current Format: QCOW2 I currently use QCOW2 with compression enabled. This format offers several features like snapshots, compression, and sparse files, which are useful when the underlying filesystem doesn't support them. However, if the f...

[Updates: 2024-10-21] I found a working solution: - Disable sandboxing in configuration.nix - Build a tarball image using `nixos-generator -f lxc` - Create an LXC container with `lxc-create -t none` - Modify the config of the LXC container (e.g. specify rootfs path, set unconfined AppArmor) - Create the rootfs directory and remove all POSIX ACL (setfacl --remove-all) - Extract the tarball into rootfs/ I planned to try NixOS in LXC. I have found a few successfull stories: 1 , 2, 3 . However they are all using Proxmox LXC, and/or the image file is for LXD. First, I tried to download the official image via lxc-create. The image can boot, but I have trouble running `nix-channel --update`, which complains about sandboxing. I think it's related to unprivileged LXC containers. Further, as part of the nice feature of NixOS, I cannot easily disable sandbox from there. Second, I tried to build a NixOS image from scratch, using nixos-generators. This is mentioned in the 3rd link above. This ...