Skip to main content

mkosi: First Impressions

I stumbled upon the Gentoo wiki page for systemd-nspawn, which in turn led me to nspawn.org, mkosi, and later systemd-sysupdate.

mkosi quickly caught my eye because it's almost exactly what I wanted to build myself, as mentioned in a previous post. So, I decided to spend my "sysadmin fun quota" on it.

Overview

mkosi is similar to docker build or podman build, but it's designed for creating full OS images. It focuses on development and testing. For example, much like nix-shell, mkosi can quickly launch a sandboxed shell with a specific distribution and selected packages installed. The systemd project itself uses mkosi for testing across different distros.

The re-introduction article is a great read.

Speed

Note that this is by no means a rigid benchmark.

My setup is an SSD with LUKS and an ext4 filesystem (without reflink support).

Building Container Images

mkosi is pretty fast. A simple mkosi command creates a fresh Debian image. I used the --incremental flag for subsequent builds.

  • First run: ~30s

  • Second run (after trivial changes): ~5s

Using mkosi -p systemd allows the container to boot (via systemd-nspawn -b), which adds only a few seconds to the build time.

Building VM Images

Building a VM image with mkosi --include mkosi-vm is a bit slower, likely due to the extra steps for installing a bootloader and kernel.

  • First run: ~1m 30s

  • Second run (after minor changes): ~30s

Comparison with bootc

I tried to build a fresh CentOS image using both tools.

mkosi --include mkosi-vm -d centos

  • Duration: ~1m 30s

  • Output disk size: 1.2 GB

podman pull quay.io/centos-bootc/bootc-image-builder:latest && \
podman run ... \
    quay.io/centos-bootc/bootc-image-builder:latest \
    --type raw ... \
    quay.io/centos-bootc/centos-bootc:stream9

  • Duration: ~4m 30s

  • Output disk size: 1.9 GB

Notes:

  • The bootc-image-builder was pre-pulled, and this time isn't included in the measurement.
  • The time to pull the base CentOS image is included. 
  • I'm generating a raw image here instead of QCOW2.

Again, these numbers aren't directly comparable outside of my specific setup.

  • bootc-image-builder runs in a VM, while mkosi runs directly on the host.

  • centos and centos-bootc are different distributions, and their configurations (like installed packages) are also very different. This is obvious from the difference in their final image sizes.

Running Images with systemd-nspawn

I attempted to get unprivileged systemd-nspawn working but failed:

  • systemd-nsresourced.socket and systemd-mountfsd.socket must be running.

  • systemd-mountfsd complains that the image is untrusted unless it's signed or located in a trusted location.

  • I got stuck on another error.

Eventually, I resorted to using sudo systemd-nspawn -U, which worked well. The -b flag "boots" the image by running systemd/init as PID 1.

Running Images with QEMU

mkosi --kvm vm works nicely.

Notes:

Observations, Thoughts and Concerns

  • mkosi is deeply integrated with systemd. Its configuration files are also following the systemd style: e.g. declarative, ini, drop-in overrides.

  • I wasn't able to test the performance benefits of reflink, because my filesystem doesn't support it and the disk images were small anyway.

  • I also wasn't able to test if SELinux works. Supposedly, it needs an extra flag in mkosi.conf and might be slow. On the other hand, it works out-of-the-box in bootc images.

  • I don't really miss Containerfiles much. I usually just need to copy files, and for my use case, a Containerfile would essentially just be running my scripts with bind mounts. Plus, I don't use many layers. But I might miss having an immutable Linux setup.

  • mkosi supports many popular distributions. while bootc only support Fedora/CentOS.

  • mkosi may add surprising modifications to the image:

  • zvol doesn't seem very reliable, so I'll probably avoid using it for another few years.

Conclusion

mkosi is a very interesting tool. While I'm not ready to migrate my entire image-building pipeline yet, I might consider replacing my current LXC setup with it.

Labels:

Comments

Post a Comment

Popular posts from this blog

A Rocky Migration: Moving from docker-compose to Podman and gVisor

I've been running a few containers for several years. They were all running under rootless Docker with a single user. Initially, I planned to  migrate the containers to VMs , but I couldn't get a stable workflow after about two months of effort. Later,  gVisor caught my attention , and I decided to migrate to Podman with gVisor instead. The new plan is to run each container with  --userns=auto  and use Quadlet for systemd integration. This approach provides better isolation and makes writing firewall rules easier. I'm now close to migrating all my containers. Here are a couple of rough edges I'd like to share. Network Layout I compared  various networking options  and spent a few hours trying the one-interface-per-group approach before giving up. I settled on a single macvlan network and decided to use static IP addresses for my containers. To prevent a randomly assigned IP address from conflicting with a predefined one, I allocated a large IP range for my ...
Post a Comment
Read More »

GameConqueror 0.09 -- Linux Game Hacking Tool

If you are a game hacker If you've been looking for a `CheatEngine for Linux` Then you can't miss this. ============================================== GameConqueror is a game hacking tool for linux, it's written in PyGTK and uses scanmem as its backend. It's supposed to be with most useful features of CheatEngine for Linux. Currently, I've implemented almost everything about scanning, involving variant data types and scan types: Data Types: int{8/16/32/64}, float{32/64}, unknown type(int or float) and unknown width(will try each of them), byte array and string Scan Types: equal, greater, less, changed, unchanged, increased(by), decreased(by) This should be enough for most cases, so I decided to release it at the current status. ============================================= Here's how you can get it PPA (for Ubuntu users) https://launchpad.net/~coolwanglu/+archive/scanmem (I've not test it in 32bit environments or Jaunty, do please inform me if it doe...
Post a Comment
Read More »

Fix Google Security Code

Google Security Code (http://g.co/sc) is one type of 2-step verification. This is particularly useful when security keys and passkeys are not available. I have been using it in my LXC containers, until today I found out that it stopped working. It just kept saying "The code is invalid". It is easy to rule out some factors: The code works on other browsers on my laptop. The code works on other devices that are directly connected to the router. So it appears that Google also checks IP addresses besides the security code. Recently I have IPv6 enabled, so most devices that are directly connected to the router have both IPv4 and IPv6 addresses. But  I only enabled IPv4 for my LXC containers. So I guess when a code is generated by device A and used by device B, Google should be able to check that device A and device B are closely located. But in my case, IPv6 address appears on device A but not on device B, which may look suspicious. To fix the problem, I just needed to disable IPv...
Post a Comment
Read More »