WangLu's Notes

I've been experimenting with Qubes OS on my new laptop and wanted to share some notes on the experience. Hardware Overall, Qubes OS works quite well on my hardware. Aside from typical issues like deep sleep, speaker performance, and touchpad scroll speed, the experience has been smooth. I particularly like that I can boot directly from a microSD card. This allowed me to move the  /boot  partition to the card while completely disabling USB access in  dom0  for better security. Detached  /boot  and LUKS Header Moving  /boot  and the LUKS header to a microSD card is a fun project, but it has some drawbacks: I have to remember to mount  /boot  before updating  dom0 . The system won't shut down properly if I forget to unmount  /boot . Testing Qubes OS 4.3 rc3 I decided to test the Qubes OS 4.3 rc3 release by performing an in-place upgrade. Unfortunately, the system failed to boot afterward. dracut  Issues After the upgrade, the...

I've been running a few containers for several years. They were all running under rootless Docker with a single user. Initially, I planned to  migrate the containers to VMs , but I couldn't get a stable workflow after about two months of effort. Later,  gVisor caught my attention , and I decided to migrate to Podman with gVisor instead. The new plan is to run each container with  --userns=auto  and use Quadlet for systemd integration. This approach provides better isolation and makes writing firewall rules easier. I'm now close to migrating all my containers. Here are a couple of rough edges I'd like to share. Network Layout I compared  various networking options  and spent a few hours trying the one-interface-per-group approach before giving up. I settled on a single macvlan network and decided to use static IP addresses for my containers. To prevent a randomly assigned IP address from conflicting with a predefined one, I allocated a large IP range for my ...

I want to filter the outgoing network traffic for all of my containers based on a set of rules. For example: Some containers should be blocked from accessing the internet entirely. Some containers should have unrestricted internet access. Some containers should be able to access the internet, but not a specific list of URLs. Some containers should only be allowed to access a specific list of URLs. To manage this, I will define logical policy groups and assign each container to one. As a general rule, only DNS and HTTP/HTTPS traffic will be permitted. Option 1: A Proxy for Each Policy Group Imagine Container A is only allowed to access  www.google.com . Here’s how this approach would work: Create an Nginx (or  socat ) container that listens on port 443 and acts as a reverse proxy for  www.google.com . Place both the Nginx proxy and Container A into an  internal  container network. Within this network, add  www.google.com  as a network alias for the Ngin...

For the last few days, I've been experimenting with Podman. My goal was to get a feel for the setup, create a minimal yet scalable environment for a few containers, and identify potential problems early on. Here are my notes from this experience. [Updates 2025-09-21] Added more networking options and other information. Quadlet Quadlet  allows you to define containers, networks and more using a syntax similar to systemd. This includes helpful features like drop-in overrides and templates. The framework is tightly integrated with systemd, and Quadlet actually generates real systemd units. This means I can directly write systemd options in my Quadlet files. One of the biggest benefits I've found is how easy Quadlet makes it to set up socket activation. This allows me to place some containers in an internal network or even without a network at all. Hardening Defaults Let's say I have a group of Systemd and Quadlet units, all named in the format of  xyz-* . My goal is to define ...

My original plan was to stabilize my  VM pipeline  before deploying containers using a hardened stack of Podman, QEMU, SELinux, and user namespaces ( --userns=auto ). However, the pipeline's complexity grew, requiring script rewrites and schema redesigns, and the process took much longer than anticipated. In the meantime, an interesting alternative has captured my attention:  gVisor . It occupies a unique space between traditional SELinux policies and full-blown virtual machines, offering a compelling set of trade-offs. What is gVisor? At its core, gVisor is an application kernel, written in the memory-safe language Go, that provides an additional layer of isolation between containerized applications and the host operating system. It's essentially a user-space implementation of the Linux kernel's system call interface. The security model is explained  here . gVisor in Practice gVisor provides an OCI-compliant runtime called  runsc , which can be almost transpare...

Configuration files for different services are rarely independent. For example, in nftables, I might tag traffic with a firewall mark, and that mark is then used by systemd-networkd or in ip routes. Similarly, when the name of the primary network interface changes, multiple services like nftables, postfix, and samba need to be updated. Requirements I want to define core data in one place, then update all config files with a simple command. If a configuration file is modified by an external process (for example, a package update from a vendor or distribution), the changes must be handled gracefully. Either the merge should be automatic and permanent, or I should be notified to easily resolve any conflicts. It should be obvious within the config file itself what changes I have made. Existing Solutions I did some quick survey and found a few options. 1. Templates These tools render a template using provided data sources. To manage  /etc/config.txt , I would create a  /etc/config....

Now that I've settled on my VM image pipeline , the next logical step is to tackle networking. My Requirements So far, I've been using QEMU's default user-mode networking. It's convenient for quick tasks, allowing for easy port forwarding, Samba shares, and DNS with just a few flags. However, this setup is ultimately insufficient for my needs for a couple of key reasons: Security and Isolation: In the default user-mode setup, a VM can access the host's services via localhost . Worse, because it uses NAT, the VM can also access the host's entire LAN using the host's IP address. Ideally, VMs should have their own identifiable IP addresses, and more importantly, there should be strong network isolation between the host and the VMs. Centralized Auditing: I want to audit all network traffic from my VMs through a centralized solution. This means I need a way to route all VM traffic through a single point of control. Choosing the Right Tool For most people, tools...

I recently tried to create a simple systemd service to send an email notification, but my initial approach with mail and sendmail failed with a strange permission error. My original service file looked like this: [Service] ExecStart=mail --subject=Subject recipient@example.com The error message was a bit of a head-scratcher: warning: mail_queue_enter: create file maildrop/....: Permission denied . A quick search pointed me to the cause : the postdrop binary has setgid. However, the systemd setting NoNewPrivileges=true prevents this. While I hadn't explicitly used that setting, I was using DynamicUser=true , which implies and enforces NoNewPrivileges=true . This meant my service, running as a temporary user, couldn't get the permissions it needed to interact with the mail queue. Note that this implication cannot be disabled/overriden. I wanted to avoid creating a new, dedicated user for this task. I realized that the problem was how mail and sendmail directly interact wit...

I stumbled upon the Gentoo wiki page for systemd-nspawn , which in turn led me to nspawn.org , mkosi , and later systemd-sysupdate . mkosi quickly caught my eye because it's almost exactly what I wanted to build myself, as mentioned in a previous post . So, I decided to spend my "sysadmin fun quota" on it. Overview mkosi is similar to docker build or podman build , but it's designed for creating full OS images. It focuses on development and testing. For example, much like nix-shell , mkosi can quickly launch a sandboxed shell with a specific distribution and selected packages installed. The systemd project itself uses mkosi for testing across different distros. The re-introduction article  is a great read. Speed Note that this is by no means a rigid benchmark. My setup is an SSD with LUKS and an ext4 filesystem (without reflink support). Building Container Images mkosi is pretty fast. A simple mkosi command creates a fresh Debian image. I used the --incrementa...