WangLu's Notes

My original plan was to stabilize my  VM pipeline  before deploying containers using a hardened stack of Podman, QEMU, SELinux, and user namespaces ( --userns=auto ). However, the pipeline's complexity grew, requiring script rewrites and schema redesigns, and the process took much longer than anticipated. In the meantime, an interesting alternative has captured my attention:  gVisor . It occupies a unique space between traditional SELinux policies and full-blown virtual machines, offering a compelling set of trade-offs. What is gVisor? At its core, gVisor is an application kernel, written in the memory-safe language Go, that provides an additional layer of isolation between containerized applications and the host operating system. It's essentially a user-space implementation of the Linux kernel's system call interface. The security model is explained  here . gVisor in Practice gVisor provides an OCI-compliant runtime called  runsc , which can be almost transpare...

Configuration files for different services are rarely independent. For example, in nftables, I might tag traffic with a firewall mark, and that mark is then used by systemd-networkd or in ip routes. Similarly, when the name of the primary network interface changes, multiple services like nftables, postfix, and samba need to be updated. Requirements I want to define core data in one place, then update all config files with a simple command. If a configuration file is modified by an external process (for example, a package update from a vendor or distribution), the changes must be handled gracefully. Either the merge should be automatic and permanent, or I should be notified to easily resolve any conflicts. It should be obvious within the config file itself what changes I have made. Existing Solutions I did some quick survey and found a few options. 1. Templates These tools render a template using provided data sources. To manage  /etc/config.txt , I would create a  /etc/config....

Now that I've settled on my VM image pipeline , the next logical step is to tackle networking. My Requirements So far, I've been using QEMU's default user-mode networking. It's convenient for quick tasks, allowing for easy port forwarding, Samba shares, and DNS with just a few flags. However, this setup is ultimately insufficient for my needs for a couple of key reasons: Security and Isolation: In the default user-mode setup, a VM can access the host's services via localhost . Worse, because it uses NAT, the VM can also access the host's entire LAN using the host's IP address. Ideally, VMs should have their own identifiable IP addresses, and more importantly, there should be strong network isolation between the host and the VMs. Centralized Auditing: I want to audit all network traffic from my VMs through a centralized solution. This means I need a way to route all VM traffic through a single point of control. Choosing the Right Tool For most people, tools...

I recently tried to create a simple systemd service to send an email notification, but my initial approach with mail and sendmail failed with a strange permission error. My original service file looked like this: [Service] ExecStart=mail --subject=Subject recipient@example.com The error message was a bit of a head-scratcher: warning: mail_queue_enter: create file maildrop/....: Permission denied . A quick search pointed me to the cause : the postdrop binary has setgid. However, the systemd setting NoNewPrivileges=true prevents this. While I hadn't explicitly used that setting, I was using DynamicUser=true , which implies and enforces NoNewPrivileges=true . This meant my service, running as a temporary user, couldn't get the permissions it needed to interact with the mail queue. Note that this implication cannot be disabled/overriden. I wanted to avoid creating a new, dedicated user for this task. I realized that the problem was how mail and sendmail directly interact wit...

The central question is: how do you manage secrets like SSH keys, API keys, and passwords for disposable VMs ? 🤷‍♂️ Let's establish some ground rules for this scenario. Suppose I want to pass an API key to the VM  chimera , which is run by the  chimera-runner user on the host. My security requirements are: On the host, only root and  chimera-runner  should have access to the secrets. In VM  chimera , only root and relevant service users should have access to the secrets. No one from other VMs, including their root users, should have access to VM  chimera 's secrets. The guest VMs themselves are not trusted . The bootc documentation on this topic is very informative. On a high level, there are a few ways to achieve this. 1. OEM Strings / Firmware QEMU can pass data to a VM via SMBIOS OEM strings ( -smbios ) or firmware configuration ( -fw_cfg ). Notably, both methods are supported by systemd-creds using special keys. This approach is practical for sma...

[Update] I managed to work out the AppArmor profile, and decided to go with guestmount for now. I am setting up a maintenance pipeline for my virtual machines . The pipeline has two main routines: The BACKUP routine: Every day, this routine shuts down each VM, backs up the data in /var, updates the VM's disk image if a new version is available, and then restarts it. The BUILD routine: Every week, this routine uses a special builder VM to create new disk images for all VMs. There is a scheduling conflict with the builder VM: the BACKUP routine needs to shut it down, while the BUILD routine needs it running. To resolve this, I merged both into a single set of systemd services that runs daily. The BUILD routine starts automatically when the builder VM starts, at the end of the BACKUP routine. The builder VM's systemd unit has an ExecCondition= property, which is skipped 6 days a week. Surprisingly, the most difficult part of this pipeline was not the scheduling, but the backup pro...

Just learned about GNU Stow , which is a tool for managing symlink farm. Basically the idea is to store all files in one place, then create symlink all around the system pointing to your files. There are various use cases, like dot files and installing/uninstalling packages. But I mostly use it for tracking system config files, similar to how NixOS works. In fact I wrote my own scripts with "cp -rs", but GNU Stow works much better.

Recently I explored various options of creating email aliases. I care about the following factors: [Critical] Limits: I may need to create ~100 or ~200 aliases. [Critical] Retain email envelope: I should be able to see the original "from" and "to" addresses. [Critical] Catch-all: I want to catch all emails that are sent to unknown addresses. [Critical] Reliability: The service should "just work". [Important] Easy to create a new alias: it should be easy to create a new one. API support would be ideal. [Important] Tagging/Comment: I'd like to add a comment to an alias to remember why it was created. Note that the alias itself may be totally random [Nice to have] Send As: it's nice to be able to send email as the alias. Cloudflare Email Routing Doc Maximum of 200 rules A few clicks to create a new rule. There is easy-to-use API. I saw  an example  where the service is not so reliable, there are also similar complains in the cloudflare forum. Allows a...

I recently read this article , which talks about restricting (proactive) internet access of a process. It is easy to completely disable internet/network access, by throwing a process into a new private network namespace. I think all popular sandboxing tools support it nowadays: unshare -n bwrap --unshare-net systemd.service has PrivateNetwork=yes docker has internal network But the trickier, and more realistic scenario is: [Inbound] The process needs to listen one or more ports, and/or [Outbound] The process needs to access one or more specific IP address/domain I can think of a few options. Option 1: Firewall Rules Both iptables and nftables support filter packets by uid and gid. So the steps are clear: Run the process with a dedicate uid and/or gid Filter packets in the firewall If needs, regularly query DNS and update the allowed set of IP addresses. This is somehow similar to reresolve-dns.sh from WireGuard. This option is not very complicated, and I think the overhead is low....

nftables has been enabled by default in latest Ubuntu and Debian, but not fully supported by Docker. I've been hestitating about migrating from iptables to nftables, but managed to do it today. Here are my thoughts. Scripting nftables The syntax of iptables and nftables are different, but not that different, both are more or less human readable. However, nftables is clearly more friendly for scripting. I spent quite some time in a python script to generate a iptables rule set, and I was worried that I need lots of time migrating the script. Aftering studying the syntax of nftables, I realized that I could just write /etc/nftables.conf directly.  In the conf file I can manage tables and chains in a structured way. I'm free to use indentations and new lines, and I no longer need to write "-I CHAIN" for every rule. Besides, I can group similar rules (e.g. same rule for different tcp ports) easily, and I can define variables and reuse them.  Eventually I was able to write...

 There are three ways of running Docker: Privileged: dockerd run with root, container root = host root Unprivileged: dockerd run with root, container root = mapped user Rootless: dockerd run with some user, container root = some user I've been hestitating between Unprivileged and Rootless. On one hand, rootless sounds like a great idea; on the other hand, some considers unprivileged user namespace as a security risk . Today I decided to migrate all my unprivileged containers to rootless ones. I had to enable unprivileged user namespace for a rootless LXC container anyways. A Cryptic Issue The migration is overall smooth, except for a cryptic issue: sometimes DNS does not work inside the container. The symptom is rather unusual: curl works but apt-get does not work. For quite a while I'd thought that apt-get uses some special DNS mechanism. After some debugging, especially comparing files /etc/ between a unprivileged container and a rootless container, I realized that non-root u...

 I have an NGINX server that is publicly accessible. It has been deployed in the following manner: Machine A Port forwarding with socat: localhost:4443 ==>  0.0.0.0:443 Machine B Running NGINX in a Docker container Port forwarding by Docker: <container_ip>:443 ==> localhost:4443 Port forwarding by SSH to Machine A: localhost(B):4443 ==> localhost(A):4443 This in general works. Machine A is published to my domain, and the traffic to 443 is forwarded to NGINX in a few hops. However there is a problem: the NGINX server never sees the real IP of the client, so it is impossible to depoly fail2ban or other IP address based tools. So I wanted to fix it. Step 1: VPN The first step is to connect machine A and B with a VPN. I feel that it would also work without it, but the iptables rules could be more tricky.  WireGuard is my choice. I made a simple setup: Machine A has IP: 10.0.0.2/24 Machine B has IP: 10.0.0.1/24 On both machines, the interface is called wg0, Allo...

Weeks ago I  purchased a secondhand machine. Since then I have been tinkering this little box. The Perfect Media Server  site is a good place to start with. Arch Linux Wiki is my go-to learnning resource, even though I use Ubuntu. Filesystem I'd be super paranoid and careful, as this is my first time manually configuring a disk array. Basically my optoins include: ZFS btrfs Snapraid (or even combnied ZFS/btrfs) Unraid My considerations include: Data integrety, which is the most important. Maintenance. I want everything easy to set up and maintain. Popularity. There will be more doc/tutorial/discussions if the technology is more popular. Eventually I decided to use ZFS with raidz2 on 4 disks.  I also took this chance to learn configuring disk encryption. I decided to use LUKS beneath ZFS. I could have just used ZFS's built-in encryption, but I thought LUKS is fun to learn. It really was. The commands are way more user-friendly that I had expected. Hardening SSH Most popula...

For quite a while, I have not been so confused (and excited) by a technical issue. The problem is: I got very slow download speed (~10Mbps) on my Windows laptop, when connected to Wifi. To make it a bit more puzzling: - The upload speed is much better (~60Mbps). - Sometimes the download speed may reach ~100Mbps for a few seconds, before quickly fall back to ~10Mbps. Having been suffering from it for a few months, I finally decide to take on this issue today. Beginning of the Journey As usual, there are some troubleshooting routine I should do. Also as usual, they don't help much. - Try other devices with the same Wifi - Try to connect to other Wifi - Try to connect to Ethernet - Reset the Wifi router - Run Windows troubleshooter Well, eventually I did get some information. The download & upload speed is actually normal for all my other devices, which includes Android/iOS phones, MacBook Pro etc.  This pieced of info narrowd the root cause down to the laptop itself: The rou...

surface pro stylus calibration http://www.reddit.com/r/Surface/comments/18cd96/improved_pen_accuracy_with_a_100_point/ http://www.reddit.com/r/Surface/comments/1cy8hn/this_pen_calibration_is_so_far_the_best_solution/ Search for 'UserLinearityData' in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet and remove 'devicekind=pen'   merge tool: meld   linux虚拟机的硬盘大小不够，扩容以后又调整了分区，于是grub就不能进入了。启动说unknown filesystem然后进入grub rescue。 根据 http://askubuntu.com/questions/142300/fixing-grub-error-error-unknown-filesystem 在grub rescue输入 set prefix=(hd0,5)/boot/grub insmod linux linux (hd0,5)/boot/vmlinuz-3.11.0-15-generic initrd (hd0,5)/boot/initrd.img-3.11.0-15-generic boot 然后grub这步正常了，但是会卡到initramfs/busybox，说找不到init 于是在linux命令后面加上root=/dev/sda5就可以正常进入系统 进入系统以后赶紧用grub-install和update-initramfs再修正一下。