WangLu's Notes

Previously, I decided to set up a headless VM orchestrator for daily driving and gaming. In general, installing a minimal Debian and QEMU is fairly straightforward, but the devil is in the details. Here are my notes on the process. See also: part 1 Overall Setup The machine is a laptop. The built-in display is connected to the iGPU, and an external display is connected to the dGPU. I prefer to use only the external display; the built-in one is rarely used. I have two VMs: one as a daily driver and the other for gaming. Both have GPU passthrough. Only one runs at a time. Resources Reserved for the Host CPU: 1 core, 2 threads. RAM: 2GB (typical usage is around 500MB). Disk: 32GB. The OS only needs ~2GB, but I need extra space for cache, log, OS images, etc. CPU The main QEMU process and IO thread should be pinned to the host CPUs. vCPU threads should be pinned to the CPUs reserved for the VM. CPU pinning cannot be done through QEMU command-line arguments. I had to talk to the QEM...

I recently tried to install Qubes OS alongside an existing Windows installation. It turned out to be surprisingly difficult—way harder than my last attempt—likely due to a combination of my encrypted  /boot  setup and older hardware. Here are some notes from the process. Shrinking an NTFS Volume I needed to free up some space from a Windows NTFS volume. Normally, this just takes a few clicks in Disk Management. This time, however, Windows reported a "shrinkable volume" that was suspiciously small. Following  this answer , I tried the standard fixes: Disabled hibernation ( powercfg /h off ) Disabled the pagefile Disabled system protection This increased the shrinkable volume a bit, but nowhere near the actual free space left on the partition. Digging into the Windows Application logs in Event Viewer, I finally found the culprit:  The last unmovable file appears to be: \$Mft::$DATA . It turns out  $Mft  is a special block in NTFS that cannot be easily moved, ...

I've been running Qubes OS on Machine A alongside Windows for a while. My setup involved storing the unencrypted  /boot  partition and the LUKS header on an external USB drive. Recently, I planned to install Qubes on Machine B, also in a dual-boot configuration. However, the complexity jumped significantly: Machine B has Secure Boot enabled because BitLocker requires it. On previous installs, I grew tired of toggling Secure Boot in the BIOS every time I switched operating systems. I only have one USB drive. Managing separate  /boot  partitions for two different Qubes installations on a single thumb drive is messy. After some experimentation, I found a way to solve both problems. Sharing One USB Drive for Multiple Qubes Installations The solution is elegant: Don't store  /boot  on the USB drive. Instead, move  /boot  to the encrypted internal disk partition. The USB drive's only job is to unlock that partition and hand over control to the system. O...

Today, various services (native, LXC, Docker) are running on my server. I'm mostly happy with the setup, but I decided to revisit my server's defenses under the assumption that a remote attacker or malicious code could compromise my services. A service might break out of its container or even gain root privilege. VMs are a better security boundary than containers; they can limit the damage if an attacker gains root privilege. I cannot afford to run a dedicated VM for each service, so I will need to carefully group the services and run a dedicated VM for each group. Each group should be carefully designed based on the data accessed and the features/capabilities required. For example, some VMs may have access to my photos, while others may not have network access. The Goal There are two particular issues I want to address: First, I want VM images to be easily reproducible, which makes backup and restore trivial. NixOS and GNU Guix System are great examples, where you only need to...

I decided to spend some time playing with bootc. Mostly I'm inspired by the following articles:  CoreOS + native container Hand-on demo (the last video), build bootc and auto update from registry bootc desktop bootc for homelab Day 1 To install bootc in a VM I need an image. bootc-image-builder requires root and I don't want to run this on the host. So I chose CoreOS as the inital system and installed it to QEMU. I thought it is a great idea to share a folder from host to guest as podman container storage. However, it was not as smooth as I had expected: virtiofsd on Debian is too old, so I set up NFS. rootless podman doesn't work well with NFS . rootfull podman complains upstream fs of overlayfs missing features, the performance was terrible. I gave up. I guess I'll just use the CoreOS disk, whose size is 10G, not enough. Day 2 I didn't find a way of resizing a qcow2 image online. On the other hand I figured maybe I don't need build a disk image after all. Cor...

When I have important data on a device, I back it up to my server using dedicated user accounts. The other day, I checked /etc/passwd on my server and found entries like this: some-backup-user1:x:1003:1004:... some-backup-user2:x:1004:1007:... A few inconsistencies immediately bothered me: UID/GID Mismatches: Many users have UIDs that don't match their primary GIDs. While this technically works and might seem like just an aesthetic concern, I realized that UIDs and GIDs are crucial metadata. I need to preserve them accurately for future system migrations to maintain correct file ownership.ID Ambiguity: ID Ambiguity: The same number (e.g., 1004) could represent a User ID for one account and a Group ID for a completely different group. This overlap is a recipe for mistakes during administration tasks if I'm not paying close attention. Lack of Structure: User and group accounts created for very different purposes – regular logins, backup processes, container management, specifi...

My current server setup, based on Debian Stable and Docker, has served me reliably for years. It's stable, familiar, and gets the job done. However, an intriguing article I revisited recently about Fedora CoreOS, rpm-ostree, and OSTree native containers sparked my curiosity and sent me down a rabbit hole exploring alternative approaches to system management. Could there be a better way? Core Goals & Requirements Before diving into new technologies, I wanted to define what "better" means for my use case: The base operating system must update automatically and reliably. Hosted services (applications) should be updatable either automatically or manually, depending on the service. Configuration and data files need to be easy to modify, and crucially, automatically tracked and backed up. Current Setup: Debian Stable + Docker My current infrastructure consists of several servers, all running Debian Stable. System Updates are andled automatically via unattended-upgrades. Se...

A few days ago, while browsing security topics online, Qubes OS surfaced—whether via YouTube recommendations or search results, I can't recall precisely. Intrigued by its unique approach to security through compartmentalization, I delved into the documentation and watched some demos. My interest was piqued enough that I felt compelled to install it and give it a try firsthand. My overall first impression of Qubes OS is highly positive. Had I discovered it earlier, I might have reconsidered starting my hardware password manager project. Conceptually, Qubes OS is not much different from running a bunch of virtual machines simultaneously. However, its brilliance lies in the seamless desktop integration and the well-designed template system, making it far more user-friendly than a manual VM setup. I was particularly impressed by the concept of disposable VMs for temporary tasks and the clear separation of critical functions like networking (sys-net) and USB handling (sys-usb) into the...

Windows 10 will soon reach its end of life, and my 1-gen Surface Pro is not supported by Windows 11. I (finally) decided to install Linux to it. Fortunately, it's a not-so-easy nice adventure: The device has only one USB port, so I have to bring back my 10+-year old USB hub. My live USB drive cannot boot directly, I have to disable Secure Boot first, by holding Volume Up during boot. I think years ago I learned that booting on USB might not work through a USB hub, but fortunatelly it worked well with my setup. This is done by holding Volume Down during boot. Wifi adapter was detected in the live Linux environment, but not functional. And I don't have a USB-Ethernet adapter. Luckily, nowadays we have USB-tethering from Android phones, which works out-of-the-box. Originally I planned to following this guide to set up root on ZFS, however, the system froze when building the ZFS kernel module. Then I decided to just use EXT4, yet I still learned a lot from the guide about disk par...

[UPDATE] Later I was able to successfully migrate an amd64 Ubuntu server to Debian, following the same procedure. I had a Ubuntu 22.04.3 on a Raspberry Pi 4. Recently I decided to switch it to Debian 12.5 in-place . Mostly I was following this script: https://github.com/alexmyczko/autoexec.bat/blob/master/config.sys/ubuntu-deluxe   It worked surprisingly well, however, the system was eventually broken (unsurprisingly) near the end, when I tried to upgrade the kernel. I tried to fix initramfs by copying dtbs files, which seemed to work. Some references: https://qiita.com/takasan/items/ef93be9e9d3f791eee66 https://bugs.launchpad.net/ubuntu/+source/flash-kernel/+bug/2012750 But the kernel did not load after reboot. Reverting /boot/config.txt didn't help. Then I downloaded a Debian image and replaced only the boot partition of the micro SD card. This time the kernel was able to boot, but it couldn't load the filesystem. I modified cmdline.txt, replaced "root=LABEL=RASPIROOT...

我的小服务器上一直装了个Ubuntu Desktop，不过安装之后一直没用过GUI，而且各种依赖的包有时也挺烦人的，比如gvfs和tracker自带的systemd user service，我还得手动给若干用户禁用掉。 本来我是想着留个Desktop，万一紧急情况可以上网查查命令。不过有网的话，最差情况我应该也能临时装一个X和浏览器，估计问题不太大。 于是我决定把Ubuntu Desktop换成Ubuntu Server，主要还是把gnome的包都删了。 一番折腾以后，安装包的数量从大约1800降到了800以下。舒服！

I recently read this article , which talks about restricting (proactive) internet access of a process. It is easy to completely disable internet/network access, by throwing a process into a new private network namespace. I think all popular sandboxing tools support it nowadays: unshare -n bwrap --unshare-net systemd.service has PrivateNetwork=yes docker has internal network But the trickier, and more realistic scenario is: [Inbound] The process needs to listen one or more ports, and/or [Outbound] The process needs to access one or more specific IP address/domain I can think of a few options. Option 1: Firewall Rules Both iptables and nftables support filter packets by uid and gid. So the steps are clear: Run the process with a dedicate uid and/or gid Filter packets in the firewall If needs, regularly query DNS and update the allowed set of IP addresses. This is somehow similar to reresolve-dns.sh from WireGuard. This option is not very complicated, and I think the overhead is low....

nftables has been enabled by default in latest Ubuntu and Debian, but not fully supported by Docker. I've been hestitating about migrating from iptables to nftables, but managed to do it today. Here are my thoughts. Scripting nftables The syntax of iptables and nftables are different, but not that different, both are more or less human readable. However, nftables is clearly more friendly for scripting. I spent quite some time in a python script to generate a iptables rule set, and I was worried that I need lots of time migrating the script. Aftering studying the syntax of nftables, I realized that I could just write /etc/nftables.conf directly.  In the conf file I can manage tables and chains in a structured way. I'm free to use indentations and new lines, and I no longer need to write "-I CHAIN" for every rule. Besides, I can group similar rules (e.g. same rule for different tcp ports) easily, and I can define variables and reuse them.  Eventually I was able to write...

 There are three ways of running Docker: Privileged: dockerd run with root, container root = host root Unprivileged: dockerd run with root, container root = mapped user Rootless: dockerd run with some user, container root = some user I've been hestitating between Unprivileged and Rootless. On one hand, rootless sounds like a great idea; on the other hand, some considers unprivileged user namespace as a security risk . Today I decided to migrate all my unprivileged containers to rootless ones. I had to enable unprivileged user namespace for a rootless LXC container anyways. A Cryptic Issue The migration is overall smooth, except for a cryptic issue: sometimes DNS does not work inside the container. The symptom is rather unusual: curl works but apt-get does not work. For quite a while I'd thought that apt-get uses some special DNS mechanism. After some debugging, especially comparing files /etc/ between a unprivileged container and a rootless container, I realized that non-root u...

 I have an NGINX server that is publicly accessible. It has been deployed in the following manner: Machine A Port forwarding with socat: localhost:4443 ==>  0.0.0.0:443 Machine B Running NGINX in a Docker container Port forwarding by Docker: <container_ip>:443 ==> localhost:4443 Port forwarding by SSH to Machine A: localhost(B):4443 ==> localhost(A):4443 This in general works. Machine A is published to my domain, and the traffic to 443 is forwarded to NGINX in a few hops. However there is a problem: the NGINX server never sees the real IP of the client, so it is impossible to depoly fail2ban or other IP address based tools. So I wanted to fix it. Step 1: VPN The first step is to connect machine A and B with a VPN. I feel that it would also work without it, but the iptables rules could be more tricky.  WireGuard is my choice. I made a simple setup: Machine A has IP: 10.0.0.2/24 Machine B has IP: 10.0.0.1/24 On both machines, the interface is called wg0, Allo...

Weeks ago I  purchased a secondhand machine. Since then I have been tinkering this little box. The Perfect Media Server  site is a good place to start with. Arch Linux Wiki is my go-to learnning resource, even though I use Ubuntu. Filesystem I'd be super paranoid and careful, as this is my first time manually configuring a disk array. Basically my optoins include: ZFS btrfs Snapraid (or even combnied ZFS/btrfs) Unraid My considerations include: Data integrety, which is the most important. Maintenance. I want everything easy to set up and maintain. Popularity. There will be more doc/tutorial/discussions if the technology is more popular. Eventually I decided to use ZFS with raidz2 on 4 disks.  I also took this chance to learn configuring disk encryption. I decided to use LUKS beneath ZFS. I could have just used ZFS's built-in encryption, but I thought LUKS is fun to learn. It really was. The commands are way more user-friendly that I had expected. Hardening SSH Most popula...

Lesson learned today: An invalid sudoers file can break the sudo command, which in turn prevent the sudoers file from being edited via sudo.  The good practice is to always use visudo to modify sudoers file. In my case I needed to modify a file inside /etc/sudoers.d, where I should have used `visudo -f`. To recover from invalid sudoers files, it is possible to run `pkexec bash` to gain root access. However I got an error "polkit-agent-helper-1: error response to PolicyKit daemon: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: No session for cookie" Solution to this error: Source: https://github.com/NixOS/nixpkgs/issues/18012#issuecomment-335350903 - Open two terminals. (tmux also works) - In terminal #1, get PID by running `echo $$` - In terminal #2, run `pkttyagent --process <PID>` - In terminal #1, run `pkexec bash`

surface pro stylus calibration http://www.reddit.com/r/Surface/comments/18cd96/improved_pen_accuracy_with_a_100_point/ http://www.reddit.com/r/Surface/comments/1cy8hn/this_pen_calibration_is_so_far_the_best_solution/ Search for 'UserLinearityData' in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet and remove 'devicekind=pen'   merge tool: meld   linux虚拟机的硬盘大小不够，扩容以后又调整了分区，于是grub就不能进入了。启动说unknown filesystem然后进入grub rescue。 根据 http://askubuntu.com/questions/142300/fixing-grub-error-error-unknown-filesystem 在grub rescue输入 set prefix=(hd0,5)/boot/grub insmod linux linux (hd0,5)/boot/vmlinuz-3.11.0-15-generic initrd (hd0,5)/boot/initrd.img-3.11.0-15-generic boot 然后grub这步正常了，但是会卡到initramfs/busybox，说找不到init 于是在linux命令后面加上root=/dev/sda5就可以正常进入系统 进入系统以后赶紧用grub-install和update-initramfs再修正一下。

png优化工具：pngnq（有损）和 pngcrush（无损） ubuntu gnome 13.10的input source里找不到输入法：手动修改dconf键值 org.gnome.desktop.input-sources.sources ，在数组里添加 ('ibus','sunpinyin') 和 ('ibus', 'anthy') 。 但是mozc似乎不行，原因不明。观望。 mozc在dconf里要打('ibus', 'mozc-jp')

I need to install Ubuntu 13.10 into an old Linux machine. The BIOS has been locked such that I cannot boot from CD or USB drives. The machine has grub 0.97 and a pretty old CentOS installed on it, which I've got the root access, so I decided to try to load the ubuntu iso file with it. Based on https://help.ubuntu.com/community/Installation/FromLinux I tried to enter the grub commands there, but then it failed with error like this: stdin: error 0 /init: line 7: can't open /dev/sr0: No medium found After spending hours in searching and trying, I found this url: http://askubuntu.com/questions/47076/usb-boot-problems which solved the problem for me. The full commands I used are: root (hd1,0) kernel /casper/vmlinuz.efi boot=casper live-media-path=/casper/ ignore_uuid initrd /casper/initrd.lz This link http://manpages.ubuntu.com/manpages/karmic/man7/casper.7.html describes the parameters for casper