WangLu's Notes

Today, various services (native, LXC, Docker) are running on my server. I'm mostly happy with the setup, but I decided to revisit my server's defenses under the assumption that a remote attacker or malicious code could compromise my services. A service might break out of its container or even gain root privilege. VMs are a better security boundary than containers; they can limit the damage if an attacker gains root privilege. I cannot afford to run a dedicated VM for each service, so I will need to carefully group the services and run a dedicated VM for each group. Each group should be carefully designed based on the data accessed and the features/capabilities required. For example, some VMs may have access to my photos, while others may not have network access. The Goal There are two particular issues I want to address: First, I want VM images to be easily reproducible, which makes backup and restore trivial. NixOS and GNU Guix System are great examples, where you only need to...

I decided to spend some time playing with bootc. Mostly I'm inspired by the following articles:  CoreOS + native container Hand-on demo (the last video), build bootc and auto update from registry bootc desktop bootc for homelab Day 1 To install bootc in a VM I need an image. bootc-image-builder requires root and I don't want to run this on the host. So I chose CoreOS as the inital system and installed it to QEMU. I thought it is a great idea to share a folder from host to guest as podman container storage. However, it was not as smooth as I had expected: virtiofsd on Debian is too old, so I set up NFS. rootless podman doesn't work well with NFS . rootfull podman complains upstream fs of overlayfs missing features, the performance was terrible. I gave up. I guess I'll just use the CoreOS disk, whose size is 10G, not enough. Day 2 I didn't find a way of resizing a qcow2 image online. On the other hand I figured maybe I don't need build a disk image after all. Cor...

When I have important data on a device, I back it up to my server using dedicated user accounts. The other day, I checked /etc/passwd on my server and found entries like this: some-backup-user1:x:1003:1004:... some-backup-user2:x:1004:1007:... A few inconsistencies immediately bothered me: UID/GID Mismatches: Many users have UIDs that don't match their primary GIDs. While this technically works and might seem like just an aesthetic concern, I realized that UIDs and GIDs are crucial metadata. I need to preserve them accurately for future system migrations to maintain correct file ownership.ID Ambiguity: ID Ambiguity: The same number (e.g., 1004) could represent a User ID for one account and a Group ID for a completely different group. This overlap is a recipe for mistakes during administration tasks if I'm not paying close attention. Lack of Structure: User and group accounts created for very different purposes – regular logins, backup processes, container management, specifi...

My current server setup, based on Debian Stable and Docker, has served me reliably for years. It's stable, familiar, and gets the job done. However, an intriguing article I revisited recently about Fedora CoreOS, rpm-ostree, and OSTree native containers sparked my curiosity and sent me down a rabbit hole exploring alternative approaches to system management. Could there be a better way? Core Goals & Requirements Before diving into new technologies, I wanted to define what "better" means for my use case: The base operating system must update automatically and reliably. Hosted services (applications) should be updatable either automatically or manually, depending on the service. Configuration and data files need to be easy to modify, and crucially, automatically tracked and backed up. Current Setup: Debian Stable + Docker My current infrastructure consists of several servers, all running Debian Stable. System Updates are andled automatically via unattended-upgrades. Se...

A few days ago, while browsing security topics online, Qubes OS surfaced—whether via YouTube recommendations or search results, I can't recall precisely. Intrigued by its unique approach to security through compartmentalization, I delved into the documentation and watched some demos. My interest was piqued enough that I felt compelled to install it and give it a try firsthand. My overall first impression of Qubes OS is highly positive. Had I discovered it earlier, I might have reconsidered starting my hardware password manager project. Conceptually, Qubes OS is not much different from running a bunch of virtual machines simultaneously. However, its brilliance lies in the seamless desktop integration and the well-designed template system, making it far more user-friendly than a manual VM setup. I was particularly impressed by the concept of disposable VMs for temporary tasks and the clear separation of critical functions like networking (sys-net) and USB handling (sys-usb) into the...

Windows 10 will soon reach its end of life, and my 1-gen Surface Pro is not supported by Windows 11. I (finally) decided to install Linux to it. Fortunately, it's a not-so-easy nice adventure: The device has only one USB port, so I have to bring back my 10+-year old USB hub. My live USB drive cannot boot directly, I have to disable Secure Boot first, by holding Volume Up during boot. I think years ago I learned that booting on USB might not work through a USB hub, but fortunatelly it worked well with my setup. This is done by holding Volume Down during boot. Wifi adapter was detected in the live Linux environment, but not functional. And I don't have a USB-Ethernet adapter. Luckily, nowadays we have USB-tethering from Android phones, which works out-of-the-box. Originally I planned to following this guide to set up root on ZFS, however, the system froze when building the ZFS kernel module. Then I decided to just use EXT4, yet I still learned a lot from the guide about disk par...

[UPDATE] Later I was able to successfully migrate an amd64 Ubuntu server to Debian, following the same procedure. I had a Ubuntu 22.04.3 on a Raspberry Pi 4. Recently I decided to switch it to Debian 12.5 in-place . Mostly I was following this script: https://github.com/alexmyczko/autoexec.bat/blob/master/config.sys/ubuntu-deluxe   It worked surprisingly well, however, the system was eventually broken (unsurprisingly) near the end, when I tried to upgrade the kernel. I tried to fix initramfs by copying dtbs files, which seemed to work. Some references: https://qiita.com/takasan/items/ef93be9e9d3f791eee66 https://bugs.launchpad.net/ubuntu/+source/flash-kernel/+bug/2012750 But the kernel did not load after reboot. Reverting /boot/config.txt didn't help. Then I downloaded a Debian image and replaced only the boot partition of the micro SD card. This time the kernel was able to boot, but it couldn't load the filesystem. I modified cmdline.txt, replaced "root=LABEL=RASPIROOT...

我的小服务器上一直装了个Ubuntu Desktop，不过安装之后一直没用过GUI，而且各种依赖的包有时也挺烦人的，比如gvfs和tracker自带的systemd user service，我还得手动给若干用户禁用掉。 本来我是想着留个Desktop，万一紧急情况可以上网查查命令。不过有网的话，最差情况我应该也能临时装一个X和浏览器，估计问题不太大。 于是我决定把Ubuntu Desktop换成Ubuntu Server，主要还是把gnome的包都删了。 一番折腾以后，安装包的数量从大约1800降到了800以下。舒服！

I recently read this article , which talks about restricting (proactive) internet access of a process. It is easy to completely disable internet/network access, by throwing a process into a new private network namespace. I think all popular sandboxing tools support it nowadays: unshare -n bwrap --unshare-net systemd.service has PrivateNetwork=yes docker has internal network But the trickier, and more realistic scenario is: [Inbound] The process needs to listen one or more ports, and/or [Outbound] The process needs to access one or more specific IP address/domain I can think of a few options. Option 1: Firewall Rules Both iptables and nftables support filter packets by uid and gid. So the steps are clear: Run the process with a dedicate uid and/or gid Filter packets in the firewall If needs, regularly query DNS and update the allowed set of IP addresses. This is somehow similar to reresolve-dns.sh from WireGuard. This option is not very complicated, and I think the overhead is low....

nftables has been enabled by default in latest Ubuntu and Debian, but not fully supported by Docker. I've been hestitating about migrating from iptables to nftables, but managed to do it today. Here are my thoughts. Scripting nftables The syntax of iptables and nftables are different, but not that different, both are more or less human readable. However, nftables is clearly more friendly for scripting. I spent quite some time in a python script to generate a iptables rule set, and I was worried that I need lots of time migrating the script. Aftering studying the syntax of nftables, I realized that I could just write /etc/nftables.conf directly.  In the conf file I can manage tables and chains in a structured way. I'm free to use indentations and new lines, and I no longer need to write "-I CHAIN" for every rule. Besides, I can group similar rules (e.g. same rule for different tcp ports) easily, and I can define variables and reuse them.  Eventually I was able to write...

 There are three ways of running Docker: Privileged: dockerd run with root, container root = host root Unprivileged: dockerd run with root, container root = mapped user Rootless: dockerd run with some user, container root = some user I've been hestitating between Unprivileged and Rootless. On one hand, rootless sounds like a great idea; on the other hand, some considers unprivileged user namespace as a security risk . Today I decided to migrate all my unprivileged containers to rootless ones. I had to enable unprivileged user namespace for a rootless LXC container anyways. A Cryptic Issue The migration is overall smooth, except for a cryptic issue: sometimes DNS does not work inside the container. The symptom is rather unusual: curl works but apt-get does not work. For quite a while I'd thought that apt-get uses some special DNS mechanism. After some debugging, especially comparing files /etc/ between a unprivileged container and a rootless container, I realized that non-root u...

 I have an NGINX server that is publicly accessible. It has been deployed in the following manner: Machine A Port forwarding with socat: localhost:4443 ==>  0.0.0.0:443 Machine B Running NGINX in a Docker container Port forwarding by Docker: <container_ip>:443 ==> localhost:4443 Port forwarding by SSH to Machine A: localhost(B):4443 ==> localhost(A):4443 This in general works. Machine A is published to my domain, and the traffic to 443 is forwarded to NGINX in a few hops. However there is a problem: the NGINX server never sees the real IP of the client, so it is impossible to depoly fail2ban or other IP address based tools. So I wanted to fix it. Step 1: VPN The first step is to connect machine A and B with a VPN. I feel that it would also work without it, but the iptables rules could be more tricky.  WireGuard is my choice. I made a simple setup: Machine A has IP: 10.0.0.2/24 Machine B has IP: 10.0.0.1/24 On both machines, the interface is called wg0, Allo...

Weeks ago I  purchased a secondhand machine. Since then I have been tinkering this little box. The Perfect Media Server  site is a good place to start with. Arch Linux Wiki is my go-to learnning resource, even though I use Ubuntu. Filesystem I'd be super paranoid and careful, as this is my first time manually configuring a disk array. Basically my optoins include: ZFS btrfs Snapraid (or even combnied ZFS/btrfs) Unraid My considerations include: Data integrety, which is the most important. Maintenance. I want everything easy to set up and maintain. Popularity. There will be more doc/tutorial/discussions if the technology is more popular. Eventually I decided to use ZFS with raidz2 on 4 disks.  I also took this chance to learn configuring disk encryption. I decided to use LUKS beneath ZFS. I could have just used ZFS's built-in encryption, but I thought LUKS is fun to learn. It really was. The commands are way more user-friendly that I had expected. Hardening SSH Most popula...

Lesson learned today: An invalid sudoers file can break the sudo command, which in turn prevent the sudoers file from being edited via sudo.  The good practice is to always use visudo to modify sudoers file. In my case I needed to modify a file inside /etc/sudoers.d, where I should have used `visudo -f`. To recover from invalid sudoers files, it is possible to run `pkexec bash` to gain root access. However I got an error "polkit-agent-helper-1: error response to PolicyKit daemon: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: No session for cookie" Solution to this error: Source: https://github.com/NixOS/nixpkgs/issues/18012#issuecomment-335350903 - Open two terminals. (tmux also works) - In terminal #1, get PID by running `echo $$` - In terminal #2, run `pkttyagent --process <PID>` - In terminal #1, run `pkexec bash`

surface pro stylus calibration http://www.reddit.com/r/Surface/comments/18cd96/improved_pen_accuracy_with_a_100_point/ http://www.reddit.com/r/Surface/comments/1cy8hn/this_pen_calibration_is_so_far_the_best_solution/ Search for 'UserLinearityData' in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet and remove 'devicekind=pen'   merge tool: meld   linux虚拟机的硬盘大小不够，扩容以后又调整了分区，于是grub就不能进入了。启动说unknown filesystem然后进入grub rescue。 根据 http://askubuntu.com/questions/142300/fixing-grub-error-error-unknown-filesystem 在grub rescue输入 set prefix=(hd0,5)/boot/grub insmod linux linux (hd0,5)/boot/vmlinuz-3.11.0-15-generic initrd (hd0,5)/boot/initrd.img-3.11.0-15-generic boot 然后grub这步正常了，但是会卡到initramfs/busybox，说找不到init 于是在linux命令后面加上root=/dev/sda5就可以正常进入系统 进入系统以后赶紧用grub-install和update-initramfs再修正一下。

png优化工具：pngnq（有损）和 pngcrush（无损） ubuntu gnome 13.10的input source里找不到输入法：手动修改dconf键值 org.gnome.desktop.input-sources.sources ，在数组里添加 ('ibus','sunpinyin') 和 ('ibus', 'anthy') 。 但是mozc似乎不行，原因不明。观望。 mozc在dconf里要打('ibus', 'mozc-jp')

I need to install Ubuntu 13.10 into an old Linux machine. The BIOS has been locked such that I cannot boot from CD or USB drives. The machine has grub 0.97 and a pretty old CentOS installed on it, which I've got the root access, so I decided to try to load the ubuntu iso file with it. Based on https://help.ubuntu.com/community/Installation/FromLinux I tried to enter the grub commands there, but then it failed with error like this: stdin: error 0 /init: line 7: can't open /dev/sr0: No medium found After spending hours in searching and trying, I found this url: http://askubuntu.com/questions/47076/usb-boot-problems which solved the problem for me. The full commands I used are: root (hd1,0) kernel /casper/vmlinuz.efi boot=casper live-media-path=/casper/ ignore_uuid initrd /casper/initrd.lz This link http://manpages.ubuntu.com/manpages/karmic/man7/casper.7.html describes the parameters for casper

- To remove the 'new release available' message, remove the file /etc/update-motd.d/91-release-upgrade http://blog.koppi.me/2011/12/howto-disable-new-release-available-on-ubuntu-11-10/ - To remove unused language packs from Windows 8, run `lpksetup` http://answers.microsoft.com/en-us/windows/forum/windows_7-windows_programs/how-to-uninstall-extra-language-packs-from-windows/9ac2194d-1665-498d-8571-32e4cce6efa9 - If Cygwin X server crashed with `-multiwindow` on HD4000 graphics card, try updating the graphics card driver http://serverfault.com/questions/488306/cygwin-x-crashing-on-multiwindow-option/488556#488556 - halt vs poweroff: halt does not send the acpi power off signal http://unix.stackexchange.com/questions/42572/halt-and-poweroff-options-for-shutdown-command - MyPaint picker offset on windows 64bit with high dpi - Also how to set the high dpi compatibility setting for 64bit programs http://forum.intilinux.com/mypaint-help-and-tips/color-pick...

To reset all shortcut definitions in evince, run rm ~/.config/evince/accels GCC would complain if you define a variable without actually using it. To silience it, use (void)var; DNS is not working in VirtualBox: Run VBoxManage modifyvm "VM name" --natdnshostresolver1 on Ref   A useful link talking about ravlue reference in C++11 http://thbecker.net/articles/rvalue_references/section_01.html   To clear disk cache in Linux: Run sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' Ref  "screen-256color not found": Run infocmp xterm-256color > screen-256color.ti on a machine without this error, copy the file to the target machine and then run tic screen-256color.ti Ref Just switch to fcitx from ibus. I like a theme called 'anran'. If fcitx cannot follow the cursor on Firefox, turn on preedit, as said in the official wiki. Just played with a vim plugin called youcompleteme, for C++ autocomplete, which is quite powerful. Also I installed vund...

之前evince的快捷键乱套了，使用 rm ~/.config/evince/accels 复位 定义了变量var又不用的话，gcc会报warning"unused variable"，解法是 (void)var; VirtualBox中DNS不工作：运行 VBoxManage modifyvm "VM name" --natdnshostresolver1 on Ref   一个讲c++11右值引用的link： http://thbecker.net/articles/rvalue_references/section_01.html   Linux清除disk cache: sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' Ref  遇到错误screen-256color not found需要支持新的终端类型：找一台支持screen-256color的机器执行 infocmp xterm-256color > screen-256color.ti 把文件拷贝到目标机器后执行 tic screen-256color.ti Ref  最近从ibus换到了fcitx，起初是由于sogou发布了，不过后来还是卸掉了。anran主题不错。fcitx在firefox中光标跟随有点问题，按官方wiki说法打开preedit后就没事了。  最近在vim里折腾youcompleteme这个补全插件，挺好用。顺便也装了vundle来管理插件，很不错  vim更新.vimrc后想直接应用，执行 so %