Skip to main content

An Adventure with Qubes OS

I've been experimenting with Qubes OS on my new laptop and wanted to share some notes on the experience.

Hardware

Overall, Qubes OS works quite well on my hardware. Aside from typical issues like deep sleep, speaker performance, and touchpad scroll speed, the experience has been smooth. I particularly like that I can boot directly from a microSD card. This allowed me to move the /boot partition to the card while completely disabling USB access in dom0 for better security.

Detached /boot and LUKS Header

Moving /boot and the LUKS header to a microSD card is a fun project, but it has some drawbacks:

  • I have to remember to mount /boot before updating dom0.
  • The system won't shut down properly if I forget to unmount /boot.

Testing Qubes OS 4.3 rc3

I decided to test the Qubes OS 4.3 rc3 release by performing an in-place upgrade. Unfortunately, the system failed to boot afterward.

dracut Issues

After the upgrade, the system would hang before prompting me for my LUKS password. Eventually, the watchdog timer would kick in and drop me into an emergency shell.

Using the emergency shell and the installation media, I was able to investigate. I realized something was wrong with dracut, as it seemed unable to detect the encrypted disk. I tried including more files in the dracut configuration based on information from various sources, but that didn't help. So I gave up. Thankfully, the upgrade tool created an LVM snapshot, which made it easy to revert the changes. I did have to manually downgrade the kernel and Xen using the installation media to get my Xen domains working again.

After more research, I found a bug (1, 2) in the version of dracut included in Qubes OS 4.3. Essentially, the crypt module stops working when systemd is available. I also believe the systemd-cryptsetup module wasn't automatically included because the LUKS header was on a separate device, leading dracut to assume that LUKS decryption wasn't needed.

The fix was simple: manually enable the systemd-cryptsetup module in the dracut configuration.

amdgpu Issues

After the upgrade, only the old kernel worked, that's how I could fix the dracut issue. With the new kernels, the system would boot to a blank screen. Removing the rhgb and quiet kernel parameters revealed some log messages, but the blank screen remained.

Adding the nomodeset parameter disabled the graphics driver, which allowed me to enter the LUKS password and log in. However, this caused Xorg and LightDM to fail, with Xorg repeatedly crashing.

It was clear this was an issue with the amdgpu module. I tried several kernel parameters without success:

  • amdgpu.modeset=1
  • amdgpu.dc=0
  • amdgpu.dpm=0
  • amdgpu.ppfeaturemask=0xffffb

Eventually, I found that amdgpu.dcdebugmask=0x10 resolved the problem.

qubesd Issues

After finally booting into the system, I couldn't attach any block devices, including my /boot partition, to dom0. This turned out to be a bug in qubesd, which I reported.

Backup Strategy

Backing up data in Qubes OS can be tricky due to its design. Here is the high-level strategy I've been planning:

  • Create as few templates as possible.
  • Only use Salt to configure templates. So I only need to back up Salt files in dom0.
  • Changes to dom0 are either managed by Salt, or the relevant files are included in the backup. Examples
    • Xfce settings.
    • Qube settings/features, /etc/qubes
    • /etc/default/grub
    • /etc/dracut.conf.d

My backup process is as follows:

  • I use a dedicated disposable VM with restic installed. The actual backup scripts and SSH keys remain in dom0.
  • Data from dom0 and other qubes is archived using tar. Use --transform to prepend the VM name to the path.
  • Use qvm-run --pass-io --no-gui to pass the scripts, SSH keys an data to the disposable VM, which then runs the scripts to execute the backup.

There are a few things to consider:

  • Using --pass-io could have security implications. It might be possible to mitigate this by limiting the number of bytes passed and saving output to a file.
  • The tar archives are currently extracted in the disposable VM. If the data isn't trusted, this could be a security risk. In such cases, the extraction step could be skipped.
  • Accessing the backed-up data requires starting a VM. An alternative could be to create an LVM snapshot and back that up directly.

Thought: Manging Qubes like Containers

There are many similarities between Qubes VMS and containers (docker, podman):

  • Template VMs are similar to building containers.
  • AppVMs are like running containers with persistent volumes.
  • Disposable VMs are like running containers without persistent volumes.

There are some gaps:

  • One container can be based on others, the shared part are often stored only once as layers. While cloning a template VM can use reflink for initial efficienty, changes will evetually cause the data to diverge. bootc might help bridge this gap.
  • Containers support bind mounts, which is very convenient for backup. While virtiofsd could work for Xen, but I guess there are security concerns. An alternative could be to centralize all data in one qube and share it with others over the network, but again, there could be security concerns.

Conclusion

It's been a fun and challenging journey exploring Qubes OS. While there were some hurdles with hardware and system upgrades, working through them has been a valuable learning experience. The security architecture is powerful, and I'm excited to continue finding new ways to make it work for my setup.

Labels:

Comments

Post a Comment

Popular posts from this blog

A Rocky Migration: Moving from docker-compose to Podman and gVisor

I've been running a few containers for several years. They were all running under rootless Docker with a single user. Initially, I planned to  migrate the containers to VMs , but I couldn't get a stable workflow after about two months of effort. Later,  gVisor caught my attention , and I decided to migrate to Podman with gVisor instead. The new plan is to run each container with  --userns=auto  and use Quadlet for systemd integration. This approach provides better isolation and makes writing firewall rules easier. I'm now close to migrating all my containers. Here are a couple of rough edges I'd like to share. Network Layout I compared  various networking options  and spent a few hours trying the one-interface-per-group approach before giving up. I settled on a single macvlan network and decided to use static IP addresses for my containers. To prevent a randomly assigned IP address from conflicting with a predefined one, I allocated a large IP range for my ...
Post a Comment
Read More »

GameConqueror 0.09 -- Linux Game Hacking Tool

If you are a game hacker If you've been looking for a `CheatEngine for Linux` Then you can't miss this. ============================================== GameConqueror is a game hacking tool for linux, it's written in PyGTK and uses scanmem as its backend. It's supposed to be with most useful features of CheatEngine for Linux. Currently, I've implemented almost everything about scanning, involving variant data types and scan types: Data Types: int{8/16/32/64}, float{32/64}, unknown type(int or float) and unknown width(will try each of them), byte array and string Scan Types: equal, greater, less, changed, unchanged, increased(by), decreased(by) This should be enough for most cases, so I decided to release it at the current status. ============================================= Here's how you can get it PPA (for Ubuntu users) https://launchpad.net/~coolwanglu/+archive/scanmem (I've not test it in 32bit environments or Jaunty, do please inform me if it doe...
Post a Comment
Read More »

Fix Google Security Code

Google Security Code (http://g.co/sc) is one type of 2-step verification. This is particularly useful when security keys and passkeys are not available. I have been using it in my LXC containers, until today I found out that it stopped working. It just kept saying "The code is invalid". It is easy to rule out some factors: The code works on other browsers on my laptop. The code works on other devices that are directly connected to the router. So it appears that Google also checks IP addresses besides the security code. Recently I have IPv6 enabled, so most devices that are directly connected to the router have both IPv4 and IPv6 addresses. But  I only enabled IPv4 for my LXC containers. So I guess when a code is generated by device A and used by device B, Google should be able to check that device A and device B are closely located. But in my case, IPv6 address appears on device A but not on device B, which may look suspicious. To fix the problem, I just needed to disable IPv...
Post a Comment
Read More »