Skip to main content

Linux Daily Driver Setup Part 3: VM Control Panel

Now that I have my VMs running, what if I want to switch between them? What if I want to put the VM or the host to sleep, or shut them down?

In this post, I’ll discuss a few options.

Just a quick overview of my setup:

  • Only one VM is running at a time.
  • All VMs have GPU passthrough, and the GPU is connected to a display.
  • I have a secondary display, which I prefer not to use unless absolutely necessary.

See also:

Option 1: Deep Integration with Guest OS

Ideally, I could just click on a “Power off and Switch to VM X” menu from within the guest OS.

This isn’t difficult to support on the host side: the guest OS could pass the VM name to the host (e.g., via a serial port), and the host would automatically start the next VM when the current one exits.

However, I didn’t find an easy way to implement this menu, especially considering I plan to support multiple operating systems. I also couldn’t find a way to prevent the guest from shutting down without providing the next VM’s name.

It sounds like it would require a lot of custom scripting, so I decided to pass on this idea.

Option 2: GPU Rebinding + TTY Menu

I figured the logic had to be implemented on the host side. The idea here is that the host reclaims the GPU when a VM exits.

This requires two steps:

  1. Rebind the GPU from vfio-pci to a “real” driver after a VM exits.
  2. Rebind the GPU from the “real” driver back to vfio-pci before a VM starts.

Step 1 is generally easy because vfio-pci is pretty lightweight, but Step 2 is not, especially for nouveau. I also needed to make sure to disconnect all possible GPU usages (e.g., fbcon), otherwise the driver would likely hang.

In the end, I couldn’t make it stable enough, and I definitely didn’t want to risk hanging the host. Time for a new option.

Option 3: Menu in VM

Since I prefer to stick with the vfio-pci driver, a natural idea was to launch a dedicated VM just for the menu.

The kernel and rootfs could be heavily stripped down to do nothing but show a menu and pass the user’s choice back to the host. It’s a bit of work, but doable.

In practice, however, a minimal VM took about 6 seconds to boot. I managed to pinpoint the bottleneck: GPU initialization. The VM boots much faster without it.

Ultimately, I couldn’t find a good solution. My only finding was that OVMF/UEFI is required to initialize the GPU; without it, the GPU won’t work and the kernel might just hang (the RIP register doesn’t change). Using a dumped or downloaded ROM file didn’t help in my case.

I didn’t bother implementing the menu because the boot delay was just too slow for practical use.

Option 4: Web Server

Another idea was to implement a web server with a simple UI, allowing me to control the host using my phone.

I don’t think it would be difficult to build something that just works, but making it secure enough would be a challenge. Plus, I don’t like that it requires a second device.

Option 5: TTY Menu in Secondary Monitor

Running out of ideas for reusing the primary monitor, I decided to compromise and use the secondary display.

It was straightforward to implement:

  • The menu is built using whiptail.
  • Mask the default getty@tty2 service and run my menu service on TTY2.
  • Use chvt to switch terminals, and setfont to set a huge font size.
  • Write to /sys/module/kernel/parameters/consoleblank to make the screen turn off automatically.
  • The screen turns on automatically when a VM stops, probably due to keyboard/mouse events after evdev passthrough switching.
  • I also added a few options to the menu like “sleep” and “power off”, so I can control the host without having to log in.

In practice, this works really well. It just requires that secondary display.

Thoughts

While Option 5 ended up being the best compromise, I really wish Option 2 or 3 had worked better. That way, the entire setup would work on a single display. Maybe there is a better solution for GPU initialization out there. I’ll probably revisit this later.

Labels:

Comments

Post a Comment

Popular posts from this blog

A Rocky Migration: Moving from docker-compose to Podman and gVisor

I've been running a few containers for several years. They were all running under rootless Docker with a single user. Initially, I planned to  migrate the containers to VMs , but I couldn't get a stable workflow after about two months of effort. Later,  gVisor caught my attention , and I decided to migrate to Podman with gVisor instead. The new plan is to run each container with  --userns=auto  and use Quadlet for systemd integration. This approach provides better isolation and makes writing firewall rules easier. I'm now close to migrating all my containers. Here are a couple of rough edges I'd like to share. Network Layout I compared  various networking options  and spent a few hours trying the one-interface-per-group approach before giving up. I settled on a single macvlan network and decided to use static IP addresses for my containers. To prevent a randomly assigned IP address from conflicting with a predefined one, I allocated a large IP range for my ...
Post a Comment
Read More »

GameConqueror 0.09 -- Linux Game Hacking Tool

If you are a game hacker If you've been looking for a `CheatEngine for Linux` Then you can't miss this. ============================================== GameConqueror is a game hacking tool for linux, it's written in PyGTK and uses scanmem as its backend. It's supposed to be with most useful features of CheatEngine for Linux. Currently, I've implemented almost everything about scanning, involving variant data types and scan types: Data Types: int{8/16/32/64}, float{32/64}, unknown type(int or float) and unknown width(will try each of them), byte array and string Scan Types: equal, greater, less, changed, unchanged, increased(by), decreased(by) This should be enough for most cases, so I decided to release it at the current status. ============================================= Here's how you can get it PPA (for Ubuntu users) https://launchpad.net/~coolwanglu/+archive/scanmem (I've not test it in 32bit environments or Jaunty, do please inform me if it doe...
Post a Comment
Read More »

Fix Google Security Code

Google Security Code (http://g.co/sc) is one type of 2-step verification. This is particularly useful when security keys and passkeys are not available. I have been using it in my LXC containers, until today I found out that it stopped working. It just kept saying "The code is invalid". It is easy to rule out some factors: The code works on other browsers on my laptop. The code works on other devices that are directly connected to the router. So it appears that Google also checks IP addresses besides the security code. Recently I have IPv6 enabled, so most devices that are directly connected to the router have both IPv4 and IPv6 addresses. But  I only enabled IPv4 for my LXC containers. So I guess when a code is generated by device A and used by device B, Google should be able to check that device A and device B are closely located. But in my case, IPv6 address appears on device A but not on device B, which may look suspicious. To fix the problem, I just needed to disable IPv...
Post a Comment
Read More »