Skip to main content

[Updated] Failed Attempt: NixOS in LXC

[Updates: 2024-10-21]

I found a working solution:

- Disable sandboxing in configuration.nix

- Build a tarball image using `nixos-generator -f lxc`

- Create an LXC container with `lxc-create -t none`

- Modify the config of the LXC container (e.g. specify rootfs path, set unconfined AppArmor)

- Create the rootfs directory and remove all POSIX ACL (setfacl --remove-all)

- Extract the tarball into rootfs/



I planned to try NixOS in LXC.

I have found a few successfull stories: 1, 2, 3. However they are all using Proxmox LXC, and/or the image file is for LXD.


First, I tried to download the official image via lxc-create. The image can boot, but I have trouble running `nix-channel --update`, which complains about sandboxing. I think it's related to unprivileged LXC containers. Further, as part of the nice feature of NixOS, I cannot easily disable sandbox from there.

Second, I tried to build a NixOS image from scratch, using nixos-generators. This is mentioned in the 3rd link above. This tools supports lxc format, however, lxc-create -t local does not like the metadata generated by this tool.


At this point I lost interests and decided the stop. The original idea is I might try NixOS longer if it is easier to maintain than Arch, but it doesn't seems so. If I have to make something work. Maybe I can use the generate image from option 2 to override the rootfs in option 1. Perhaps it'll work.




Labels:

Comments

Post a Comment

Popular posts from this blog

A Rocky Migration: Moving from docker-compose to Podman and gVisor

I've been running a few containers for several years. They were all running under rootless Docker with a single user. Initially, I planned to  migrate the containers to VMs , but I couldn't get a stable workflow after about two months of effort. Later,  gVisor caught my attention , and I decided to migrate to Podman with gVisor instead. The new plan is to run each container with  --userns=auto  and use Quadlet for systemd integration. This approach provides better isolation and makes writing firewall rules easier. I'm now close to migrating all my containers. Here are a couple of rough edges I'd like to share. Network Layout I compared  various networking options  and spent a few hours trying the one-interface-per-group approach before giving up. I settled on a single macvlan network and decided to use static IP addresses for my containers. To prevent a randomly assigned IP address from conflicting with a predefined one, I allocated a large IP range for my ...
Post a Comment
Read More »

Fix Google Security Code

Google Security Code (http://g.co/sc) is one type of 2-step verification. This is particularly useful when security keys and passkeys are not available. I have been using it in my LXC containers, until today I found out that it stopped working. It just kept saying "The code is invalid". It is easy to rule out some factors: The code works on other browsers on my laptop. The code works on other devices that are directly connected to the router. So it appears that Google also checks IP addresses besides the security code. Recently I have IPv6 enabled, so most devices that are directly connected to the router have both IPv4 and IPv6 addresses. But  I only enabled IPv4 for my LXC containers. So I guess when a code is generated by device A and used by device B, Google should be able to check that device A and device B are closely located. But in my case, IPv6 address appears on device A but not on device B, which may look suspicious. To fix the problem, I just needed to disable IPv...
Post a Comment
Read More »

Exploring Immutable Distros and Declarative Management

My current server setup, based on Debian Stable and Docker, has served me reliably for years. It's stable, familiar, and gets the job done. However, an intriguing article I revisited recently about Fedora CoreOS, rpm-ostree, and OSTree native containers sparked my curiosity and sent me down a rabbit hole exploring alternative approaches to system management. Could there be a better way? Core Goals & Requirements Before diving into new technologies, I wanted to define what "better" means for my use case: The base operating system must update automatically and reliably. Hosted services (applications) should be updatable either automatically or manually, depending on the service. Configuration and data files need to be easy to modify, and crucially, automatically tracked and backed up. Current Setup: Debian Stable + Docker My current infrastructure consists of several servers, all running Debian Stable. System Updates are andled automatically via unattended-upgrades. Se...
Post a Comment
Read More »