Intro

MicroVMs are a term referring to running a minimal Virtual Machine. Some of the popular ones are implemented in Rust, and so benefit from the Rust’s memory safety.

Additionally they can provide a much smaller attack surface then traditional VMs since they often expose a more limited API, which can also help speed up boot times due to needing to loading less drivers and kernel modules.

microvm.nix

microvm.nix is a Nix project which aims to help create and manage MicroVMs on NixOS, and currently supports a number of hypervisors:

I have not tried all of the hypervisors, and tend to default to Cloud Hypervisor, or QEMU.

Advantages

  • When used imperatively, deployment of VMs occurs separately from deployment of the Host machine. Meaning nixos-rebuild’s of the host rarely cause downtime for VMs, which is a useful property in certain usecases.
  • Significantly stronger isolation between VM’s and Host, than comparable software such as Containers (docker/podman/nspawn) or systemd based isolation/hardening.
  • Easiest way (in my opinion) to use technologies such as Cloud Hypervisor, Firecracker and Crosvm.

Disadvantages

  • Each VM is a new NixOS Configuration, evaluation of NixOS Configurations have a minimum time-to-eval. So if you have many VMs, even evaluating all the configurations can take a long time and require a lot of RAM (if done concurrently).
    • this is particularly an issue with microvm.nix’s declarative mode, which lets you define MicroVMs inline alongide the Host’s NixOS Configuration.
    • evaluating a host with 20 declarative microvms requires evaluating 21 NixOS configurations, not in paralel.

My Use Cases

Development

Isolate development work in seperate Virtual Machines.

I can (recklessly) update my dev machine with unstable and strange configs, while ensuring I always have a runnable dev virtual machine.

Since the host machine and dev VMs are not updated at the same time, this can allow running the stable channel inside dev machines, and updating occassionally while using unstable on the host and having a more regular cadence for updates.

As long as I don’t fully break workflows on my Host, I always have a stable running Dev environment for client projects.

Pros

  • isolate client projects from each other
    • client specific VPNs don’t affect the rest of my host
    • never accidentally use the wrong credentials between projects
    • tooling between projects cannot affect other projects
    • put down a project for months, and have the full working VM ready to go whenever you need it again
  • fully separate your own work from client work
  • run untrusted toolchains away from all your user data

Cons

  • extra storage usage of /nix/store (when not sharing the /nix/store between host and VM)

Home Server (or internet facing) Services

Some more critical software on my Home Server, I may want to run in a more isolated environment. Particularly if that software is internet facing.

A few useful examples:

  • isolating Jellyfin into a VM so I don’t get yelled at if a deploy causes Jellyfin to restart while someone is actively watching something.
  • isolating an internet facing Nextcloud to reduce the attack surface of a Nextcloud bug opening doors into the rest of my server or homelab.

AI VM

Spin up / down an AI focused VM with GPU Passthrough.

If you want to experiment with all this fancy AI-related technology, you likely will have to hack around and not necessarily use Nix, or run not very trusted (Rube Goldberg (Python) Machines) software.

I prefer to isolate all this away from my work and the rest of my data, as much as possible.

Gaming VM (WIP)

I have used Virtual Machine Manager and libvirt for many years to run my Windows Gaming VM, since before I used NixOS.

My main motivation was wanting to play games (before Proton got good) on my main gaming machine, which used to be my most powerful machine, and not wanting to dual-boot and run Windows on bare metal anymore since it had increasingly started evolving from an Operating System into Malware designed for surveillance (which has gotten orders of magnitude worse since I started running the Gaming VM).

Now that I am using microvm.nix more and more, I want to remove the “legacy” use-case of using Windows for many games, and simply run a NixOS-based gaming rig, and also deprecate separately managing my Gaming VM via Virtual Machine Manager, and start using microvm.nix to declaratively manage my Gaming VM.