← Notes

Backing up OpenWrt hosts and their VMs to Proxmox Backup Server

· openwrt, proxmox, backup, qemu, ucode

Some of my OpenWrt boxes are x86 machines that also run a VM or two under plain QEMU. Everything else in the homelab is backed up to Proxmox Backup Server, so these should be too. Two packages in my OpenWrt feed do that.

The client

PBS backups need proxmox-backup-client, which is Rust and not in OpenWrt. The feed’s proxmox-backup package builds just the client from the upstream sources, with a few patches: a client-only cargo build, no libsystemd, and the system zstd. So far I have only tested it on x86_64.

pbs-helper: backing up the host

pbs-helper adds pbs-backup, which backs up the router’s root filesystem as a .pxar archive. Configuration is UCI:

# /etc/config/pbs
config proxmox_backup_server 'pbs_server'
	option enabled '1'
	option repository 'backup@pbs@pbs.example.org:datastore'
	option password '...'
	option backup_name 'rootfs'
	option prefix '/'
	list backup_arg '--exclude=/tmp'
	option hc_ping_url 'https://hc-ping.com/<uuid>'
	option gotify_url 'https://gotify.example.org'
	option gotify_app_token '...'

and a cron line such as 0 1 * * * /usr/bin/pbs-backup.

A few details I care about:

The job runner behind it, pbs.helper, is a small ucode module shared with the VM backups below. All side effects — running commands, HTTP, logging — are injected, so it is unit-tested without a router.

qemu-instance-init: VMs as services

qemu-instance-init started from the init script on the OpenWrt wiki, ported to procd. Each VM is a UCI section:

# /etc/config/qemu
config instance 'vm1'
	option vcpus '2'
	option ram '2G'
	option root_disk '/dev/vg0/vm1'
	option root_disk_type 'lvm'
	option net0_mac '52:54:00:12:34:56'
	option net0_bridge 'br-lan'
	option guest_agent '1'
	option backup '1'
	option backup_server 'pbs_server'

procd starts and respawns the guests. On stop the script sends an ACPI system_powerdown over QMP and gives the guest time to shut down cleanly before procd kills it. UEFI guests work with OVMF, and the guest agent socket is one option away.

The package comes in two variants. qemu-instance-init is just the init script. qemu-instance-init-pbs adds /etc/init.d/qemu backup [instance], which runs through the same pbs.helper job runner, with the same Healthchecks and Gotify reports.

Consistent backups of running VMs

Reading the disk of a running VM gives you a backup of a disk that was changing while you read it. The -pbs variant keeps it still:

  1. Freeze the guest filesystems through the guest agent, so the guest flushes its caches and stops writing.
  2. Snapshot the disk:
    • LVM disks get an lvcreate -s snapshot. OpenWrt’s kmod-dm is built without snapshot support, so the feed has a small kmod-dm-snapshot package for that.
    • qcow2 disks get a temporary overlay through QMP blockdev-snapshot-sync: guest writes go to the overlay, and the base file stops changing.
  3. Thaw right away — the guest is frozen only for the snapshot itself.
  4. Back up the snapshot or the base file.
  5. Clean up: remove the LVM snapshot, or merge the overlay back with block-commit and delete it.

Missing pieces degrade with a warning in the report rather than failing: without the guest agent you get a crash-consistent snapshot, without a snapshot a flush and then the live disk. Leftovers are handled too — a stale LVM snapshot is removed by the next backup, and an overlay left behind by a killed backup is merged back by the next backup or, if the VM was stopped, on the next start.