Backup concepts

Backup points, sessions, change tracking, exports and restores of Kyvernon, and what the controller and backup software each keep.

Kyvernon provides the means to back up virtual machines without agents in the guest: a consistent point in time of every disk, the list of blocks that changed since the last backup, and a read-only network export of the disk at that point. The controller can take and keep backups itself: it writes them to a repository, runs schedules and keeps the newest points. Backup software can instead hold its own repository, the schedule and the retention, and use the backup API of the controller (/v1/backup) to take and read each backup.

Controller backups

A backup that the controller takes writes every disk of a virtual machine into the repository as a backup point. A point is incremental when the previous point has change ids: the controller copies the previous image and writes the changed areas onto the copy, so every point holds whole images and can be restored or deleted on its own. See Back up a virtual machine.

Sessions and backup points

A backup starts a session for one virtual machine. The controller asks the host to create a backup point: the content of all selected disks at one instant, taken in one step on the host, with an optional freeze of the guest file systems. What the software reads is the disk at that instant, whatever the virtual machine writes meanwhile. No snapshot appears in the snapshot tree of the virtual machine.

A session moves through the states REQUESTED, SNAPSHOT_TAKEN, EXPORTING and COMPLETED. It ends as ABORTED when the client reports a failure or the host fails, and as EXPIRED when it is not kept alive within its lifetime (6 hours by default, at most 48 hours). A virtual machine has at most one active session. The platform keeps the last 1,024 finished sessions readable.

Consistency

The request sets quiesce. NONE takes a crash-consistent point. PREFERRED, the default, freezes the file systems through the guest agent when it runs and otherwise takes a crash-consistent point. REQUIRED fails when the freeze fails. The session reports what it achieved as CRASH_CONSISTENT or FILESYSTEM_CONSISTENT.

While software reads a slow export, a copy-before-write image (fleecing) can protect the running virtual machine: guest writes to blocks not yet read go to the disk after the old content is copied aside.

Change tracking and change ids

Kyvernon VMM keeps a record of the changed blocks of every writable disk (cbt = true in agent.toml, the default). Each completed backup returns a change id per disk, an opaque string. The next session passes it back and receives only the areas that changed since. The value * asks for everything that is allocated, which is a full backup of a thin disk without reading its holes.

Areas are listed in pages, in ascending order, at a granularity of 64 KiB. A change id stays valid across power-off, host restart, live migration and the move of a disk to another datastore. It becomes stale, and the next backup must be full, after a reset of change tracking, a resize of the disk, a revert to a snapshot, a restore onto the disk, loss of the tracking data, or when newer change ids push it out (the newest 4 per disk are kept). With fallbackToFull a stale change id gives a full backup; without it the request fails with backup.change_id.stale.

Export of the disk

For each disk of a session in the state EXPORTING the host serves a read-only export over the NBD protocol. An export to another machine always uses TLS 1.3. The certificate comes from the platform CA and its fingerprint is in the session, so the client can pin it. A backup proxy on the same host can use a Unix socket instead. The session names the export, and its token is shown only to the user that started the session. The export lasts until the session ends or expires.

Off-host exports are off until the controller setting remote_exports in [backup] is on.

Restore

A restore writes a full image into a new disk of a virtual machine. The software creates or chooses a powered-off virtual machine, asks for a writable export of a disk of the stated capacity, writes the image and commits it. The restored disk starts a new tracking period, so the first backup after a restore is full. Kyvernon also keeps the backups that it took itself. A restore writes a point into a new virtual machine, or in place of the disks of a powered-off virtual machine.

A restore can be verified on the platform: the restored virtual machine starts on a quarantine network without uplink, and the platform records whether it booted, received an address and answers on the ports you name. See Restore a virtual machine.

Limits

Item	Value
Granularity of changed areas	64 KiB
Session lifetime	6 hours by default, at most 48 hours
Active sessions	256
Finished sessions kept	1,024
Change ids kept per disk	4
Disks per session	64
Areas per page	4,096 by default, at most 65,536
Timeout of a host call	600 seconds by default (`agent_timeout`)