8. Backup and recovery#

RAID keeps your data through a failed disk, and snapshots bring back a deleted or encrypted file. Neither survives the server itself: a fire, a theft, a power surge, or a mistake that reaches every disk. For that you need a backup: a copy somewhere else, with versions to go back to.

A good plan keeps three copies of what matters, on two kinds of storage, one of them off site. OSM gives you:

Backup tasks#

Backup > Create makes a task. It backs up with restic: only what changed since the last run is sent, every run is a version you can restore from, and everything is encrypted before it leaves the server.

1. Name the task.
2. Choose where it goes (below).
3. Choose what:
   • the shared folders to back up, and the home folders (everyone's);
   • The server's settings and accounts: the accounts with their passwords (not their two-step sign-in), the groups, the shared folders and their permissions, the file services' settings, the other settings a configuration backup holds, and the installed apps with their settings. A version that has them can rebuild a server on a new installation (below);
   • The apps' data: the apps' settings, databases and files (below).
4. Choose when: every day or every week at an hour you choose (02:00 by default), or by hand. A weekly task runs on Sundays; What, when and how long... in its ··· menu changes the day. Tasks run one at a time: a task that cannot start at its hour (another backup still runs, or the server was off) starts as soon as it can, the same day.
5. Choose what stays: one version a day for 7 days, one a week for 4 weeks, one a month for 12 months and one a year for 2 years, by default. Older versions are deleted, and the latest one always stays.
6. Give it a password of at least 8 characters.

Keep the password somewhere else. Without it nobody can read the backup, OSM included. A backup whose password is lost cannot be restored, and nobody can recover the password.

Back up now runs a task at once. The task list shows each task's last run, and whether its last test restore passed (Proved). A run that fails, or that misses folders that are no longer there, is a notification.

Where a backup can go#

An off-site destination (an S3 store, or a server at another place) protects you from what happens to the building.

The apps' data#

The apps keep their own settings, databases and files in an area of the volume that is not a shared folder. A backup task with The apps' data (on by default) saves it in a version of its own, labelled The apps' data in the list of versions, next to the version of the shared folders made by the same run.

A database copied while it is being written can be unusable, so OSM reads the apps' data as it stood at one moment:

• On Btrfs and ZFS volumes, the apps are paused for about a second while OSM takes a snapshot of their data, then run on while the backup reads the snapshot. The snapshot is deleted when the backup ends.
• On ext4 volumes, which have no snapshots, the apps are paused while their data is read. This usually takes a minute or less at night, and never more than two hours. If reading takes longer, it is stopped, the apps run again, and you are told that the apps' data was not backed up (the shared folders still are). For apps with much data, a Btrfs or ZFS volume avoids the wait.
• If the backup goes to the server itself (an app of it, such as an S3 or SFTP app), or its destination does not answer while the apps are paused, the apps do not wait, since the backup would wait for them: on an ext4 volume, their data is then read while they run.

Paused apps always run again, even if the server restarts during a backup. Caches that an app makes again by itself (Jellyfin's cache, Plex's transcodes, Immich's models) are left out.

The virtual machines' disks are not part of the apps' data: see Virtual machines.

Backups that prove themselves#

A backup is only as good as its restore. OSM checks every backup task on its own:

• Every month, a test restore, two hours after the task's time. Twenty files of the latest version, picked at random from as many folders as possible, are restored into a scratch folder and compared with the files on the server (those not changed since the backup). restic also reads a twelfth of the backup's data each month, so all of it is read once a year.
• A report on the 1st of each month: every task, its last run and its last test restore, as a notification and by email if email is set up. Report... shows it now.
• Test a restore now and Check the backup (reads 5 % of it) are in each task's ··· menu.

A file that does not come back, or comes back different, is an error notification.

Restoring#

Files and folders#

1. Backup > Versions on the task, then Browse... on a version.
2. Tick the files and folders to restore. Open a folder by its name; the path at the top goes back.
3. Choose where:
   • Back where it was: what is there now is replaced;
   • Into a new folder: a folder named after the task and the date, under the folder you give (for example /volume1/Restore). Nothing on the server is replaced.

Restoring an app's data back where it was stops the app first, and starts it again afterwards. The data comes back exactly as it was in that version: files the app wrote since are removed, so a database is never mixed with a newer journal. If the restore fails, the data that was there is put back.

A restore needs the volume or USB drive it writes to: if it is not mounted (or the drive is unplugged), the restore stops before it writes anything, rather than fill the system disk.

The whole server, onto a new installation#

When a server is lost, a version made with The server's settings and accounts brings its accounts, shared folders, files and apps back on a new installation:

1. Install OSM on the new server, and make its volumes with the same names as before (Volume 1, Volume 2...).
2. Backup > Open a backup of another server...: give the destination of the old backup and its password. For a backup on another server over SFTP, the new server signs in with a key of its own: the dialog shows it, and you add it to that account's ~/.ssh/authorized_keys first.
3. Versions on the task it makes, then Restore everything... on the version you want, and choose:
   • The shared folders' files too;
   • The apps, with their data.

OSM restores, in this order: the groups and accounts (with their passwords), the settings, the shared folders and their permissions, then the files, then the apps with their data. Accounts that exist on the new server already stay as they are; a shared folder that exists already takes the backup's description and permissions. When the old server was a domain controller, the backup's copy of the domain goes among the Domain app's copies: Domain makes the domain again from it (Domain).

It does not bring back the network, email and notifications, the backup, cloud and replication tasks, the scheduled tasks, the VPN's devices, the certificates, the firewall's lists, the password rules, or two-step sign-in and passkeys: set them up again.

Cloud tasks#

Backup > Cloud keeps a shared folder and a folder in a cloud alike: WebDAV (Nextcloud, ownCloud...), S3 (AWS, Wasabi, Cloudflare R2, MinIO), Backblaze B2, SFTP, Google Drive, Dropbox, or another folder such as a USB drive.

• Which way: Upload (new and changed files go up), Mirror up (the cloud becomes like the folder, deletes included), Download, or Mirror down.
• When: every hour, every day, or by hand. Test the connection checks the settings first.
• Google Drive and Dropbox need a token: run rclone authorize "drive" (or "dropbox") on a computer with a browser, sign in, and paste what it prints.

A cloud task is not a backup: a mirror copies deletions and damaged files too. For versions, use a backup task.

Replication#

Backup > Replication copies the snapshots of a shared folder on a Btrfs volume to another OSM server: the first time in full, then only what changed. After a disaster the other server can turn the newest replica into a shared folder of its own.

1. On the server that receives: install SSH in Settings > SSH. In Backup > Replication > Receive, turn on Take replicas from other OSM servers, choose the Btrfs volume that keeps them, and Add a sender with a name and the sender's key (shown on the sender, in the same page).
2. On the server that sends: Replicate a shared folder: the shared folder, the other server's address and SSH port, when (every hour, every day, or by hand), and how many replicas it keeps (30 by default).

On the receiving server, Replicas here lists them, and Make it a shared folder... makes a copy of the newest replica that you can open and change; the replicas stay as they are.

The system itself#

• Restore points: OSM makes one before every update, automatic or not, and goes back to it by itself if a service fails in the 15 minutes afterwards. Settings > Updates and Recovery > Restore points lists them, and Make one now makes one by hand. See System.
• Configuration backup: in Settings > Updates and Recovery > Configuration backup, Export saves, and Import... brings back on a new installation, Desktop Access, automatic updates, the server's screen, the groups, the shared folders with their permissions, and whether the firewall is on. It does not bring back the accounts and passwords, the network, the backup tasks or the apps. An import shows what it could not make.

Reset to the defaults#

Settings > Updates and Recovery > Reset to the defaults puts OSM's settings back to how a new installation has them, and keeps your data. The page lists exactly what is reset and what stays. In short:

• Reset: the network (DHCP), the web desktop's ports and sign-in settings, the firewall's lists, the ports on the router, the proxy, the password rules, automatic updates, the email settings, and everyone's sign-ins.
• Kept: the volumes, shared folders and every file, snapshots, the apps and virtual machines with their data, the backup, cloud and replication tasks, restore points, certificates, and the accounts, unless you choose to reset them too.

A restore point is made first. The reset menu on the server's own screen (key R) does the same, for when the web desktop cannot be reached.

A plan that works#

1. A backup task with everything (shared folders, home folders, settings and accounts, apps' data) to a USB drive or another volume, every night.
2. A second task to an off-site destination (an S3 store, or a server at another place), every night or week.
3. Immutable snapshots of the shared folders that matter most (Storage), so that ransomware cannot destroy every copy, and two-step sign-in for every Administrator.
4. The backup passwords written down away from the server.
5. Read the monthly report, and once a year try a full restore onto a spare computer or a virtual machine.