2. Storage
Everything about the disks is in the Storage app: Overview, Storage (pools and volumes), Disks, Snapshots, USB Drives, Analyzer, iSCSI and Settings.
How OSM stores your data
- Disks are the hard disks and SSDs in the server.
- A volume is where your data lives: shared folders, apps' data, virtual machines. Volumes are numbered (Volume 1, Volume 2...) and each has a file system.
- A RAID protects a volume from a disk failure by keeping the data on more than one disk. OSM builds even a one-disk volume as a RAID array: a Basic volume is a one-disk mirror that becomes a real mirror when you add a second disk. A ZFS volume uses ZFS's own RAID.
- A storage pool holds one or more volumes on a group of disks. A pool can use Flexible RAID, which puts disks of different sizes to work.
- The system (OSM itself) has a small partition of its own on each system disk. With two system disks or more, these mirror each other, so a failed system disk does not stop the server.
Choosing a RAID level
| Level | Disks | Usable room | Disks that may fail | Good for |
|---|---|---|---|---|
| Basic | 1 | one disk | none | a start: add a mirror later |
| JBOD | 1 or more | all of them | none | room without protection |
| RAID 0 | 2 or more | all of them | none (one failure loses everything) | speed for data you can lose |
| RAID 1 | 2 or more | one disk | all but one | two disks; the simplest protection |
| RAID 5 | 3 or more | all but one | one | most home servers with three to six disks |
| RAID 6 | 4 or more | all but two | two | many or large disks, where a rebuild takes long |
| RAID 10 | 4 or more (even) | half | one per mirrored pair | speed and protection, e.g. virtual machines |
| Flexible RAID | 1 or more, any sizes | almost all, by size layers | none with one disk, then one; two with four disks or more | disks of different sizes, a pool that grows a disk at a time |
RAID protects against a failed disk, not against deleting a file, ransomware or a fire: you still need snapshots and backups.
Choosing a file system
- ext4 (OSM's default): simple and fast, with quotas for each account as well as each shared folder. It has no snapshots.
- Btrfs: snapshots (earlier versions in Files and in Windows, immutable snapshots), replication to a second server, checksums of your data, and backups of the apps' data while they keep running. Quotas are for shared folders, not for each account. Choose it if you want earlier versions of your files.
- ZFS: install ZFS support first, in Storage > Settings or when you choose ZFS in Create a volume. Your server then builds ZFS for its own kernel (this needs the internet and a few minutes). ZFS is for volumes on disks of their own; storage pools take ext4 or Btrfs. ZFS has its own RAID levels, snapshots and checksums.
Creating storage
- Open Storage > Storage.
- Create a volume... makes a volume on disks of its own (one RAID array), with a RAID level and a file system. When a storage pool has room, it first asks where the volume goes. Create a storage pool... makes a pool first (one RAID level, or Flexible RAID), then Create a volume in the pool now or later; a pool's volumes have a size, and grow while in use (Grow...).
- Choose the disks. A disk that holds something (another system's data, or what a deleted volume left behind) says so, and is erased.
- Self-healing (for a volume on disks of its own, with RAID 1, 5, 6 or 10 and ext4 or Btrfs): each block of the volume gets a checksum. A block that a disk returns damaged is noticed when it is read, and repaired from the other disks of the RAID. Writes are slower (about half the speed), about 1 % of each disk holds the checksums, and each disk is prepared once, which takes hours on a large disk. The server opens self-healing volumes by itself at every start, and the volume's card shows how many damaged blocks were mended.
New ext4 volumes have quotas turned on. An older ext4 volume made without them has Turn on quotas... on its card. The volume is then unmounted for a moment: Containers and File Protocols stop meanwhile, and virtual machines must be shut down first.
While the disks of a new volume are written for the first time, its state shows the progress in orange (a new storage pool shows Repairing). You can use it right away. A new RAID 5 or 6 fills its last disk this way: it is building, not degraded.
Delete... on a volume erases it for good, after you type its name: its disks become empty, or its room goes back to its storage pool. OSM refuses while apps keep their data on it, or while it holds immutable snapshots.
Adding disks
On a Basic volume, Add a mirror disk... makes it a mirror (RAID 1). Add disks... on a volume or a storage pool:
- a Basic storage pool becomes a mirror (RAID 1);
- a RAID 1 gets another mirror, or, with two disks, becomes RAID 5;
- a RAID 5 grows, or becomes RAID 6 (four disks or more);
- a RAID 6 or a JBOD grows;
- a storage pool takes new disks: a Flexible RAID pool puts them to work whatever their size.
Replace a disk... on a storage pool puts a new disk (as big or bigger) in another's place, while the pool keeps working: a bigger disk gives a Flexible RAID pool more room.
The volume grows by itself when the RAID has finished spreading the data again (it shows Growing meanwhile), which can take hours on large disks. The server stays in use meanwhile.
Add a hot spare... (on RAID 1, 5, 6 and 10 volumes of their own) gives a volume a disk that waits: when a disk of the volume fails, the hot spare takes its place at once and the volume repairs itself onto it.
When a disk fails
The bell (and your email, if set up) tells you which disk failed and where it is connected. A volume whose RAID allows it keeps working, Degraded: your files are all there.
- Open Storage > Disks to see which disk failed and its port.
- Replace the disk. On a port with hot-plug (the Port column says so), the new disk is noticed as soon as it is plugged in; otherwise shut the server down to change it. Rescan for new disks looks for a disk the server has not noticed.
- In Storage > Storage, choose Repair... on the volume (or on its storage pool), and the new disk. The rebuild runs in the background.
If the failed disk was a system disk, a server with two system disks or more keeps starting from the others. Repair... in the System part gives the new disk what the failed one held: OSM itself, its start-up files and its part of the volumes.
Prefer CMR hard disks to SMR ones for RAID: rebuilding onto an SMR disk can take days.
The system disks
The System part of Storage > Storage shows OSM's own disks.
- Mirror the system... puts a second disk next to a single system disk: the server then starts from either one. With one system disk, a failure of that disk stops the server until OSM is installed again.
- Repair... appears when a system disk failed: it gives the disk you put in its place what the failed one held.
- Use the free space on the system disks... appears when the system disks are bigger than the system, and gives the system that room while OSM runs. In a virtual machine whose disks were made bigger, OSM takes the room by itself at the next start.
- An encrypted system shows its state here. You can make a new recovery key, and bind the TPM again after a change of the firmware or of its Secure Boot settings.
Disk health
- Storage > Disks lists every disk with its port and SMART health. Details shows its temperature and SMART data; Test runs a quick test (about 2 minutes) or an extended test (hours).
- Storage > Settings:
- Data scrubbing, on the first Sunday of every month or of every three months, reads every RAID array (not RAID 0 or JBOD) and every Btrfs volume, and compares the copies. A disk that starts to fail is found while the others still have good copies, and a self-healing volume mends the damaged blocks it finds. The results are on this page (Last time, Result) and in the bell. A ZFS volume has Check the disks now on its card.
- SMART self-tests on a schedule.
- Spin-down of idle hard disks.
Snapshots
A snapshot is a read-only picture of a shared folder at one moment. It takes almost no room at first, and keeps what changes after it. Snapshots work for the shared folders OSM made on Btrfs and ZFS volumes, but not for encrypted ones.
- Storage > Snapshots takes a snapshot by hand (Take one), or on a schedule (Schedule...: every hour or every day), kept by rules (the last so many hourly, daily, weekly and monthly ones). List shows a folder's snapshots: Browse opens one in Files, and Lock keeps one whatever the rules say.
- Everyone with access to the shared folder sees the snapshots as Earlier versions in Files, and Windows shows them as Previous Versions over SMB.
- Immutable snapshots: in a shared folder's Schedule..., set Immutable for (days, 0: no), up to 3650 (10 years). Every snapshot taken after that, by hand or on schedule, is kept that long. Until then OSM refuses to delete it, for every account, Administrators included, so ransomware that reaches your shared folders cannot remove it. Its volume cannot be deleted before then either. The shared folder itself can still be deleted, and its snapshots stay until their time (on ZFS, the folder's files can be deleted only after that). The protection is in the file system itself (an immutable flag on Btrfs, a hold on ZFS): only root, at the server's command prompt, could lift it, on purpose. Keep two-step sign-in on for every Administrator, since an Administrator can become root.
SSD cache
Add an SSD cache... on a volume of a storage pool puts SSDs in front of it: a Read-only cache (one SSD, or two mirrored), or a Read-write cache (two SSDs, mirrored, because a read-write cache holds data not yet written to the pool's disks). SSD cache... then shows how often the cache serves reads and writes, and Remove the cache... takes it off while the volume stays in use. A volume with a cache does not grow: remove the cache first, and add it again afterwards. Volumes of their own have no SSD cache.
USB drives
USB drives are mounted when they are plugged in, unless you turn off Mount USB drives when they are plugged in. Everyone who may sign in finds them in Files, under USB Drives. In Storage > USB Drives you can:
- Eject a drive before you unplug it;
- Format... it (exFAT for any computer, ext4 for Linux, FAT32 for old devices);
- copy a folder of it into a shared folder whenever it is plugged in (Auto-copy...: new files only, nothing deleted on either side), or at once (Copy now).
A USB drive can also be a backup target. Storage never puts one in a volume, a storage pool or the system disks, not even to repair them: a drive that can be unplugged is no place for a RAID.
The Analyzer
Storage > Analyzer shows what fills a volume or a shared folder: the biggest folders and files, the kinds of files, the oldest ones, and duplicates.
iSCSI
iSCSI Storage gives disks over the network to Windows, VMware, Proxmox and Linux: Install it on the iSCSI page the first time. Create a LUN makes a disk (a file on a volume, thin: it takes room only as it is written). Create a target makes LUNs reachable: for the computers you name (by their initiator names), for those that know its CHAP name and password, or both. Only one computer at a time should write to a LUN, unless it holds a cluster file system. LUNs called nb- and a name are the disks of Network Boot's computers, each in a target of its own: they are changed there (Network Boot).