There has been a question on Mastodon how to install Debian on software RAID1. This turned out a little more complicated than I initially thought.
There are several things to consider:
- The EFI partition. According to the Debian documentation you can’t put this on software RAID. But they offer a simple GRUB hook to keep two EFI partitions in sync. We will use that.
- The
rootandswappartition. I would have liked to have these in LVM logical volumes because that enables me to let them grow as needed. But there is a serious problem with booting LVM RAID1 when one disk is missing: the system will not automatically activate incomplete logical volumes and fall back toinitramfs. The LVM2 people do not support this and you would have to type commands at the console in this case. Also the Debian installer did not hear (yet) about LVM RAID1 and it would be manual work. - Instead we put
rootandswapon DM RAID partitions.
Everything documented here is based on Debian 12.8.
So, fire up your Debian installer. Select an expert installation method (text or graphical). When it comes to partitioning disks, select Manual.
- For all of the following: When there is no partition table on a disk yet you just see the disk itself, no
FREE SPACEnor partitions. Select the disk.
Create new empty partition table: Yes
For partition table select:- When using Legacy BIOS:
msdos(this is MBR) - When using UEFI BIOS:
gpt
- When using Legacy BIOS:
- With FREE SPACE I mean something like a few or more GBs. Ignore FREE SPACE at the beginning or the end of the partition table with a size of 1.0 MB.
- When creating partitions on
msdosformatted disks, you get asked if you wish to create a primary or logical partition. I give the answer to these questions here. When usinggptformatted disks, you do net get these questions.
UEFI only: create EFI partitions
You have to do this part only if you run an UEFI BIOS.
You cannot put the ESP partition on software RAID, see here. We will have to create two partitions and sync them later.
Select FREE SPACE on first disk (sda)
Create a new partition, 1 GB, Beginning
Use as: EFI System Partition
Done setting up the partition
Select FREE SPACE on second disk (sdb)
Create a new partition, 1 GB, Beginning
Use as: FAT32 file system (not! EFI System Partition!)
Mount point: Enter manually, /boot/efi2
Done setting up the partition
Create the software RAID for swap
Select FREE SPACE on first disk (sda)
Create a new partition, size: as you wish for swap (i.e. 1 GB), Primary, Beginning
Use as: physical volume for RAID
Done setting up the partition
Select FREE SPACE on second disk (sdb)
Create a new partition, same size as previous, Primary, Beginning
Use as: physical volume for RAID
Done setting up the partition
Configure software RAID
Write the changes to the storage devices: Yes
Create MD device, RAID1, active devices: 2, spare devices: 0
Depending on BIOS:
- when using Legacy BIOS: select /dev/sda1 and /dev/sdb1
- when using UEFI BIOS: select /dev/sda2 and /dev/sdb2
Finish
Select the space in RAID1 device #0
Use as: swap area
Done setting up the partition
Create the software RAID for root
Select FREE SPACE on first disk (sda)
Create a new partition, full offered size, Primary
Use as: physical volume for RAID
Done setting up the partition
Select FREE SPACE on second disk (sdb)
Create a new partition, full offered size, Primary
Use as: physical volume for RAID
Done setting up the partition
Configure software RAID
Write the changes to the storage devices: Yes
Create MD device, RAID1, active devices: 2, spare devices: 0
Depending on BIOS:
- when using Legacy BIOS: select /dev/sda2 and /dev/sdb2
- when using UEFI BIOS: select /dev/sda3 and /dev/sdb3
Finish
Select the space in RAID1 device #1
Use as: Ext 4, Mount point: /
Done setting up the partition
Review the configuration
Your partition table should now look like this (on an UEFI system):
RAID1 device #0 - 999.3 MB Linux Software RAID Array
> #1 999.3 MB F swap swap
RAID1 device #1 - 8.7 GB Software RAID device
> #1 8.7 GB f ext4 /
SCSI3 (0,0,0) (sda) - 10.7 GB QEMU QEMU HARDDISK
> 1.0 MB FREE SPACE
> #1 999.3 MB B F ESP
> #2 1.0 GB K raid
> #3 8.7 GB K raid
> 1.0 MB FREE SPACE
SCSI4 (0,0,1) (sdb) - 10.7 GB QEMU QEMU HARDDISK
> 1.0 MB FREE SPACE
> #1 999.3 MB F fat32 /boot/efi2
> #2 1.0 GB K raid
> #3 8.7 GB K raid
> 1.0 MB FREE SPACE
Of course you can split the space and add RAID devices for /home, /var and whatever you like here. I’m no fan of that.
Finish partitioning and write changes to disk
Write the changes to disks: Yes
Continue with installation as you would do without RAID.
When you get asked where to install the GRUB bootloader (only on Legacy BIOS): /dev/sda
When you get asked if to Force GRUB installation to the EFI removable media path: I never had the need to say yes here. Try “No”.
Now finish the installation and boot your system.
After reboot
Log in and switch to root.
# cat /proc/swaps
Filename Type Size Used Priority
/dev/md0 partition 975868 0 -2
# df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 391M 512K 391M 1% /run
/dev/md1 7.9G 1.5G 6.0G 20% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda1 952M 5.9M 946M 1% /boot/efi
/dev/sdb1 952M 4.0K 952M 1% /boot/efi2
We have / on /dev/md1, swap on /dev/md0 and /boot/efi on /dev/sda1.
(You don’t have /boot/efi* if you are running Legacy BIOS).
# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] ...
md0 : active (auto-read-only) raid1 sda2[0] sdb2[1]
975872 blocks super 1.2 [2/2] [UU]
md1 : active raid1 sda3[0] sdb3[1]
8521728 blocks super 1.2 [2/2] [UU]
/dev/md0 is a RAID1 using /dev/sda2 and /dev/sdb2/dev/md1 is a RAID1 using /dev/sda3 and /dev/sdb3
Your system might boot a little slow. It may be that the installer did not get the UUID of the swap device right.
# blkid -o value -s UUID /dev/md0
Copy the UUID.
# vi /etc/initramfs-tools/conf.d/resume
RESUME=UUID=516d4a5f-b3b4-4896-bac2-d8b8fdd42298
Overwrite the UUID (marked green) with the one you copied.
# update-initramfs -u
Now reboots should run faster.
UEFI only: poor man’s EFI RAID1
From the documentation we know UEFI does not support software RAID for the EFI partition. Follow the guidelines there:
# vi /etc/fstab
UUID=... /boot/efi vfat errors=remount-ro,nofail,umask=0077 0 0 UUID=... /boot/efi2 vfat errors=remount-ro,nofail,noauto,umask=0077,utf8 0 0
Add/edit the highlighted parts.
Don’t forget to edit the last field in the first line to 0.
# systemctl daemon-reload
# vi /etc/grub.d/90_copy_to_boot_efi2
#!/bin/sh
# https://wiki.debian.org/UEFI#RAID_for_the_EFI_System_Partition
set -e
if mountpoint --quiet --nofollow /boot/efi; then
mount /boot/efi2 || :
rsync --times --recursive --delete /boot/efi/ /boot/efi2/
# Being FAT it is even better when not mounted, won't be damaged in case of computer crash or power outage.
# Ref: https://forums.debian.net/viewtopic.php?p=759692&sid=ff62e9207a064b027f041d2cd0a612a4#p759692
umount /boot/efi2
fi
exit 0
# chmod 755 /etc/grub.d/90_copy_to_boot_efi2
# apt install -y rsync
# update-grub
Now every time you or a system update modifies something within GRUB, it will run update-grub and the contents of the EFI partition will be synced to /boot/efi2.
You still need to make it bootable:
# efibootmgr -v
BootCurrent: 0005
Timeout: 3 seconds
BootOrder: 0005,0002,0003,0001,0000,0004
...
Boot0004* EFI Internal Shell
FvVol(7cb8bdc9-f8eb-4f34-aaea-3ee4af6516a1)/
FvFile(7c04a583-9e3e-4f1c-ad65-e05268d0b4d1)
Boot0005* debian
HD(1,GPT,317bcf96-fca5-47b3-beb2-8a8212018793,0x800,0x1dc800)/
File(\EFI\debian\shimx64.efi)
# efibootmgr -c -d /dev/sdb -p 1 -w -L debian2 \
-l '\EFI\debian\shimx64.efi'
Make the highlighted parts identical.
Check how BootCurrent: 0005 matches the debian entry.
Now reboot the system and tell your BIOS to boot debian2. After that check again:
# efibootmgr -v
BootCurrent: 0006
Timeout: 3 seconds
BootOrder: 0005,0002,0006,0003,0001,0000,0004,0007,0008,0009,000A
Boot0005* debian
HD(1,GPT,317bcf96-fca5-47b3-beb2-8a8212018793,0x800,0x1dc800)/
File(\EFI\debian\shimx64.efi)
Boot0006* debian2
HD(1,GPT,a52bb6ee-bf2e-40b9-8d3f-7959cf8a90d0,0x800,0x1dc800)/
File(\EFI\debian\shimx64.efi)
Now BootCurrent: 0006 matches the debian2 entry.
Test with lost disks
To test the setup: remove a disk. Let’s begin with removing the second disk. Power off the system, remove the second drive, power on. You should see a normal boot, just without the second disk.
Your system will happily boot now. Power off the system, re-connect the second drive, power on.
# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] ...
md1 : active raid1 sda3[0]
8521728 blocks super 1.2 [2/1] [U_]
md0 : active (auto-read-only) raid1 sdb2[1] sda2[0]
975872 blocks super 1.2 [2/2] [UU]
The underscore tells us that something is missing here. Compare with the earlier command. We have to re-add /dev/sdb3:
# mdadm /dev/md1 --add /dev/sdb3
mdadm: added /dev/sdb3
# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] ...
md1 : active raid1 sdb3[2] sda3[0]
8521728 blocks super 1.2 [2/1] [U_]
[====>................] recovery = 21.1% (1801728/8521728) finish=0.4min speed=225216K/sec
...
The RAID is in recovery know and will finish shortly. Then it will be shown as [UU] again.
This is a very important point! MD-RAID is not self healing!
I advise you to monitor your RAID devices closely!
For example CheckMK does it out of the box.
Now power off the system again, remove the first drive, power on. You should see a normal boot, just without the first disk.
Power off the system, re-connect the first drive, power on.
# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] ...
md0 : active (auto-read-only) raid1 sda2[0] sdb2[1]
975872 blocks super 1.2 [2/2] [UU]
md1 : active raid1 sdb3[2]
8521728 blocks super 1.2 [2/1] [_U]
Again the underscore tells us that something is missing here. We have to re-add /dev/sda3:
# mdadm /dev/md1 --add /dev/sda3
mdadm: added /dev/sda3
Wait for the recovery to finish.
That’s it! Your system is running completely on RAID1 and can survive a disk loss.