Posted: 2019-07-09 09:06:41 by Alasdair Keyes
I've been interested in running ZFS for a while but have always held off making the leap due to worries about features and stability. ZFS was originally developed for Solaris and has been ported over to Linux by the ZFS on Linux (ZoL) project https://zfsonlinux.org/.
Recently ZoL 0.8 was released with native encryption which is really a must. Unfortunately the latest Debian release 'Buster' only has 0.7.12 so the native encryption feature isn't available.
I've been experimenting with a Virtualbox VM to develop and test a suitable setup that I would be happy with on my production hardware.
My existing production setup runs Debian Stretch using Linux software Raid with LUKS Encryption on top and running ext4 as a filesystem.
For this test setup I'm using Virtualbox with 4x 2GB disks for ZFS with Striped/Mirrored configuration, it's essentially ZFS's version of RAID 10. For a configuration like this you should ensure you have at least 2GB RAM, I did try with 1GB however the LUKS encrypted devices were failing to startup at boot with out of memory errors. Debian 'Buster' is the OS.
The Disk setup is
/dev/sdbZFS disk 1
/dev/sdcZFS disk 2
/dev/sddZFS disk 3
/dev/sdeZFS disk 4
apt update && apt upgrade -y
contribto the Debian apt repo list in
deb http://deb.debian.org/debian buster main contrib deb-src http://deb.debian.org/debian buster main contrib
apt update && apt install dpkg-dev linux-headers-amd64 cryptsetup -y
This can take some time, make a cup of tea.
apt install zfs-dkms zfsutils-linux -y
cryptsetup -y luksFormat /dev/sdb cryptsetup -y luksFormat /dev/sdc cryptsetup -y luksFormat /dev/sdd cryptsetup -y luksFormat /dev/sde
Get the UUID for each LUKS device
# ls -l /dev/disk/by-uuid/ total 0 lrwxrwxrwx 1 root root 10 Jul 9 08:55 0af47096-987d-41b5-b5a7-98827850f46d -> ../../sda1 lrwxrwxrwx 1 root root 9 Jul 9 08:55 5888dfc8-4df0-410e-8aec-992aad7abd97 -> ../../sdc lrwxrwxrwx 1 root root 9 Jul 9 08:55 abd4a557-de16-4ecd-ab73-e4d41293dcf4 -> ../../sde lrwxrwxrwx 1 root root 9 Jul 9 08:55 e2f1931b-2413-4181-9500-baad1a74c12d -> ../../sdd lrwxrwxrwx 1 root root 9 Jul 9 08:55 edc129d6-dc90-4338-bc2e-9476843ff41f -> ../../sdb lrwxrwxrwx 1 root root 10 Jul 9 08:55 fc1b09a1-41e2-4503-8c4f-d2e532dea5aa -> ../../sda5
/etc/crypttab file with your disk configuration, it should look similar to this, the target name can be any unique name that you want.
# <target name> <source device> <key file> <options> sdb_crypt UUID=edc129d6-dc90-4338-bc2e-9476843ff41f none luks sdc_crypt UUID=5888dfc8-4df0-410e-8aec-992aad7abd97 none luks sdd_crypt UUID=e2f1931b-2413-4181-9500-baad1a74c12d none luks sde_crypt UUID=abd4a557-de16-4ecd-ab73-e4d41293dcf4 none luks
As you can see the UUID mapping in
/dev/disk/by-uuid is mapped against a unique name for device mapper.
This isn't required, however it's good to ensure that your LUKS setup is correct before proceeding. You will be asked for your LUKS passwords on boot. Once you log back in again, you should be able to run the following
ls and see the LUKS devices are initialized correctly
$ ls -l /dev/mapper/ total 0 crw------- 1 root root 10, 236 Jul 9 08:55 control lrwxrwxrwx 1 root root 7 Jul 9 08:55 sdb_crypt -> ../dm-0 lrwxrwxrwx 1 root root 7 Jul 9 08:55 sdc_crypt -> ../dm-1 lrwxrwxrwx 1 root root 7 Jul 9 08:56 sdd_crypt -> ../dm-3 lrwxrwxrwx 1 root root 7 Jul 9 08:55 sde_crypt -> ../dm-2
You will sometimes get a warning that the zfs kernel module isn't loaded, just follow the instructions and run...
This will only need to be run once, once a pool is configured the module will be loaded automatically.
# zpool create pool01 mirror /dev/mapper/sdb_crypt /dev/mapper/sdc_crypt mirror /dev/mapper/sdd_crypt /dev/mapper/sde_crypt
Check the setup
# zpool status pool: pool01 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM pool01 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 sdb_crypt ONLINE 0 0 0 sdc_crypt ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 sdd_crypt ONLINE 0 0 0 sde_crypt ONLINE 0 0 0 errors: No known data errors
Rebooting again will ensure that everything is configured and the LUKS devices are brought up before ZFS mounts the pool, otherwise you will end up with ZFS errors and the pool won't load.
zpool status again and you should see the same output as above. If the LUKS devices fail to initialize and none of the devices are available, you will see an error about
no pool available.
If only some of the LUKS devices fail to initialize you will see the state being something other than
ONLINE and you can check
/var/log/kern.log for information as to why.
If you found this useful, please feel free to donate via bitcoin to 1NT2ErDzLDBPB8CDLk6j1qUdT6FmxkMmNz
I'm now available for IT consultancy and software development services - Cloudee LTD.
Happy user of Digital Ocean (Affiliate link)