This has been an interesting one to look at recently – and how wrong was I when I thought that installing ZFS would be as simple as installing the apt package. Turns out I was slightly ahead of the curve…?
Background to this – I have been running FreeNAS as my home NAS for a number of years now. Protecting the data on that system used to be down to Crashplan. When they announced they were closing the CrashPlan Home plan, I was grandfathered into the CrashPlan Small Business plan for around a year at a generous discount until that ended in April. I had already started looking at solutions a while back, but still firming up on my options. This blog is going to focus getting ZFS running on a Raspberry Pi, to determine if ZFS snapshot replication to a remote system would be the answer. I have also been trying other options. Ultimately, between two solutions it will involve placing a device in another families home and shipping my backups to that device for off-site protection.
Idea here is that I want to use an SBC (Raspberry Pi or fruity variant) to replicate ZFS snapshots to for offsite backups. These devices are not very powerful, however I would not be accessing the data on these regularly – rather using it as an archival point to re-ingest ZFS datasets should my main FreeNAS run up against a wall.
There is plenty of other discussions and opinions on backup strategies. I am fully aware and of them, but will not be divulging in those with this post – this is more the technical capability / configuration to make this work.
First of all trying this on a brand new Raspberry Pi 4. This awesome little device has just been revamped and has much better performance vs it’s predecessors and also a choice of RAM sizes. The Pi 4 I have at the moment is packing 4GB of RAM. Typically a good choice for ZFS is to be above 2GB for expected home performance, but I can touch on how it’s utilizing that later. Find out more about these on Raspberrypi.org
Of course, being a brand new Pi to the market, Raspbian Buster is currently the only distribution offering full hardware support. I might reconsider this as other distributions start to catch up and offer support, however my choices are limited right now – hence the comment around perhaps being ahead of the curve.
So, the Raspberry Pi 4, given it has a bus connected 1Gbps network adapter, and USB 3.0 connectivity looks like it will be a nice fit to send ZFS snapshots to for off-site protection. Attach a USB 3.0 HDD (I’m using a Seagate 5TB portable drive) and configure for ZFS.
I’m not going to go into installing Raspbian on the Pi, there’s plenty of resources around that elsewhere. Let’s assume you have Buster online, updated and ready to install ZFS.
First of all, you are going to want to add the Debian contrib repository, as the Raspbian repo’s currently do not have all of the ZFS apt packages available. **Update**: I’ve removed the need to grab the keys and add as it’s a debian source, and fixed echoing the line out to sources.list. This worked for me as of May 2020.
echo deb http://deb.debian.org/debian buster contrib >>/etc/apt/sources.list
If you do need to add keys, try the below (big thanks to Colin Ng for helping here!):
sudo gpg –keyserver keyserver.ubuntu.com –recv-key 04EE7237B7D453EC sudo gpg -a –export 04EE7237B7D453EC | sudo apt-key add
Then we are ready to install ZFS. This will take a while, as it compiles the ZFS support for your installed Linux kernel.
sudo apt install -y zfs-dkms zfsutils-linux
Word of note here. The installer does mention and warn about you installing ZFS on a 32-bit system and it not being recommended. Being SoC is 64-bit capable, I did look for a 64-bit OS but did not turn up much. I proceeded knowing that this could cause issues, but will re-evaluate once a 64-bit distribution comes available for the Pi 4.
If the install happens to fail – it could be you are missing the kernel headers. Try installing them first then run the previous command again. (Again thanks to Colin Ng for the clarification here!)
sudo apt-get install raspberrypi-kernel-headers
Once complete, either restart your Pi, or you can load the ZFS module:
sudo modprobe zfs
Now we should ready to create our ZFS filesystem. If you got any errors in the previous steps, try rebooting and running the commands again.
You will need to identify the disk(s) you want to create your ZFS pool on. Check for disks using:
Once you know which disk to you, you can create your zpool:
sudo zpool create -f <Pool Name> <Disk Path>
For my setup I did:
sudo zpool create -f files /dev/sda
That’s it! You can view the pool by doing:
Now you can look at configuring your ZFS snapshot replication and testing. More to come on that front.
General around memory usage – I have found the Pi sits around 750Mb of RAM used, and a load average of 3.30 during ZFS receive operations. Seems pretty reasonable – I am doing lz4 compression on transmission via SSH, and the datasets also have lz4 compression enabled.
Throughput wise, seems that I am getting just over 60Mb/s through the network adapter. I am guessing the bottleneck will be the HDD on the Pi. Pretty reasonable given the cost and power consumption of the Pi!
If you do see slow speeds, make sure your power supply / USB cable can provide enough juice to the Pi. I started off with a USB cable plugged into a USB port on a laptop and it was not happy. I kept getting throttled warnings in the /var/log/messages as it was under-voltage. Once I swapped the Pi onto the power adapter for my Nintendo Switch those warnings went away and the performance shot up!
Will post again with more specifics around getting FreeNAS to replicate to the ZFS pool on the Pi soon. Stay tuned for that!