{"id":204,"date":"2019-07-04T07:30:24","date_gmt":"2019-07-04T07:30:24","guid":{"rendered":"http:\/\/www.oshelp.co.uk\/?p=204"},"modified":"2020-05-14T15:22:35","modified_gmt":"2020-05-14T15:22:35","slug":"installing-zfs-on-debian-buster","status":"publish","type":"post","link":"https:\/\/www.oshelp.co.uk\/?p=204","title":{"rendered":"Installing ZFS on Debian Buster"},"content":{"rendered":"<p>This has been an interesting one to look at recently &#8211; and how wrong was I when I thought that installing ZFS would be as simple as installing the apt package. Turns out I was slightly ahead of the curve&#8230;?<\/p>\n<p>Background to this &#8211; I have been running FreeNAS as my home NAS for a number of years now. Protecting the data on that system used to be down to Crashplan. When they announced they were closing the CrashPlan Home plan, I was grandfathered into the CrashPlan Small Business plan for around a year at a generous discount until that ended in April. I had already started looking at solutions a while back, but still firming up on my options. This blog is going to focus getting ZFS running on a Raspberry Pi, to determine if ZFS snapshot replication to a remote system would be the answer. I have also been trying other options. Ultimately, between two solutions it will involve placing a device in another families home and shipping my backups to that device for off-site protection.<\/p>\n<p>Idea here is that I want to use an SBC (Raspberry Pi or fruity variant) to replicate ZFS snapshots to for offsite backups. These devices are not very powerful, however I would not be accessing the data on these regularly &#8211; rather using it as an archival point to re-ingest ZFS datasets should my main FreeNAS run up against a wall.<\/p>\n<p>There is plenty of other discussions and opinions on backup strategies. I am fully aware and of them, but will not be divulging in those with this post &#8211; this is more the technical capability \/ configuration to make this work.<\/p>\n<p>First of all trying this on a brand new Raspberry Pi 4. This awesome little device has just been revamped and has much better performance vs it&#8217;s predecessors and also a choice of RAM sizes. The Pi 4 I have at the moment is packing 4GB of RAM. Typically a good choice for ZFS is to be above 2GB for expected home performance, but I can touch on how it&#8217;s utilizing that later. Find out more about these on <a href=\"https:\/\/www.raspberrypi.org\/\">Raspberrypi.org<\/a><\/p>\n<div style=\"width: 856px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" src=\"https:\/\/www.raspberrypi.org\/homepage-9df4b\/static\/hero-shot-3ad1d131ea382fa6f006b18aefc820aa.png\" alt=\"\" width=\"846\" height=\"560\" \/><p class=\"wp-caption-text\">Raspberry Pi 4 Model B<\/p><\/div>\n<p>Of course, being a brand new Pi to the market, Raspbian Buster is currently the only distribution offering full hardware support. I might reconsider this as other distributions start to catch up and offer support, however my choices are limited right now &#8211; hence the comment around perhaps being ahead of the curve.<\/p>\n<p>So, the Raspberry Pi 4, given it has a bus connected 1Gbps network adapter, and USB 3.0 connectivity looks like it will be a nice fit to send ZFS snapshots to for off-site protection. Attach a USB 3.0 HDD (I&#8217;m using a Seagate 5TB portable drive) and configure for ZFS.<\/p>\n<div style=\"width: 3010px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" class=\"size-large\" src=\"https:\/\/www.seagate.com\/files\/www-content\/about\/newsroom\/media-kits\/_shared\/images\/backup-plus-portable-5tb-silver-main-packaging-hi-res.jpg\" width=\"3000\" height=\"3000\" \/><p class=\"wp-caption-text\">Seagate 5TB Drive<\/p><\/div>\n<p>I&#8217;m not going to go into installing Raspbian on the Pi, there&#8217;s plenty of resources around that elsewhere. Let&#8217;s assume you have Buster online, updated and ready to install ZFS.<\/p>\n<p>First of all, you are going to want to add the Debian contrib repository, as the Raspbian repo&#8217;s currently do not have all of the ZFS apt packages available. **Update**: I&#8217;ve removed the need to grab the keys and add as it&#8217;s a debian source, and fixed echoing the line out to sources.list. This worked for me as of May 2020.<\/p>\n<pre class=\"\">echo deb http:\/\/deb.debian.org\/debian buster contrib &gt;&gt;\/etc\/apt\/sources.list\r\n<\/pre>\n<p>If you do need to add keys, try the below (big thanks to Colin Ng for helping here!):<\/p>\n<pre class=\"lang:default decode:true\">sudo gpg \u2013keyserver keyserver.ubuntu.com \u2013recv-key 04EE7237B7D453EC\r\nsudo gpg -a \u2013export 04EE7237B7D453EC | sudo apt-key add<\/pre>\n<p>Then we are ready to install ZFS. This will take a while, as it compiles the ZFS support for your installed Linux kernel.<\/p>\n<pre class=\"\">sudo apt install -y zfs-dkms zfsutils-linux<\/pre>\n<p>Word of note here. The installer does mention and warn about you installing ZFS on a 32-bit system and it not being recommended. Being SoC is 64-bit capable, I did look for a 64-bit OS but did not turn up much. I proceeded knowing that this could cause issues, but will re-evaluate once a 64-bit distribution comes available for the Pi 4.<\/p>\n<p>If the install happens to fail &#8211; it could be you are missing the kernel headers. Try installing them first then run the previous command again. (Again thanks to Colin Ng for the clarification here!)<\/p>\n<pre class=\"lang:default decode:true \">sudo apt-get install raspberrypi-kernel-headers<\/pre>\n<p>Once complete, either restart your Pi, or you can load the ZFS module:<\/p>\n<pre class=\"\">sudo modprobe zfs<\/pre>\n<p>Now we should ready to create our ZFS filesystem. If you got any errors in the previous steps, try rebooting and running the commands again.<\/p>\n<p>You will need to identify the disk(s) you want to create your ZFS pool on. Check for disks using:<\/p>\n<pre class=\"\">lsblk -f<\/pre>\n<p>Once you know which disk to you, you can create your zpool:<\/p>\n<pre class=\"\">sudo zpool create -f &lt;Pool Name&gt; &lt;Disk Path&gt;<\/pre>\n<p>For my setup I did:<\/p>\n<pre class=\"\">sudo zpool create -f files \/dev\/sda<\/pre>\n<p>That&#8217;s it! You can view the pool by doing:<\/p>\n<pre class=\"\">zpool list<\/pre>\n<p>Now you can look at configuring your ZFS snapshot replication and testing. More to come on that front.<\/p>\n<p>General around memory usage &#8211; I have found the Pi sits around 750Mb of RAM used, and a load average of 3.30 during ZFS receive operations. Seems pretty reasonable &#8211; I am doing lz4 compression on transmission via SSH, and the datasets also have lz4 compression enabled.<\/p>\n<p>Throughput wise, seems that I am getting just over 60Mb\/s through the network adapter. I am guessing the bottleneck will be the HDD on the Pi. Pretty reasonable given the cost and power consumption of the Pi!<\/p>\n<p>If you do see slow speeds, make sure your power supply \/ USB cable can provide enough juice to the Pi. I started off with a USB cable plugged into a USB port on a laptop and it was not happy. I kept getting throttled warnings in the \/var\/log\/messages as it was under-voltage. Once I swapped the Pi onto the power adapter for my Nintendo Switch those warnings went away and the performance shot up!<\/p>\n<p>Will post again with more specifics around getting FreeNAS to replicate to the ZFS pool on the Pi soon. Stay tuned for that!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This has been an interesting one to look at recently &#8211; and how wrong was I when I thought that installing ZFS would be as simple as installing the apt package. Turns out I was slightly ahead of the curve&#8230;? &hellip; <a href=\"https:\/\/www.oshelp.co.uk\/?p=204\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/204"}],"collection":[{"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=204"}],"version-history":[{"count":8,"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/204\/revisions"}],"predecessor-version":[{"id":220,"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/204\/revisions\/220"}],"wp:attachment":[{"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=204"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=204"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.oshelp.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=204"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}