ZFS on FreeBSD 7.1

I've been flirting with ZFS on FreeBSD 7.1-RELEASE AMD64, and I must say I like it so far.

I'm hoping and dreaming of using it as my main filestore in a 'raidz' array of 1.5 TB drives to give me a massive temporary file store for my ever increasing Video and Music files.

Why ZFS?

I'm excited about ZFS mostly for being transactional, and for the raidz support - This is an enhanced version of RAID-5. I could have ran a GENOM RAID-5 array, but when you add in the other features of ZFS (and it has many - go check) it's very tempting to go with ZFS instead.

Unfortunately it's still experimental in FreeBSD, which may have some sticking with more tested technologies.

I should give a few tips if you're trying out ZFS

1) Don't bother with i386

You can run ZFS under i386 with some tweaks, but why? It's going to be limited, and I think you're asking for trouble. ZFS needs a lot of memory address space. Give it 64 Bits and you just have to put

zfs_enable="YES"

in /etc/rc.conf to make it start up.

2) Give it some power

I love using old hardware to run FreeBSD servers, but ZFS needs some oomph to run proprly. Sure people are running it in i386 with 384 Meg, but why? You can grab a decent Socket 775 board and 64 bit chip for $120, throw in 2 gigs of RAM for $60, and you're still under $200

3) Give it respect

This isn't UFS. It's still experimental, don't put your only copy of critical data on it. I'm going to put things that I really don't want to lose - MP3's and Videos, but not accounting data or similar. Will the world end if I lost it? No, but I'd be pretty pissed, so you know I'm doing a lot of testing on my hardware so I know how to recover from faults. Learn how to use it if you're going to want any dependability from it.

4) Don't try and boot from it

It won't. There are tricks to make the /usr and other partitions ZFS and the main boot on UFS, but why frig around and play with fire? We all have an old 80 or 40 gig HD around. Make that your UFS boot, and setup your other drives as dedicated ZFS.

5) Speed

In a very slap-dash test, my GEOM stripe was delivering data via samba at the same speed as my ZFS raidz pool. In some cases, even faster. Of course I'd have to do proper tests to make the playing field level, so this is just a quickie test to see if a ZFS raidz could hold up in speed to my GEOM stripe on my box for samba network use, and it does that just fine. It may suck for a MySQL db, but I haven't tested that yet.

6) Moving to different hardware

- If you build your pool on /dev/ad0, /dev/ad2, ad4 (etc) and then move to a new controller that is discovered as da0, da1, etc. You simply do this;

zpool tank export
zpool tank import

presto. My ad* based zpool is now showing up on my da* based controller.

7) Failure Redundancy

I setup my raidz, transferred a few gig of data to it, then pulled a drive while it was running. A few complaints on the console, but my data was still there. Speed seemed okay, and data was intact without a lot of rebuilding necessary. I later on rebooted and reconnect the drive, added it back into the pool, and life went on.

GEOM is annoying after a crash as it has to check the entire array before it lets you back in. On a 1.5 TB array, that takes some time. With ZFS, you can still fire up and access data.

You can also run a background check by typing

zfs scrub

No more off-line fscks! Schedule a low-priority scrub to run in the background, and you're ensuring your data is always clean and matching.

This simple crash test proved ZFS can do two main things for me ; 1) Notify me when something fails, and 2) accept the drive back after it's been fixed.

I now need to see if I pull a drive, format it, and put it back if it accepts it properly and rebuilds the array on it. One would hope, but I'd like to manually test it before putting all that media on it.


Conclusions:

Get FreeBSD 7.1-RELEASE AMD64 and grab 2 or more drives and play with it. Check the links below for setup info - it's really quite simple. I'm loving it, and I guess I will until it screws me over. :-)


Links for more info:

http://blog.thefrog.net/2008/04/zfs-on-freebsd.html

(Top link references the two bottoms ones which I used to setup my ZFS)

http://wiki.freebsd.org/ZFSQuickStartGuide
http://wiki.freebsd.org/ZFSTuningGuide

I also recommend

http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
http://kerneltrap.org/FreeBSD/BSDCan_2008_ZFS_Internals
http://opensolaris.org/os/project/sdosug/past_meetings/ZFS-general.20070919s.pdf
http://www.tech-recipes.com/rx/1405/zfs-how-to-fsck-or-check-filesystem-integrity-with-scrub/