ZFS on FreeBSD 8.1

I've been using ZFS on my FreeBSD box for some time now, and it's been serving out gigs and gigs of data faithfully for some time.

I am looking into OpenSolaris, but with the slow (or never) release of 2010.x I'm still fully a FreeBSD shop at this stage for Unix.

I've just upgraded to 8.1 Release a few days ago, and I'm liking the new release - I took the opportunity to also upgrade my Samba to 3.4.8 at the same time, so the system seems perkier. I didn't have the time to do any real speed tests, so take that for what it's worth.

BUT - I've been getting some "kmem_map to small" panics when I tried to copy down a 160 gig file. I can't recall if I've ever moved such a large file before, so I don't think this is an 8.1 issue.

Some digging around and I found this information on the FreeBSD mailing list that I wanted to pass along.


From: Steve Polyack
Subject: Re: Freebsd 8.0 kmem map too small
Date: Wednesday, May 5, 2010 - 7:46 am


On a system here with 8GB of RAM and a very large zpool consisting of
multiple zdevs, little tuning was needed. The system is running
8-STABLE(amd64) as of a month or two ago. The only things I have set in
/boot/loader.conf are:
vm.kmem_size="12G"
vfs.zfs.arc_max="4G"

Setting kmem_size to 12G came from a combination of recommendations I
saw in various mailing list posts (you can probably dig them up).
Setting it to the physical memory size was the initial recommendation,
while others recommended 1.5x physical memory size to help prevent
fragmentation/wasted space in kmem.

Regardless, this has served us quite well for the ~6 months the system
has been in use. It has never crashed, even under intensive
multi-threaded benchmarking.

Other people recommend vm.kmem_size to be 1/2 of available RAM, and then vsf.zfs.arc_max to be 512M less than that, but that doesn't make sense to me - What's the point of RAM if you can't use it? This box only runs ZFS and Samba, let these things have some memory.

I've now set those same values as Steve on my FreeBSD system in question is running 8 gigs as well. I'm going to start my 160 gig copy again (which BTW, I get ~34 Meg/Sec according to Windows 7)

If you don't hear back from me, it's working well for me. :-)