lagg performance penalty between ESXi 4.1 and FreeBSD 8.1

While doing some benchmarks of my ZFS/NFS/ESX setups, I started fiddling with the lagg driver to create a fail-over connection between my ESX server and the ZFS server.

Both boxes have the Intel x520DA 10Gbe adapter, which has 2 10 Gig ports.

ESXi doesn't offer proper load-balance between two ports unless you get into the Cisco Nexus vswitch, which I currently don't have.


I ran a few quick tests with my usual 2003 32 Bit VM and Performance test 6.1, and here's what I found;

Type of Test        With lagg                Without lagg    % Diff

File Server:            478.16                     491                     2.6

Webserver:            84.19                       86.46                  2.6

Workstation:          0.52                         0.6                      13

Database:              127.88                     136.15                 6

Every test was slower with lagg being used.

This is something interesting to think about, as a lot of admins enable lagg or similar failover technology without thinking that there may be a performance impact. 

At this stage, I don't know if it's on the lagg side, or ESX's side - I'd have to run further tests. I'll add it to the list of things that I'd like to know if I had time.

But for now, I know that I'll take a small performance hit to have this fail-over setup.