Showing posts with label LACP. Show all posts
Showing posts with label LACP. Show all posts

Wednesday, March 7, 2012

FreeBSD 9.0 and Intel X520's - ixgbe 2.3.11 won't allow LACP to function, but 2.4.4 does

Working with the Intel x520 adapters can be a little bit challenging if you're not used to them. I'll expand on what I've learned over the last year or so in a separate post.

For now, I hope to save someone a weekend of wasted time - If you're attempting to use a FreeBSD 9.0 system with an Intel x520 (ix/ixgbe driver), and you want to setup a LACP link, it won't work.

Here's what you'll see:

lagg1: flags=8843 metric 0 mtu 9000
        options=401bb
        ether d4:ae:52:5c:5d:af
        inet 172.16.0.30 netmask 0xffffff00 broadcast 172.16.0.255
        inet6 fe80::d6ae:52ff:fe5c:5daf%lagg1 prefixlen 64 scopeid 0xe
        nd6 options=29
        media: Ethernet autoselect
        status: active
        laggproto lacp
        laggport: ix1 flags=18 COLLECTING,DISTRIBUTING
        laggport: ix0 flags=18 COLLECTING,DISTRIBUTING


You'll notice it's just COLLECTING, DISTRIBUTING, where a properly functioning link should be ACTIVE, COLLECTING, DISTRIBUTING.

Check your driver version like so:

#sysctl dev.ix

dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.3.11
dev.ix.0.%driver: ix
dev.ix.0.%location: slot=0 function=0
dev.ix.0.%pnpinfo: vendor=0x8086 device=0x10f8 subvendor=0x8086 subdevice=0x000c class=0x020000



Older drivers seem to work on slightly older FreeBSD 9-Beta systems (like ixgbe 2.3.10).

The eventual fix was to download the latest FreeBSD driver from Intel's website (v2.4.4) compile it as a module, and then add it to my /boot/loader.conf file. Note, you can rename the produced ixgbe to to ixgbe2.4.4.ko before you copy it to /boot/kernel/, allowing you more control over which version of ixgbe you load. My kernel had ixgbe compiled in, but it still accepts the new 2.4.4 loaded via loader.conf:

ixgbe2.4.4_load="YES"

If you try loading a .ko over your existing ixgbe built into the kernel, make sure you check your driver version with a "sysctl dev.ix" command - I found that an older FreeBSD-9.0-RELEASE wouldn't load my new 2.4.4, but systems closer to the build time of my test FreeBSD-9-STABLE machine would.

This brings up questions of if we really want a lot of device drivers in the kernel, as it can make troubleshooting harder, but that's another post as well. 

You'll have to do a quick edit of the code to get it to compile,as it's complaining about the bool typedef.  Basically you need to remove the reference, and change the 4-6 instances of boolean_t to bool (you'll see the errors).

If that's too complex, hopefully Jack will have 2.4.4 ported to FreeBSD-9-STABLE shortly, I made a post to the stable list, and he's aware of it.