If you want to pass a PCIe device (such as a LSI SAS controller) through ESXi to a FreeBSD VM, you're probably aware that you may need to disable MSI/MSI-X interrupts to make it work.
However, I'm finding that while this setting will make the system boot, if you add a second vCPU, you will quickly stat to see Interrupt Storm messages on your console, and one of your CPU's will be nearly 100% used on interrupt tasks.
The easy fix for me? Only disable MSIX, not MSI.
Here are the relevant lines from my /boot/loader.conf:
hw.pci.enable_msix=0
hw.pci.enable_msi=1
All of my interrupt storm issues are now gone.
However, I'm finding that while this setting will make the system boot, if you add a second vCPU, you will quickly stat to see Interrupt Storm messages on your console, and one of your CPU's will be nearly 100% used on interrupt tasks.
The easy fix for me? Only disable MSIX, not MSI.
Here are the relevant lines from my /boot/loader.conf:
hw.pci.enable_msix=0
hw.pci.enable_msi=1
All of my interrupt storm issues are now gone.
Just a heads up for FreeBSD 9/stable you currently need to add a 3rd line(hw.pci.honor_msi_blacklist=0). I've sent the maintainers a patch that should remove the need for all 3 lines, keeping my fingers crossed that it does fix the problem.
ReplyDeleteBTW: This blog has helped me a ton with FreeBSD, ZFS, ESXi, and NFS.
pbucher: I'm glad my blog has been of help to you - That was my goal in creating it in the first place.
ReplyDeleteI'm very interested in your patch, could you share it here?
[Send a commend with your email (I won't make it a published comment) if it's too much of a hassle to fit into a comment box, and I'll give you my email. ]
I'll look into the honor_msi line shortly, as I still have grief with MSI/MSIX for passthru and FreeBSD.