Date: Sun, 10 Mar 2019 08:14:16 -0700 From: Enji Cooper <yaneurabeya@gmail.com> To: Evgheni Melman <mevgheni@gmail.com> Cc: freebsd-current@freebsd.org Subject: Re: PMC Sierra SPC SAS-SATA Card + ESXi Message-ID: <8A07386F-9C19-4438-B659-EDC8193043AF@gmail.com> In-Reply-To: <CAMxL4t1D-L5L78eHTe2wEVPUGO8f2mC9b4Raj-r6kUqVvgjL9A@mail.gmail.com> References: <CAMxL4t1D-L5L78eHTe2wEVPUGO8f2mC9b4Raj-r6kUqVvgjL9A@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
> On Mar 9, 2019, at 18:50, Evgheni Melman <mevgheni@gmail.com> wrote: > > So I got this weird setup, where I need to passthrough a PMC-Sierra PM8003 > card (same as PM8001, but with external QSFP afaik) to FreeBSD 11-STABLE > running in ESXi. > The card works fine in baremetal FreeBSD 11. Boot is ok, all disks > detected. When done in ESXi with passthrough though: > 1. First try, host crashed with PSOD. After tweaking passthrough.map in > esxi to do d3d0 reset, host does not psod anymore. > 2. FreeBSD kernel panics at pciAttach. After analyzing the driver > (sys\dev\pms), turns out it only works in MSI-X mode. Disabling > msi_blacklist in loader.conf makes FreeBSD not panic. > 3. FreeBSD deadlocks on boot. After analyzing the driver more (turning on > all debug features, and adding a few extra), I got this log (pastebin link: > https://pastebin.com/cEGwFZZ2). The code tries to initialize SGPIO, but for > whatever reason does not get an interrupt reply when run on ESXi which > makes it loop for infinity waiting for said interrupt. > > The deadlock happens in the > > while(!sgpioResponseSet) > { > tiCOMDelayedInterruptHandler(tiRoot, 0, 1, tiNonInterruptContext); > } > > section of SendSgpioRequest in > sys\dev\pms\RefTisa\tisa\sassata\common\tdport.c > > Sgpi seems not-so-critical feature to me, so I just limited the while loop > to 100 iterations, which lets SendSgpioRequest return an error that is not > considered fatal, and lets the code execute further which can be seen in > the log: https://pastebin.com/Rk6MD5ci (disregard the final fatal trap, I > think it is caused by me and/or my debugging code, same modifications > without debugging features do not trap and I get same trap on baremetal > with my debug kernel). > > With those modifications, pciAttach is successful, I finally get a boot, > the card exists in lspci, BUT no drives connected to it are detected. > > At this stage, I am out of ideas, looking for suggestions. If you’re trying to get the host production ready, I recommend using an LSI/Avago based card (or emulate it in ESXi), or avoid FreeBSD if you can. The driver in FreeBSD is based on a Linux driver that’s over a decade old and had a number of stability bugs the last time I used it [2 years ago]. -Enjihelp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8A07386F-9C19-4438-B659-EDC8193043AF>