Date: Mon, 8 Dec 2014 13:44:19 -0700 From: Jason Wolfe <nitroboost@gmail.com> To: Alan Somers <asomers@freebsd.org> Cc: FreeBSD-scsi <freebsd-scsi@freebsd.org> Subject: Re: LSI SAS 3008 card - 35 out of 36 disks detected Message-ID: <CAAAm0r2ZzmOFHOk0uGEoG=0p9ujJSiPsL2v=_FTOx%2Bt%2B0jg09Q@mail.gmail.com> In-Reply-To: <CAOtMX2hmTKjmcUpnO7vzP4VJ_1qY%2BgUb1hvZD%2BgZaBviN1Rs4g@mail.gmail.com> References: <54822835.3080800@crystal.harvard.edu> <CAOtMX2j8K3aKST1mZ%2BEMKo4OVKiR6MUy%2B1VBQ1rMams5h7Hy_w@mail.gmail.com> <DD7EC4AD-A4B6-46F7-8F1C-9AD111DF20C6@crystal.harvard.edu> <CAOtMX2hmTKjmcUpnO7vzP4VJ_1qY%2BgUb1hvZD%2BgZaBviN1Rs4g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Dec 8, 2014 at 11:20 AM, Alan Somers <asomers@freebsd.org> wrote: > On Mon, Dec 8, 2014 at 8:58 AM, Justin O'Conor > <oconnor@crystal.harvard.edu> wrote: >> Hi All, >> Thanks, this is encouraging. smp_discover ses0|1 see 36 sata disks. This is from the 10.1 install. >> > > There are certainly some inconsistencies in the smp_discover > responses. For example, the SEP on ses0 (phy identifier: 36) has > "connector type: SAS virtual connector" and "connector element index: > 24" But the SEP on ses1 (phy identifier: 28) has "connector type: No > information" and "connector element index: 0". Also note that "phy > identifier: 12" on ses1 has "connector element index: 0". That would > be the first slot on the rear expander, if the slots and phys are > numbered the same way. My best guess is that phy 12 and phy 28 mapped > to the same map_idx in mpr_mapping.c:1168. So the information for the > SEP overwrote the information for the first disk slot. If my guess is > true, then recabling your chassis as you suggested wouldn't help. > However, you might try the attached but untested patch. It will > prevent the SEP from being added to the mapping table while printing a > useful error message. If I'm correct, then the patch will let you use > all 36 disk slots, but you won't have ses1 anymore. > > In the meantime, I'll try to reproduce your problem. I have all the > required equipment in my lab. > > -Alan > I believe this is the same issue we ran into a few years ago on the LSI2008, where the ses0 device would map over the boot disk. It's long and spans over multiple months, so just the relevant bits: Initial report: https://lists.freebsd.org/pipermail/freebsd-scsi/2012-February/005243.html The issue seems to be a shortcoming in the detection method where it has no problem assigning the ses over an already mapped disk, LSI's initial response was to use the LSI config utility and map the drives manually: https://lists.freebsd.org/pipermail/freebsd-scsi/2012-February/005243.html https://lists.freebsd.org/pipermail/freebsd-scsi/2012-February/005267.html https://lists.freebsd.org/pipermail/freebsd-scsi/2012-March/005343.html This was not an option for us as we had 2000 of these devices in the field, and entering the LSI BIOS would be a large undertaking. In the end after an internal dialogue with LSI guys, Kashyap was kind enough to write a one off for us, that never made it upstream. It simply assigns the ses device to max target + 1 when a conflict is found. The core issue seems to be with the way LSI detects and assigns devices on FreeBSD, so this is by no means 'proper', but it's sound enough so resolve the issue for us on the LSI2008. In case it's interesting to anyone: http://nitrology.com/mps_fix-fbsd10stable.diff Jason
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAAm0r2ZzmOFHOk0uGEoG=0p9ujJSiPsL2v=_FTOx%2Bt%2B0jg09Q>