Date: Mon, 27 Feb 2012 18:41:28 -0500 From: Douglas Gilbert <dgilbert@interlog.com> To: Jason Wolfe <nitroboost@gmail.com> Cc: "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org>, "Desai, Kashyap" <Kashyap.Desai@lsi.com>, "McConnell, Stephen" <Stephen.McConnell@lsi.com> Subject: Re: LSI2008 controller clobbers first disk with new LSI mps driver Message-ID: <4F4C14A8.3050105@interlog.com> In-Reply-To: <CAAAm0r2x4wAkPzpJkCJ8FYnFkxb3RrXnENRZi-Kgfh=y3ZuqEA@mail.gmail.com> References: <CAAAm0r2NFhF=eh2bOPMnVN8E6e2o0KfaST0N-M_gWoJHpFOLmQ@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D34500@inbmail01.lsi.com> <CAFPOs6pwb44oNabH5vabDPJyFutMKa5mhgvHY=HkQVpV20YiYw@mail.gmail.com> <CAAAm0r1pWN-F=madGdk7N%2BoRuZmSD5_MAYwLh6By126L0CTGuw@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D34558@inbmail01.lsi.com> <CAAAm0r1x15_ho2MD0tX7Y7A6mnU2N6zihNOz_Qz=jpsyBkDCWQ@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D3455B@inbmail01.lsi.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D34626@inbmail01.lsi.com> <CAAAm0r3_S2jTG=Te4UhLqHPqiXq7_aAOHNp=W3jb4KLJx9PTRg@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D34748@inbmail01.lsi.com> <CAAAm0r3oRTcfipyVcp9nE1CL3dcK7cft8AUSf%2BfGYVK90b2A0w@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D347BB@inbmail01.lsi.com> <4F450814.4020100@interlog.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D34840@inbmail01.lsi.com> <CAAAm0r2x4wAkPzpJkCJ8FYnFkxb3RrXnENRZi-Kgfh=y3ZuqEA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12-02-27 02:59 PM, Jason Wolfe wrote: > On Wed, Feb 22, 2012 at 9:11 AM, Desai, Kashyap<Kashyap.Desai@lsi.com> wrote: >> >> >>> -----Original Message----- >>> From: Douglas Gilbert [mailto:dgilbert@interlog.com] >>> Sent: Wednesday, February 22, 2012 8:52 PM >>> To: Desai, Kashyap >>> Cc: Jason Wolfe; freebsd-scsi@freebsd.org; McConnell, Stephen >>> Subject: Re: LSI2008 controller clobbers first disk with new LSI mps >>> driver >>> >>> On 12-02-22 03:39 AM, Desai, Kashyap wrote: >>>> Here is a possible root cause of this issue. >>>> >>>> Enclosure which you are using in your setup (might be) not configured >>> properly. >>>> >>>> You have Enclosure with 12 Slots + 1 SES Device. >>>> See below detail from the log. >>>> >>>> EventDataLength: 5 >>>> AckRequired: 0 >>>> Event: SasEnclDeviceStatusChange (0x1d) >>>> EventContext: 0x0 >>>> EnclosureHandle: 0x2 >>>> ReasonCode: Added >>>> PhysicalPort: 0 >>>> NumSlots: 13 >>>> StartSlot: 0 >>>> PhyBits: 0xff >>>> >>>> StartSlot is 0 in this case. >>>> Correct behavior should be each device on your enclosure must have >>> different slot number starting from 0 till 12. >>>> I have doubt that SES device has not configured well and it is using >>> slot-0 as default. This can create issue for actual device which is >>> connected to slot-0. >>>> So In your setup you will have slot-0 till slot-11 assigned for actual >>> Phys of your enclosures and again slot-0 is assigned for SES device >>> instead of Slot-12. >>> >>> No. SAS-2 expanders typically have an integral SES device on an >>> expander _virtual_ phy (see SMP DISCOVER (LIST) response). Once >>> you see that virtual phy flag the slot number is irrelevant. >> >> Doug, >> >> I need some more info so that I can understand your point better. >> >> I have one Enclosure setup on FreeBSD. Here is smp_discover output. (smp_discover_list is failing for me) >> >> phy 0: inaccessible (phy vacant) >> phy 1: inaccessible (phy vacant) >> phy 2: inaccessible (phy vacant) >> phy 3: inaccessible (phy vacant) >> phy 4:S:attached:[500605b012345888:03 i(SSP+STP+SMP)] 6 Gbps >> phy 5:S:attached:[500605b012345888:02 i(SSP+STP+SMP)] 6 Gbps >> phy 6:S:attached:[500605b012345888:01 i(SSP+STP+SMP)] 6 Gbps >> phy 7:S:attached:[500605b012345888:00 i(SSP+STP+SMP)] 6 Gbps >> phy 12:D:attached:[5000c5003bc2c389:00 t(SSP)] 6 Gbps >> phy 13:D:attached:[500000e116ee91e2:00 t(SSP)] 6 Gbps >> phy 14:D:attached:[5000c5003bc308e5:00 t(SSP)] 6 Gbps >> phy 15:D:attached:[5000c5003bc2f0d1:00 t(SSP)] 6 Gbps >> phy 16:D:attached:[5000c5003bc2ff3d:00 t(SSP)] 6 Gbps >> phy 17:D:attached:[5000c5003bae5fdd:00 t(SSP)] 6 Gbps >> phy 18:D:attached:[5000c5003bae5eb1:00 t(SSP)] 6 Gbps >> phy 19:D:attached:[5000c5003bc2d135:00 t(SSP)] 6 Gbps >> phy 20:D:attached:[5000c5003baea36d:00 t(SSP)] 6 Gbps >> phy 21:D:attached:[5000c5003bc2a8c9:00 t(SSP)] 6 Gbps >> phy 22:D:attached:[5000c5003bc237a9:00 t(SSP)] 6 Gbps >> phy 23:D:attached:[5000c5003bc2cec1:00 t(SSP)] 6 Gbps >> phy 24:D:attached:[500000e01d92cb52:00 t(SSP)] 3 Gbps >> phy 25:D:attached:[500000e01d74cfb2:00 t(SSP)] 3 Gbps >> phy 26:D:attached:[500000e01d656052:00 t(SSP)] 3 Gbps >> phy 27:D:attached:[500000e01d7cad52:00 t(SSP)] 3 Gbps >> phy 28:D:attached:[500c04f2b64cdd1c:00 t(SATA)] 3 Gbps >> phy 29:D:attached:[500c04f2b64cdd1d:00 t(SATA)] 3 Gbps >> phy 30:D:attached:[500000e01d73c262:00 t(SSP)] 3 Gbps >> phy 31:D:attached:[500000e01d536b22:00 t(SSP)] 3 Gbps >> phy 32:D:attached:[500000e01d92cab2:00 t(SSP)] 3 Gbps >> phy 33:D:attached:[500000e01afd8792:00 t(SSP)] 3 Gbps >> phy 34:D:attached:[5000c5003bc30301:00 t(SSP)] 6 Gbps >> phy 35:D:attached:[5000c5003bb09a69:00 t(SSP)] 6 Gbps >> phy 36:D:attached:[500c04f2b64cdd3d:00 V i(SSP) t(SSP)] 6 Gbps<--- This has virtual phy set. >> >> What I understood from your explanation is if we have virt_phy field set, we should not trust slot for that entry. >> You are suggesting to use phy index instead of slot. Just for info: But how to see Slot details mapping with phy ? Kashyap, I haven't written a SAS discover algorithm but there must be plenty of examples out there. One way to do it is to find all the phy_ids attached to targets, in this case there are SAS (SSP) and SATA targets. Each SATA target phy_id will correspond to one SATA disk (or could be an ATAPI device (e.g. DVD/BD player)). The SSP targets are a bit trickier because two (or more) phys could be connected to the same target (either a wide port or multiple (target) ports). With a wide port each component phy has the same attached SAS address (so above you have a wide initiator port (phy ids 4,5,6,7) but no wide target ports). If a SAS disk has multiple target ports connected, FreeBSD probably has a device node for each. So for each SCSI (SSP) target port you need a REPORT LUNS command issued on LUN 0 (or the REPORT LUNS well known logical unit) to find the LUs it contains. A device node is created for each LU. Anyway I'm sure many folks in LSI know the SAS discover process better than I do. Ask them :-) Surely most of the above is already done in your HBA's firmware. BTW I don't think slot numbers are reliable and don't apply to things on virtual phys so they will just cause you problems when used in the discover process, as this thread attests. The BIOS on LSI's HBAs does a discover process but is only interested in bootable devices so SES devices don't appear. Doug Gilbert > Kashyap, > > Let me know if there are any changes agreed upon, I'm happy to test > out patches as this is affecting a large number of our machines. I > can only imagine the same for others as they start to upgrade, as this > is standard SuperMicro hardware. > > Thanks, > Jason >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F4C14A8.3050105>