From owner-freebsd-scsi@FreeBSD.ORG Wed Feb 29 16:38:35 2012 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11AE61065679 for ; Wed, 29 Feb 2012 16:38:35 +0000 (UTC) (envelope-from Kashyap.Desai@lsi.com) Received: from na3sys009aog119.obsmtp.com (na3sys009aog119.obsmtp.com [74.125.149.246]) by mx1.freebsd.org (Postfix) with ESMTP id 556078FC1C for ; Wed, 29 Feb 2012 16:38:34 +0000 (UTC) Received: from paledge01.lsi.com ([192.19.193.42]) (using TLSv1) by na3sys009aob119.postini.com ([74.125.148.12]) with SMTP ID DSNKT05UiVUu75SGaL0+GhwFIyyornuLZdL4@postini.com; Wed, 29 Feb 2012 08:38:34 PST Received: from PALCAS01.lsi.com (128.94.213.117) by PALEDGE01.lsi.com (192.19.193.42) with Microsoft SMTP Server (TLS) id 8.3.213.0; Wed, 29 Feb 2012 11:43:04 -0500 Received: from inbexch02.lsi.com (135.36.98.40) by PALCAS01.lsi.com (128.94.213.117) with Microsoft SMTP Server (TLS) id 8.3.213.0; Wed, 29 Feb 2012 11:38:32 -0500 Received: from inbmail01.lsi.com ([135.36.98.64]) by inbexch02.lsi.com ([135.36.98.40]) with mapi; Wed, 29 Feb 2012 22:08:28 +0530 From: "Desai, Kashyap" To: "dgilbert@interlog.com" , Jason Wolfe Date: Wed, 29 Feb 2012 22:08:25 +0530 Thread-Topic: LSI2008 controller clobbers first disk with new LSI mps driver Thread-Index: Acz1qVrkiusUZ1ojROe2toFMjfTklwBVq20w Message-ID: References: <4F450814.4020100@interlog.com> <4F4C14A8.3050105@interlog.com> In-Reply-To: <4F4C14A8.3050105@interlog.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-scsi@freebsd.org" , "McConnell, Stephen" Subject: RE: LSI2008 controller clobbers first disk with new LSI mps driver X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Feb 2012 16:38:35 -0000 Hi Jason, I have started discussion with LSI internal folks to get better clarity on = this issue. Since our key person is on vacation, we may get clarity on this= next week. I cannot provide some temporary workaround in upstream(because this is agai= nst our design), but if you want to use for your environment, I can provide= you some temporary patch. Doug, Thanks for providing your view and I have convey this to our architect. ~ Kashyap > -----Original Message----- > From: Douglas Gilbert [mailto:dgilbert@interlog.com] > Sent: Tuesday, February 28, 2012 5:11 AM > To: Jason Wolfe > Cc: Desai, Kashyap; freebsd-scsi@freebsd.org; McConnell, Stephen > Subject: Re: LSI2008 controller clobbers first disk with new LSI mps > driver >=20 > On 12-02-27 02:59 PM, Jason Wolfe wrote: > > On Wed, Feb 22, 2012 at 9:11 AM, Desai, Kashyap > wrote: > >> > >> > >>> -----Original Message----- > >>> From: Douglas Gilbert [mailto:dgilbert@interlog.com] > >>> Sent: Wednesday, February 22, 2012 8:52 PM > >>> To: Desai, Kashyap > >>> Cc: Jason Wolfe; freebsd-scsi@freebsd.org; McConnell, Stephen > >>> Subject: Re: LSI2008 controller clobbers first disk with new LSI mps > >>> driver > >>> > >>> On 12-02-22 03:39 AM, Desai, Kashyap wrote: > >>>> Here is a possible root cause of this issue. > >>>> > >>>> Enclosure which you are using in your setup (might be) not > configured > >>> properly. > >>>> > >>>> You have Enclosure with 12 Slots + 1 SES Device. > >>>> See below detail from the log. > >>>> > >>>> EventDataLength: 5 > >>>> AckRequired: 0 > >>>> Event: SasEnclDeviceStatusChange (0x1d) > >>>> EventContext: 0x0 > >>>> EnclosureHandle: 0x2 > >>>> ReasonCode: Added > >>>> PhysicalPort: 0 > >>>> NumSlots: 13 > >>>> StartSlot: 0 > >>>> PhyBits: 0xff > >>>> > >>>> StartSlot is 0 in this case. > >>>> Correct behavior should be each device on your enclosure must have > >>> different slot number starting from 0 till 12. > >>>> I have doubt that SES device has not configured well and it is > using > >>> slot-0 as default. This can create issue for actual device which is > >>> connected to slot-0. > >>>> So In your setup you will have slot-0 till slot-11 assigned for > actual > >>> Phys of your enclosures and again slot-0 is assigned for SES device > >>> instead of Slot-12. > >>> > >>> No. SAS-2 expanders typically have an integral SES device on an > >>> expander _virtual_ phy (see SMP DISCOVER (LIST) response). Once > >>> you see that virtual phy flag the slot number is irrelevant. > >> > >> Doug, > >> > >> I need some more info so that I can understand your point better. > >> > >> I have one Enclosure setup on FreeBSD. Here is smp_discover output. > (smp_discover_list is failing for me) > >> > >> phy 0: inaccessible (phy vacant) > >> phy 1: inaccessible (phy vacant) > >> phy 2: inaccessible (phy vacant) > >> phy 3: inaccessible (phy vacant) > >> phy 4:S:attached:[500605b012345888:03 i(SSP+STP+SMP)] 6 Gbps > >> phy 5:S:attached:[500605b012345888:02 i(SSP+STP+SMP)] 6 Gbps > >> phy 6:S:attached:[500605b012345888:01 i(SSP+STP+SMP)] 6 Gbps > >> phy 7:S:attached:[500605b012345888:00 i(SSP+STP+SMP)] 6 Gbps > >> phy 12:D:attached:[5000c5003bc2c389:00 t(SSP)] 6 Gbps > >> phy 13:D:attached:[500000e116ee91e2:00 t(SSP)] 6 Gbps > >> phy 14:D:attached:[5000c5003bc308e5:00 t(SSP)] 6 Gbps > >> phy 15:D:attached:[5000c5003bc2f0d1:00 t(SSP)] 6 Gbps > >> phy 16:D:attached:[5000c5003bc2ff3d:00 t(SSP)] 6 Gbps > >> phy 17:D:attached:[5000c5003bae5fdd:00 t(SSP)] 6 Gbps > >> phy 18:D:attached:[5000c5003bae5eb1:00 t(SSP)] 6 Gbps > >> phy 19:D:attached:[5000c5003bc2d135:00 t(SSP)] 6 Gbps > >> phy 20:D:attached:[5000c5003baea36d:00 t(SSP)] 6 Gbps > >> phy 21:D:attached:[5000c5003bc2a8c9:00 t(SSP)] 6 Gbps > >> phy 22:D:attached:[5000c5003bc237a9:00 t(SSP)] 6 Gbps > >> phy 23:D:attached:[5000c5003bc2cec1:00 t(SSP)] 6 Gbps > >> phy 24:D:attached:[500000e01d92cb52:00 t(SSP)] 3 Gbps > >> phy 25:D:attached:[500000e01d74cfb2:00 t(SSP)] 3 Gbps > >> phy 26:D:attached:[500000e01d656052:00 t(SSP)] 3 Gbps > >> phy 27:D:attached:[500000e01d7cad52:00 t(SSP)] 3 Gbps > >> phy 28:D:attached:[500c04f2b64cdd1c:00 t(SATA)] 3 Gbps > >> phy 29:D:attached:[500c04f2b64cdd1d:00 t(SATA)] 3 Gbps > >> phy 30:D:attached:[500000e01d73c262:00 t(SSP)] 3 Gbps > >> phy 31:D:attached:[500000e01d536b22:00 t(SSP)] 3 Gbps > >> phy 32:D:attached:[500000e01d92cab2:00 t(SSP)] 3 Gbps > >> phy 33:D:attached:[500000e01afd8792:00 t(SSP)] 3 Gbps > >> phy 34:D:attached:[5000c5003bc30301:00 t(SSP)] 6 Gbps > >> phy 35:D:attached:[5000c5003bb09a69:00 t(SSP)] 6 Gbps > >> phy 36:D:attached:[500c04f2b64cdd3d:00 V i(SSP) t(SSP)] 6 Gbps<- > -- This has virtual phy set. > >> > >> What I understood from your explanation is if we have virt_phy field > set, we should not trust slot for that entry. > >> You are suggesting to use phy index instead of slot. Just for info: > But how to see Slot details mapping with phy ? >=20 > Kashyap, > I haven't written a SAS discover algorithm but there > must be plenty of examples out there. One way to do it > is to find all the phy_ids attached to targets, in this > case there are SAS (SSP) and SATA targets. Each SATA > target phy_id will correspond to one SATA disk (or could be an > ATAPI device (e.g. DVD/BD player)). The SSP targets are a > bit trickier because two (or more) phys could be connected > to the same target (either a wide port or multiple (target) > ports). With a wide port each component phy has the same > attached SAS address (so above you have a wide initiator > port (phy ids 4,5,6,7) but no wide target ports). If a > SAS disk has multiple target ports connected, FreeBSD > probably has a device node for each. So for each SCSI (SSP) > target port you need a REPORT LUNS command issued on LUN 0 > (or the REPORT LUNS well known logical unit) to find the > LUs it contains. A device node is created for each LU. >=20 > Anyway I'm sure many folks in LSI know the SAS discover > process better than I do. Ask them :-) Surely most of > the above is already done in your HBA's firmware. >=20 >=20 > BTW I don't think slot numbers are reliable and don't apply > to things on virtual phys so they will just cause you > problems when used in the discover process, as this thread > attests. The BIOS on LSI's HBAs does a discover process > but is only interested in bootable devices so SES devices > don't appear. >=20 >=20 > Doug Gilbert >=20 > > Kashyap, > > > > Let me know if there are any changes agreed upon, I'm happy to test > > out patches as this is affecting a large number of our machines. I > > can only imagine the same for others as they start to upgrade, as this > > is standard SuperMicro hardware. > > > > Thanks, > > Jason > >