From owner-freebsd-scsi@freebsd.org Wed Sep 9 23:17:56 2015 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAE02A01BB9 for ; Wed, 9 Sep 2015 23:17:56 +0000 (UTC) (envelope-from jwd@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id CEA1E1BC0; Wed, 9 Sep 2015 23:17:56 +0000 (UTC) (envelope-from jwd@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 821) id CD70A1B8E; Wed, 9 Sep 2015 23:17:56 +0000 (UTC) Date: Wed, 9 Sep 2015 23:17:56 +0000 From: John De Boskey To: Alan Somers Cc: FreeBSD-SCSI Subject: Re: ses/pass devices (enclosure/processor devices) not all showing up? Message-ID: <20150909231756.GA88399@FreeBSD.org> References: <20150909033510.GA34413@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Sep 2015 23:17:57 -0000 ----- Alan Somers's Original Message ----- > On Tue, Sep 8, 2015 at 9:35 PM, John De Boskey wrote: > > Hi Folks - > > > > I have a shelf with 84 sata drives. All drives show up > > correctly and are accessible. The shelf appears to have > > multiple processor devices and one enclosure device internally. > > For instance: > > > > # camcontrol devlist | grep XYRATEX > > at scbus7 target 159 lun 0 (pass18) > > at scbus7 target 188 lun 0 (pass47) > > at scbus7 target 217 lun 0 (pass76) > > at scbus7 target 232 lun 0 (pass91) > > > > # camcontrol devlist | grep ses > > at scbus7 target 144 lun 0 (ses0,pass3) > > > > # camcontrol smprg pass18 | grep 'Number of Phys:' > > Number of Phys: 25 > > # camcontrol smprg pass47 | grep 'Number of Phys:' > > Number of Phys: 37 > > # camcontrol smprg pass76 | grep 'Number of Phys:' > > Number of Phys: 37 > > # camcontrol smprg pass91 | grep 'Number of Phys:' > > Number of Phys: 25 > > # camcontrol smprg ses0 | grep 'Number of Phys:' > > Number of Phys: 37 > > > > # camcontrol smpphylist pass18 > > 25 PHYs: > > PHY Attached SAS Address > > 0 0x5000c500585f4b52 (pass4,da0) ... > > I have linked dmesg, camcontrol devlist, and sas2ircu output below: > > > > http://people.freebsd.org/~jwd/sespass/dmesg.txt mps messaging enabled. > > > > http://people.freebsd.org/~jwd/sespass/devlist.txt > > > > http://people.freebsd.org/~jwd/sespass/sas2ircu.txt > > > > Thanks, > > John Hi Alan, > I'm not sure exactly what you're asking. Are you just wondering why In a simple form, I'm trying to figure out why 'camcontrol smpphyslist ses0' produces no output, while 'camcontrol smpphyslist pass18' does. > your pass devices don't also have ses nodes? I think I know why. > Your JBOD problem has five SAS expander chips, though ses0 might > actually be some other kind of SAS target chip. pass18, pass47, > pass76, and pass91 are configured to report ses0's SAS Address as the > address of their SEP. The LSI HBA's interpretation is that there is Could you expand on the concept of the SAS Address being reported? When I look at the address of the devices in the sas2ircu output they are unique. For example: Device is a Processor device Enclosure # : 2 Slot # : 1 SAS Address : 50050cc-1-0d2f-fabe State : Standby (SBY) Manufacturer : XYRATEX Model Number : DEFAULT-SD-R24 Firmware Revision : 3034 Serial No : PMCSIERA Device is a Processor device Enclosure # : 2 Slot # : 2 SAS Address : 50050cc-1-0d2f-fafe State : Standby (SBY) Manufacturer : XYRATEX Model Number : DEFAULT-SD-R36 Firmware Revision : 3034 Serial No : PMCSIERA ie: they show as unique. However, they don't appear to be unique as reported by camcontrol smprg. I haven't been able to determine where in the code this linkage is being done yet. > only one SES processor. So FreeBSD reports one ses device, and the > other expanders just show up as pass devices. Unless the manufacturer > royally screwed up (I doubt it), ses0 will report info for all 84 > disks in its SES status pages. camcontrol smpphylist ses0 - returns no information. camcontrol smpphylist pass18 - returns a drive list camcontrol smpphylist pass47 - returns a drive list camcontrol smpphylist pass76 - returns a drive list camcontrol smpphylist pass91 - returns a drive list > Bill is right, sg3_utils is your friend. However, I doubt you'll see > any secondary subenclosures. That feature isn't much used, and it's > not necessary in order for ses0 to report all 84 drives. Running > these two commands will probably tell you most of what you need to > know: > > sg_ses -p 1 ses0 > sg_ses -p 2 ses0 I have run the above commands and linked the output below. http://people.freebsd.org/~jwd/sespass/sgsesp1.txt http://people.freebsd.org/~jwd/sespass/sgsesp2.txt descriptor for each disk Thoughts? Thanks, John > -Alan