From owner-freebsd-scsi@FreeBSD.ORG Thu May 28 15:22:33 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6300236C; Thu, 28 May 2015 15:22:33 +0000 (UTC) (envelope-from mikej@mikej.com) Received: from mx2.paymentallianceintl.com (mx2.paymentallianceintl.com [216.26.158.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx2.paymentallianceintl.com", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 264CC322; Thu, 28 May 2015 15:22:32 +0000 (UTC) (envelope-from mikej@mikej.com) Received: from firewall.mikej.com (162-230-214-65.lightspeed.lsvlky.sbcglobal.net [162.230.214.65]) by mx2.paymentallianceintl.com (8.15.1/8.15.1) with ESMTPS id t4SFMUjq002425 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 28 May 2015 11:22:31 -0400 (EDT) (envelope-from mikej@mikej.com) X-Authentication-Warning: mx2.paymentallianceintl.com: Host 162-230-214-65.lightspeed.lsvlky.sbcglobal.net [162.230.214.65] claimed to be firewall.mikej.com Received: from mail.mikej.com (firewall [192.168.6.63]) by firewall.mikej.com (8.15.1/8.14.9) with ESMTP id t4SFMTMh051460; Thu, 28 May 2015 11:22:30 -0400 (EDT) (envelope-from mikej@mikej.com) X-Authentication-Warning: firewall.mikej.com: Host firewall [192.168.6.63] claimed to be mail.mikej.com MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 28 May 2015 11:22:29 -0400 From: Michael Jung To: Alan Somers Cc: FreeBSD-scsi , asomers@gmail.com Subject: Re: /dev/ses not created In-Reply-To: References: <76e9e3f41990026290382e5fa9c9b8d3@mail.mikej.com> Message-ID: <026292acf4288dbde00ce9bcb8686490@mail.mikej.com> X-Sender: mikej@mikej.com User-Agent: Roundcube Webmail/1.0.4 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 May 2015 15:22:33 -0000 Alan, Thank you for the information. So SGPIO is out of band. So the intent of Marvel chip is not for in-band SES-2? I see these events, one for each LSI card in dmesg: mps1: EventReply : EventDataLength: 5 AckRequired: 0 Event: SasEnclDeviceStatusChange (0x1d) <---- EventContext: 0x0 EnclosureHandle: 0x1 ReasonCode: Added PhysicalPort: 0 NumSlots: 8 <---- StartSlot: 0 PhyBits: 0x0 The motherboard is not in the same cabinet as the backplane, and and I would rather not just replace the backplane with one that had an expander on it $$$. Maybe I'm just seeking functionality that is not present but the fact that the backplane manual cleary states "The MG9072 is an enclosure management chip that supports the SES-2 controller and SES-2 protocols." make me -want- to believe something else is amiss. Thank you. --mikej On 2015-05-28 10:55, Alan Somers wrote: > SGPIO is an out-of-band protocol. It doesn't use the same wires as > the SAS connections. It's odd that the manual says the enclosure > management chip supports SES-2 and yet there is no SAS port for it. > The chassis manual also shows that headers 7-9 are for SGPIO. Are > they connected to anything? Some Supermicro motherboards have SGPIO > headers. But I would think that if yours did, then the ses device > would show up even if the cable were unplugged. LSI HBAs are also > supposed to have builtin SGPIO controllers. But I don't know how to > enable them or if FreeBSD has driver support for them. You might > learn more by asking LSI support than SuperMicro support. > > If you do have an SGPIO controller on the motherboard, you will need > FreeBSD 10.0 or later to use it. > > Finally, I know you probably don't want to spend more money, but if > you want enclosure management then your best option is to replace the > backplane with a SAS-846EL. It should be mechanically compatible. > That will give you SES, which is way better than SGPIO. It also has > an expander, so you'll only need one HBA. > > -Alan > > On Thu, May 28, 2015 at 7:45 AM, Michael Jung wrote: >> I hoping to get some direction as I simply am unsure what to do next. >> >> I have three LSI 9200-8e adapters connected to a Super Micro >> CSE-846A-R1200B >> yet no SES devices get created. The Backplane in the chassis >> BPN-SAS_846A >> does not have a expander. So I have six SFF-8088 on the backplane >> that >> connect >> directly to the ports on the 9200-8e albeit externally.The backplane >> does >> have >> a Marvel MG9072 supporting SES-2 and is jumpered for SGPIO. >> >> The server in question is running 9.3, but I have tested with >> 10.1-STABLE >> and >> 11-Current live CD's and I do not see /dev/ses created. I have also >> booted >> for testing openIndiana and do not see /dev/es/ses created. I >> have an open ticket with SM but that so far has them asking a lot of >> questions >> and giving no direction. >> >> On the LSI cards I have tested P16, P17 and P19 all in IT mode on 9.3. >> sas2icru on each of the adpapters does return output such as >> >> Enclosure information >> ------------------------------------------------------------------------ >> Enclosure# : 1 >> Logical ID : 500605b0:065e26b0 >> Numslots : 8 >> StartSlot : 0 >> ------------------------------------------------------------------------ >> >> What stands out is is not identefied for 'mps0' in >> dmesg. >> However, >> drives attached to all controllers do work fine and are identified. >> >> Full dmesg and output of sas2icru for each controller >> https://charon.gopai.com/dmesg >> >> Booting 9.3 verbosely with hw.mps.debug_level=0xd in /boot/loader.conf >> yeilds: >> >> mps1: port 0xcc00-0xccff mem >> 0xde8bc000-0xde8bffff,0xde8c0000-0xde8fffff irq 35 at device 0.0 on >> pci19 >> mps1: Firmware: 16.00.00.00, Driver: 16.00.00.00-fbsd >> mps1: IOCCapabilities: >> 1285c >> mps1: EventReply : >> mps1: EventReply : >> mps1: attempting to allocate 1 MSI-X vectors (15 supported) >> mps1: using IRQ 272 for MSI-X >> mps2: port 0xdc00-0xdcff mem >> 0xdeabc000-0xdeabffff,0xdeac0000-0xdeafffff irq 33 at device 0.0 on >> pci18 >> mps2: Firmware: 16.00.00.00, Driver: 16.00.00.00-fbsd >> mps2: IOCCapabilities: >> 1285c >> mps2: EventReply : >> mps2: EventReply : >> mps2: attempting to allocate 1 MSI-X vectors (15 supported) >> mps2: using IRQ 273 for MSI-X >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps0: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps1: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps2: EventReply : >> mps0: (0)->(mpssas_fw_work) Working on Event: [16] >> mps0: (1)->(mpssas_fw_work) Event Free: [16] >> mps0: (1)->(mpssas_fw_work) Working on Event: [1d] >> mps0: (2)->(mpssas_fw_work) Event Free: [1d] >> mps0: (2)->(mpssas_fw_work) Working on Event: [1c] >> mps1: (3)->(mpssas_fw_work) Working on Event: [16] >> mps1: (4)->(mpssas_fw_work) Event Free: [16] >> mps1: (4)->(mpssas_fw_work) Working on Event: [1d] >> mps1: (5)->(mpssas_fw_work) Event Free: [1d] >> mps1: (5)->(mpssas_fw_work) Working on Event: [1c] >> mps2: (6)->(mpssas_fw_work) Working on Event: [16] >> mps2: (7)->(mpssas_fw_work) Event Free: [16] >> mps2: (7)->(mpssas_fw_work) Working on Event: [1d] >> mps2: (8)->(mpssas_fw_work) Event Free: [1d] >> mps2: (8)->(mpssas_fw_work) Working on Event: [1c] >> (probe0:mps1:0:0:0): Down reving Protocol Version from 4 to 0? >> (probe1:mps1:0:1:0): Down reving Protocol Version from 4 to 0? >> (probe2:mps1:0:2:0): Down reving Protocol Version from 4 to 0? >> (probe3:mps1:0:4:0): Down reving Protocol Version from 4 to 0? >> (probe4:mps2:0:0:0): Down reving Protocol Version from 4 to 0? >> (probe5:mps2:0:1:0): Down reving Protocol Version from 4 to 0? >> mps1: (9)->(mpssas_fw_work) Event Free: [1c] >> mps1: (9)->(mpssas_fw_work) Working on Event: [16] >> mps1: (10)->(mpssas_fw_work) Event Free: [16] >> mps1: (10)->(mpssas_fw_work) Working on Event: [16] >> mps1: (11)->(mpssas_fw_work) Event Free: [16] >> mps1: (11)->(mpssas_fw_work) Working on Event: [16] >> mps1: (12)->(mpssas_fw_work) Event Free: [16] >> mps1: (12)->(mpssas_fw_work) Working on Event: [16] >> mps1: (13)->(mpssas_fw_work) Event Free: [16] >> mps1: (13)->(mpssas_fw_work) Working on Event: [16] >> mps1: (14)->(mpssas_fw_work) Event Free: [16] >> mps1: (14)->(mpssas_fw_work) Working on Event: [16] >> mps1: (15)->(mpssas_fw_work) Event Free: [16] >> mps1: (15)->(mpssas_fw_work) Working on Event: [16] >> mps1: (16)->(mpssas_fw_work) Event Free: [16] >> mps1: (16)->(mpssas_fw_work) Working on Event: [16] >> mps1: (17)->(mpssas_fw_work) Event Free: [16] >> mps1: (17)->(mpssas_fw_work) Working on Event: [16] >> mps1: (18)->(mpssas_fw_work) Event Free: [16] >> (probe6:mps1:0:5:0): Down reving Protocol Version from 4 to 0? >> (probe7:mps2:0:2:0): Down reving Protocol Version from 4 to 0? >> (probe8:mps2:0:6:0): Down reving Protocol Version from 4 to 0? >> (probe9:mps2:0:4:0): Down reving Protocol Version from 4 to 0? >> mps2: (18)->(mpssas_fw_work) Event Free: [1c] >> mps2: (18)->(mpssas_fw_work) Working on Event: [16] >> mps2: (19)->(mpssas_fw_work) Event Free: [16] >> mps2: (19)->(mpssas_fw_work) Working on Event: [16] >> mps2: (20)->(mpssas_fw_work) Event Free: [16] >> mps2: (20)->(mpssas_fw_work) Working on Event: [16] >> mps2: (21)->(mpssas_fw_work) Event Free: [16] >> mps2: (21)->(mpssas_fw_work) Working on Event: [16] >> mps2: (22)->(mpssas_fw_work) Event Free: [16] >> mps2: (22)->(mpssas_fw_work) Working on Event: [16] >> mps2: (23)->(mpssas_fw_work) Event Free: [16] >> mps2: (23)->(mpssas_fw_work) Working on Event: [16] >> mps2: (24)->(mpssas_fw_work) Event Free: [16] >> mps2: (24)->(mpssas_fw_work) Working on Event: [16] >> mps2: (25)->(mpssas_fw_work) Event Free: [16] >> mps2: (25)->(mpssas_fw_work) Working on Event: [16] >> mps2: (26)->(mpssas_fw_work) Event Free: [16] >> mps2: (26)->(mpssas_fw_work) Working on Event: [16] >> mps2: (27)->(mpssas_fw_work) Event Free: [16] >> mps2: (27)->(mpssas_fw_work) Working on Event: [16] >> mps2: (28)->(mpssas_fw_work) Event Free: [16] >> mps2: (28)->(mpssas_fw_work) Working on Event: [16] >> mps2: (29)->(mpssas_fw_work) Event Free: [16] >> (probe0:mps2:0:5:0): Down reving Protocol Version from 4 to 0? >> (probe1:mps0:0:0:0): Down reving Protocol Version from 4 to 0? >> (probe2:mps0:0:1:0): Down reving Protocol Version from 4 to 0? >> (probe3:mps0:0:2:0): Down reving Protocol Version from 4 to 0? >> (probe5:mps0:0:3:0): Down reving Protocol Version from 4 to 0? >> (probe6:mps0:0:4:0): Down reving Protocol Version from 4 to 0? >> mps0: (29)->(mpssas_fw_work) Event Free: [1c] >> mps0: (29)->(mpssas_fw_work) Working on Event: [16] >> mps0: (30)->(mpssas_fw_work) Event Free: [16] >> mps0: (30)->(mpssas_fw_work) Working on Event: [16] >> mps0: (31)->(mpssas_fw_work) Event Free: [16] >> mps0: (31)->(mpssas_fw_work) Working on Event: [16] >> mps0: (32)->(mpssas_fw_work) Event Free: [16] >> mps0: (32)->(mpssas_fw_work) Working on Event: [16] >> mps0: (33)->(mpssas_fw_work) Event Free: [16] >> mps0: (33)->(mpssas_fw_work) Working on Event: [16] >> mps0: (34)->(mpssas_fw_work) Event Free: [16] >> mps0: (34)->(mpssas_fw_work) Working on Event: [16] >> mps0: (35)->(mpssas_fw_work) Event Free: [16] >> mps0: (35)->(mpssas_fw_work) Working on Event: [16] >> mps0: (36)->(mpssas_fw_work) Event Free: [16] >> mps0: (36)->(mpssas_fw_work) Working on Event: [16] >> mps0: (37)->(mpssas_fw_work) Event Free: [16] >> mps0: (37)->(mpssas_fw_work) Working on Event: [16] >> mps0: (38)->(mpssas_fw_work) Event Free: [16] >> mps0: (38)->(mpssas_fw_work) Working on Event: [16] >> mps0: (39)->(mpssas_fw_work) Event Free: [16] >> mps0: (39)->(mpssas_fw_work) Working on Event: [16] >> mps0: (40)->(mpssas_fw_work) Event Free: [16] >> (probe0:mps0:0:6:0): Down reving Protocol Version from 4 to 0? >> da10 at mps0 bus 0 scbus0 target 0 lun 0 >> da0 at mps0 bus 0 scbus0 target 2 lun 0 >> da1 at mps0 bus 0 scbus0 target 3 lun 0 >> da11 at mps0 bus 0 scbus0 target 1 lun 0 >> da2 at mps0 bus 0 scbus0 target 4 lun 0 >> da18 at mps1 bus 0 scbus1 target 0 lun 0 >> da19 at mps1 bus 0 scbus1 target 1 lun 0 >> da20 at mps1 bus 0 scbus1 target 2 lun 0 >> da23 at mps1 bus 0 scbus1 target 5 lun 0 >> da14 at mps2 bus 0 scbus2 target 0 lun 0 >> da15 at mps2 bus 0 scbus2 target 1 lun 0 >> da16 at mps2 bus 0 scbus2 target 2 lun 0 >> da22 at mps1 bus 0 scbus1 target 4 lun 0 >> da6 at mps2 bus 0 scbus2 target 4 lun 0 >> pass0 at mps0 bus 0 scbus0 target 0 lun 0 >> da8 at mps2 bus 0 scbus2 target 6 lun 0 >> pass1 at mps0 bus 0 scbus0 target 1 lun 0 >> pass2 at mps0 bus 0 scbus0 target 2 lun 0 >> pass3 at mps0 bus 0 scbus0 target 3 lun 0 >> pass4 at mps0 bus 0 scbus0 target 4 lun 0 >> pass5 at mps0 bus 0 scbus0 target 6 lun 0 >> pass6 at mps1 bus 0 scbus1 target 0 lun 0 >> pass7 at mps1 bus 0 scbus1 target 1 lun 0 >> da7 at mps2 bus 0 scbus2 target 5 lun 0 >> pass8 at mps1 bus 0 scbus1 target 2 lun 0 >> pass9 at mps1 bus 0 scbus1 target 4 lun 0 >> pass10 at mps1 bus 0 scbus1 target 5 lun 0 >> pass11 at mps2 bus 0 scbus2 target 0 lun 0 >> pass12 at mps2 bus 0 scbus2 target 1 lun 0 >> pass13 at mps2 bus 0 scbus2 target 2 lun 0 >> pass14 at mps2 bus 0 scbus2 target 4 lun 0 >> pass15 at mps2 bus 0 scbus2 target 5 lun 0 >> pass16 at mps2 bus 0 scbus2 target 6 lun 0 >> da4 at mps0 bus 0 scbus0 target 6 lun 0 >> >> I admit I am new to enclosure management - What am I missing? >> >> Thank you. >> >> --mikej >> _______________________________________________ >> freebsd-scsi@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi >> To unsubscribe, send any mail to >> "freebsd-scsi-unsubscribe@freebsd.org"