Date: Thu, 28 May 2015 11:22:29 -0400 From: Michael Jung <mikej@mikej.com> To: Alan Somers <asomers@freebsd.org> Cc: FreeBSD-scsi <freebsd-scsi@freebsd.org>, asomers@gmail.com Subject: Re: /dev/ses<x> not created Message-ID: <026292acf4288dbde00ce9bcb8686490@mail.mikej.com> In-Reply-To: <CAOtMX2jKOuUAFF2FFJXJzWDX-mMac=TGUTzwTjjVbjT-NtAmYw@mail.gmail.com> References: <76e9e3f41990026290382e5fa9c9b8d3@mail.mikej.com> <CAOtMX2jKOuUAFF2FFJXJzWDX-mMac=TGUTzwTjjVbjT-NtAmYw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Alan,
Thank you for the information.
So SGPIO is out of band.
So the intent of Marvel chip is not for in-band SES-2?
I see these events, one for each LSI card in dmesg:
mps1: EventReply :
EventDataLength: 5
AckRequired: 0
Event: SasEnclDeviceStatusChange (0x1d) <----
EventContext: 0x0
EnclosureHandle: 0x1
ReasonCode: Added
PhysicalPort: 0
NumSlots: 8 <----
StartSlot: 0
PhyBits: 0x0
The motherboard is not in the same cabinet as the backplane,
and and I would rather not just replace the backplane with
one that had an expander on it $$$.
Maybe I'm just seeking functionality that is not present but
the fact that the backplane manual cleary states "The MG9072
is an enclosure management chip that supports the SES-2 controller
and SES-2 protocols." make me -want- to believe something else
is amiss.
Thank you.
--mikej
On 2015-05-28 10:55, Alan Somers wrote:
> SGPIO is an out-of-band protocol. It doesn't use the same wires as
> the SAS connections. It's odd that the manual says the enclosure
> management chip supports SES-2 and yet there is no SAS port for it.
> The chassis manual also shows that headers 7-9 are for SGPIO. Are
> they connected to anything? Some Supermicro motherboards have SGPIO
> headers. But I would think that if yours did, then the ses device
> would show up even if the cable were unplugged. LSI HBAs are also
> supposed to have builtin SGPIO controllers. But I don't know how to
> enable them or if FreeBSD has driver support for them. You might
> learn more by asking LSI support than SuperMicro support.
>
> If you do have an SGPIO controller on the motherboard, you will need
> FreeBSD 10.0 or later to use it.
>
> Finally, I know you probably don't want to spend more money, but if
> you want enclosure management then your best option is to replace the
> backplane with a SAS-846EL. It should be mechanically compatible.
> That will give you SES, which is way better than SGPIO. It also has
> an expander, so you'll only need one HBA.
>
> -Alan
>
> On Thu, May 28, 2015 at 7:45 AM, Michael Jung <mikej@mikej.com> wrote:
>> I hoping to get some direction as I simply am unsure what to do next.
>>
>> I have three LSI 9200-8e adapters connected to a Super Micro
>> CSE-846A-R1200B
>> yet no SES devices get created. The Backplane in the chassis
>> BPN-SAS_846A
>> does not have a expander. So I have six SFF-8088 on the backplane
>> that
>> connect
>> directly to the ports on the 9200-8e albeit externally.The backplane
>> does
>> have
>> a Marvel MG9072 supporting SES-2 and is jumpered for SGPIO.
>>
>> The server in question is running 9.3, but I have tested with
>> 10.1-STABLE
>> and
>> 11-Current live CD's and I do not see /dev/ses<x> created. I have also
>> booted
>> for testing openIndiana and do not see /dev/es/ses<x> created. I
>> have an open ticket with SM but that so far has them asking a lot of
>> questions
>> and giving no direction.
>>
>> On the LSI cards I have tested P16, P17 and P19 all in IT mode on 9.3.
>> sas2icru on each of the adpapters does return output such as
>>
>> Enclosure information
>> ------------------------------------------------------------------------
>> Enclosure# : 1
>> Logical ID : 500605b0:065e26b0
>> Numslots : 8
>> StartSlot : 0
>> ------------------------------------------------------------------------
>>
>> What stands out is <LSI SAS2008> is not identefied for 'mps0' in
>> dmesg.
>> However,
>> drives attached to all controllers do work fine and are identified.
>>
>> Full dmesg and output of sas2icru for each controller
>> https://charon.gopai.com/dmesg
>>
>> Booting 9.3 verbosely with hw.mps.debug_level=0xd in /boot/loader.conf
>> yeilds:
>>
>> mps1: <LSI SAS2008> port 0xcc00-0xccff mem
>> 0xde8bc000-0xde8bffff,0xde8c0000-0xde8fffff irq 35 at device 0.0 on
>> pci19
>> mps1: Firmware: 16.00.00.00, Driver: 16.00.00.00-fbsd
>> mps1: IOCCapabilities:
>> 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: attempting to allocate 1 MSI-X vectors (15 supported)
>> mps1: using IRQ 272 for MSI-X
>> mps2: <LSI SAS2008> port 0xdc00-0xdcff mem
>> 0xdeabc000-0xdeabffff,0xdeac0000-0xdeafffff irq 33 at device 0.0 on
>> pci18
>> mps2: Firmware: 16.00.00.00, Driver: 16.00.00.00-fbsd
>> mps2: IOCCapabilities:
>> 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: attempting to allocate 1 MSI-X vectors (15 supported)
>> mps2: using IRQ 273 for MSI-X
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps0: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps1: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps2: EventReply :
>> mps0: (0)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (1)->(mpssas_fw_work) Event Free: [16]
>> mps0: (1)->(mpssas_fw_work) Working on Event: [1d]
>> mps0: (2)->(mpssas_fw_work) Event Free: [1d]
>> mps0: (2)->(mpssas_fw_work) Working on Event: [1c]
>> mps1: (3)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (4)->(mpssas_fw_work) Event Free: [16]
>> mps1: (4)->(mpssas_fw_work) Working on Event: [1d]
>> mps1: (5)->(mpssas_fw_work) Event Free: [1d]
>> mps1: (5)->(mpssas_fw_work) Working on Event: [1c]
>> mps2: (6)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (7)->(mpssas_fw_work) Event Free: [16]
>> mps2: (7)->(mpssas_fw_work) Working on Event: [1d]
>> mps2: (8)->(mpssas_fw_work) Event Free: [1d]
>> mps2: (8)->(mpssas_fw_work) Working on Event: [1c]
>> (probe0:mps1:0:0:0): Down reving Protocol Version from 4 to 0?
>> (probe1:mps1:0:1:0): Down reving Protocol Version from 4 to 0?
>> (probe2:mps1:0:2:0): Down reving Protocol Version from 4 to 0?
>> (probe3:mps1:0:4:0): Down reving Protocol Version from 4 to 0?
>> (probe4:mps2:0:0:0): Down reving Protocol Version from 4 to 0?
>> (probe5:mps2:0:1:0): Down reving Protocol Version from 4 to 0?
>> mps1: (9)->(mpssas_fw_work) Event Free: [1c]
>> mps1: (9)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (10)->(mpssas_fw_work) Event Free: [16]
>> mps1: (10)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (11)->(mpssas_fw_work) Event Free: [16]
>> mps1: (11)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (12)->(mpssas_fw_work) Event Free: [16]
>> mps1: (12)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (13)->(mpssas_fw_work) Event Free: [16]
>> mps1: (13)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (14)->(mpssas_fw_work) Event Free: [16]
>> mps1: (14)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (15)->(mpssas_fw_work) Event Free: [16]
>> mps1: (15)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (16)->(mpssas_fw_work) Event Free: [16]
>> mps1: (16)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (17)->(mpssas_fw_work) Event Free: [16]
>> mps1: (17)->(mpssas_fw_work) Working on Event: [16]
>> mps1: (18)->(mpssas_fw_work) Event Free: [16]
>> (probe6:mps1:0:5:0): Down reving Protocol Version from 4 to 0?
>> (probe7:mps2:0:2:0): Down reving Protocol Version from 4 to 0?
>> (probe8:mps2:0:6:0): Down reving Protocol Version from 4 to 0?
>> (probe9:mps2:0:4:0): Down reving Protocol Version from 4 to 0?
>> mps2: (18)->(mpssas_fw_work) Event Free: [1c]
>> mps2: (18)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (19)->(mpssas_fw_work) Event Free: [16]
>> mps2: (19)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (20)->(mpssas_fw_work) Event Free: [16]
>> mps2: (20)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (21)->(mpssas_fw_work) Event Free: [16]
>> mps2: (21)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (22)->(mpssas_fw_work) Event Free: [16]
>> mps2: (22)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (23)->(mpssas_fw_work) Event Free: [16]
>> mps2: (23)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (24)->(mpssas_fw_work) Event Free: [16]
>> mps2: (24)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (25)->(mpssas_fw_work) Event Free: [16]
>> mps2: (25)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (26)->(mpssas_fw_work) Event Free: [16]
>> mps2: (26)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (27)->(mpssas_fw_work) Event Free: [16]
>> mps2: (27)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (28)->(mpssas_fw_work) Event Free: [16]
>> mps2: (28)->(mpssas_fw_work) Working on Event: [16]
>> mps2: (29)->(mpssas_fw_work) Event Free: [16]
>> (probe0:mps2:0:5:0): Down reving Protocol Version from 4 to 0?
>> (probe1:mps0:0:0:0): Down reving Protocol Version from 4 to 0?
>> (probe2:mps0:0:1:0): Down reving Protocol Version from 4 to 0?
>> (probe3:mps0:0:2:0): Down reving Protocol Version from 4 to 0?
>> (probe5:mps0:0:3:0): Down reving Protocol Version from 4 to 0?
>> (probe6:mps0:0:4:0): Down reving Protocol Version from 4 to 0?
>> mps0: (29)->(mpssas_fw_work) Event Free: [1c]
>> mps0: (29)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (30)->(mpssas_fw_work) Event Free: [16]
>> mps0: (30)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (31)->(mpssas_fw_work) Event Free: [16]
>> mps0: (31)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (32)->(mpssas_fw_work) Event Free: [16]
>> mps0: (32)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (33)->(mpssas_fw_work) Event Free: [16]
>> mps0: (33)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (34)->(mpssas_fw_work) Event Free: [16]
>> mps0: (34)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (35)->(mpssas_fw_work) Event Free: [16]
>> mps0: (35)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (36)->(mpssas_fw_work) Event Free: [16]
>> mps0: (36)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (37)->(mpssas_fw_work) Event Free: [16]
>> mps0: (37)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (38)->(mpssas_fw_work) Event Free: [16]
>> mps0: (38)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (39)->(mpssas_fw_work) Event Free: [16]
>> mps0: (39)->(mpssas_fw_work) Working on Event: [16]
>> mps0: (40)->(mpssas_fw_work) Event Free: [16]
>> (probe0:mps0:0:6:0): Down reving Protocol Version from 4 to 0?
>> da10 at mps0 bus 0 scbus0 target 0 lun 0
>> da0 at mps0 bus 0 scbus0 target 2 lun 0
>> da1 at mps0 bus 0 scbus0 target 3 lun 0
>> da11 at mps0 bus 0 scbus0 target 1 lun 0
>> da2 at mps0 bus 0 scbus0 target 4 lun 0
>> da18 at mps1 bus 0 scbus1 target 0 lun 0
>> da19 at mps1 bus 0 scbus1 target 1 lun 0
>> da20 at mps1 bus 0 scbus1 target 2 lun 0
>> da23 at mps1 bus 0 scbus1 target 5 lun 0
>> da14 at mps2 bus 0 scbus2 target 0 lun 0
>> da15 at mps2 bus 0 scbus2 target 1 lun 0
>> da16 at mps2 bus 0 scbus2 target 2 lun 0
>> da22 at mps1 bus 0 scbus1 target 4 lun 0
>> da6 at mps2 bus 0 scbus2 target 4 lun 0
>> pass0 at mps0 bus 0 scbus0 target 0 lun 0
>> da8 at mps2 bus 0 scbus2 target 6 lun 0
>> pass1 at mps0 bus 0 scbus0 target 1 lun 0
>> pass2 at mps0 bus 0 scbus0 target 2 lun 0
>> pass3 at mps0 bus 0 scbus0 target 3 lun 0
>> pass4 at mps0 bus 0 scbus0 target 4 lun 0
>> pass5 at mps0 bus 0 scbus0 target 6 lun 0
>> pass6 at mps1 bus 0 scbus1 target 0 lun 0
>> pass7 at mps1 bus 0 scbus1 target 1 lun 0
>> da7 at mps2 bus 0 scbus2 target 5 lun 0
>> pass8 at mps1 bus 0 scbus1 target 2 lun 0
>> pass9 at mps1 bus 0 scbus1 target 4 lun 0
>> pass10 at mps1 bus 0 scbus1 target 5 lun 0
>> pass11 at mps2 bus 0 scbus2 target 0 lun 0
>> pass12 at mps2 bus 0 scbus2 target 1 lun 0
>> pass13 at mps2 bus 0 scbus2 target 2 lun 0
>> pass14 at mps2 bus 0 scbus2 target 4 lun 0
>> pass15 at mps2 bus 0 scbus2 target 5 lun 0
>> pass16 at mps2 bus 0 scbus2 target 6 lun 0
>> da4 at mps0 bus 0 scbus0 target 6 lun 0
>>
>> I admit I am new to enclosure management - What am I missing?
>>
>> Thank you.
>>
>> --mikej
>> _______________________________________________
>> freebsd-scsi@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi
>> To unsubscribe, send any mail to
>> "freebsd-scsi-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?026292acf4288dbde00ce9bcb8686490>