Date: Tue, 30 May 2017 15:09:15 -0500 From: Dustin Wenz <dustinwenz@ebureau.com> To: Alan Somers <asomers@freebsd.org> Cc: FreeBSD-scsi <freebsd-scsi@freebsd.org> Subject: Re: Inferring SAS expander topology Message-ID: <5124123C-B996-4DC2-A338-64ACC33A5C09@ebureau.com> In-Reply-To: <CAOtMX2h-E_GG9wopPH5Zz2jTZ%2BrqBmyJRSVvU9wxr4fczoHBiA@mail.gmail.com> References: <18E31C87-AB53-491F-9E40-F496AE31E305@ebureau.com> <CAOtMX2h-E_GG9wopPH5Zz2jTZ%2BrqBmyJRSVvU9wxr4fczoHBiA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks! It looks like smp_discover will do what I need, though is not = particularly user-friendly for this specific application.=20 I needed to use both "smp_discover sesX" in addition to "smp_discover = --phy=3DY --brief sesX" in order to correlate SAS interfaces with the = addresses of devices attached to them. - .Dustin > On May 30, 2017, at 12:31 PM, Alan Somers <asomers@freebsd.org> wrote: >=20 > On Tue, May 30, 2017 at 11:00 AM, Dustin Wenz <dustinwenz@ebureau.com> = wrote: >> I'm working on a server that has 6 LSI SAS expanders connected to it. = Two of them are throwing intermittent command errors and need some = attention. Because the host only has four external 8088 ports, I know = some of the expanders are chained through each other. >>=20 >> Before I can send parts for replacement, I need to determine if the = two problematic expanders are either daisy-chained or directly connected = to the host. On Linux, this is trivial; I would just browse the device = topology in /sys/. However, this machine is running FreeBSD 10.3, and I = am unable to find a way to do this. I have sg3_utils available, as well = as any build-in FreeBSD tools. Using this software, is it possible to = determine if a SAS expander is chained off of another? >>=20 >> - .Dustin >=20 > I don't think sg3_utils will help you. You want sysutils/smp_utils > instead. If you install that and then run "smp_discover /dev/ses0", > it will show you what each phy is connected to. Usually, there will > be four phys connected to the upstream port. You can tell by their > EUI64s whether they're connected directly to the HBA or to another > expander. And you can tell which expander by comparing the exact > EUI64 to each other expander's SEP phy. Note that with LSI expanders, > the SEP's address usually differs from the expander's address by a few > bits in the last byte. For example, if the expander's address is > 0x50000000000000ff, then the SEP's address might by > 0x50000000000000ffd. Here's some example output from one of my > systems: >=20 > # smp_discover /dev/ses3 > phy 0:U:attached:[500093d23000a000:00 t(SATA)] 6 Gbps > phy 1:U:attached:[500093d23000a001:00 t(SATA)] 6 Gbps > phy 2:U:attached:[500093d23000a002:00 t(SATA)] 6 Gbps > phy 3:U:attached:[500093d23000a003:00 t(SATA)] 6 Gbps > phy 4:U:attached:[500093d23000a17f:11 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 5:U:attached:[500093d23000a17f:10 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 6:U:attached:[500093d23000a17f:09 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 7:U:attached:[500093d23000a17f:08 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 8:U:attached:[500093d23000a1bf:11 exp t(SMP)] 6 Gbps > <- Connected to a third expander > phy 9:U:attached:[500093d23000a1bf:10 exp t(SMP)] 6 Gbps > <- Connected to a third expander > phy 10:U:attached:[500093d23000a1bf:09 exp t(SMP)] 6 Gbps > <- Connected to a third expander > phy 11:U:attached:[500093d23000a1bf:08 exp t(SMP)] 6 Gbps > <- Connected to a third expander > phy 16:U:attached:[500093d23000a010:00 t(SATA)] 6 Gbps > phy 17:U:attached:[500093d23000a011:00 t(SATA)] 6 Gbps > phy 18:U:attached:[500093d23000a012:00 t(SATA)] 6 Gbps > phy 19:U:attached:[500093d23000a013:00 t(SATA)] 6 Gbps > phy 20:U:attached:[500093d23000a014:00 t(SATA)] 6 Gbps > phy 21:U:attached:[500093d23000a015:00 t(SATA)] 6 Gbps > phy 22:U:attached:[500093d23000a016:00 t(SATA)] 6 Gbps > phy 23:U:attached:[500093d23000a017:00 t(SATA)] 6 Gbps > phy 25:U:attached:[500093d23000a019:00 t(SATA)] 6 Gbps > phy 26:U:attached:[500093d23000a01a:00 t(SATA)] 6 Gbps > phy 29:U:attached:[500093d23000a01d:00 t(SATA)] 6 Gbps > phy 30:U:attached:[500093d23000a01e:00 t(SATA)] 6 Gbps > phy 31:U:attached:[500093d23000a01f:00 t(SATA)] 6 Gbps > phy 33:U:attached:[500093d23000a021:00 t(SATA)] 6 Gbps > phy 34:U:attached:[500093d23000a022:00 t(SATA)] 6 Gbps > phy 35:U:attached:[500093d23000a023:00 t(SATA)] 6 Gbps > phy 36:U:attached:[500093d23000a024:00 t(SATA)] 6 Gbps > phy 37:U:attached:[500093d23000a025:00 t(SATA)] 6 Gbps > phy 38:U:attached:[500093d23000a026:00 t(SATA)] 6 Gbps > phy 39:U:attached:[500093d23000a027:00 t(SATA)] 6 Gbps > phy 40:U:attached:[500605b008a93990:04 i(SSP+STP+SMP)] 6 Gbps > <- Connected to an LSI HBA > phy 41:U:attached:[500605b008a93990:07 i(SSP+STP+SMP)] 6 Gbps > <- Connected to an LSI HBA > phy 42:U:attached:[500605b008a93990:05 i(SSP+STP+SMP)] 6 Gbps > <- Connected to an LSI HBA > phy 43:U:attached:[500605b008a93990:06 i(SSP+STP+SMP)] 6 Gbps > <- Connected to an LSI HBA > phy 48:D:attached:[500093d23000a03d:00 V i(SMP) t(SSP)] 12 Gbps > <- This expander's SEP >=20 > # smp_discover /dev/pass121 > phy 0:U:attached:[500093d23000a140:00 t(SATA)] 6 Gbps > phy 1:U:attached:[500093d23000a141:00 t(SATA)] 6 Gbps > phy 2:U:attached:[500093d23000a142:00 t(SATA)] 6 Gbps > phy 3:U:attached:[500093d23000a143:00 t(SATA)] 6 Gbps > phy 4:U:attached:[500093d23000a144:00 t(SATA)] 6 Gbps > phy 5:U:attached:[500093d23000a145:00 t(SATA)] 6 Gbps > phy 6:U:attached:[500093d23000a146:00 t(SATA)] 6 Gbps > phy 7:U:attached:[500093d23000a147:00 t(SATA)] 6 Gbps > phy 8:U:attached:[500093d23000a03f:07 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 9:U:attached:[500093d23000a03f:06 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 10:U:attached:[500093d23000a03f:05 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 11:U:attached:[500093d23000a03f:04 exp t(SMP)] 6 Gbps > <- Connected to the other expander > phy 12:U:attached:[500093d23000a14c:00 t(SATA)] 6 Gbps > phy 14:U:attached:[500093d23000a14e:00 t(SATA)] 6 Gbps > phy 15:U:attached:[500093d23000a14f:00 t(SATA)] 6 Gbps > phy 16:U:attached:[500093d23000a150:00 t(SATA)] 6 Gbps > phy 17:U:attached:[500093d23000a151:00 t(SATA)] 6 Gbps > phy 18:U:attached:[500093d23000a152:00 t(SATA)] 6 Gbps > phy 19:U:attached:[500093d23000a153:00 t(SATA)] 6 Gbps > phy 20:U:attached:[500093d23000a154:00 t(SATA)] 6 Gbps > phy 21:U:attached:[500093d23000a155:00 t(SATA)] 6 Gbps > phy 22:U:attached:[500093d23000a156:00 t(SATA)] 6 Gbps > phy 23:U:attached:[500093d23000a157:00 t(SATA)] 6 Gbps > phy 24:U:attached:[500093d23000a158:00 t(SATA)] 6 Gbps > phy 25:U:attached:[500093d23000a159:00 t(SATA)] 6 Gbps > phy 26:U:attached:[500093d23000a15a:00 t(SATA)] 6 Gbps > phy 27:U:attached:[500093d23000a15b:00 t(SATA)] 6 Gbps > phy 28:U:attached:[500093d23000a15c:00 t(SATA)] 6 Gbps > phy 29:U:attached:[500093d23000a15d:00 t(SATA)] 6 Gbps > phy 30:U:attached:[500093d23000a15e:00 t(SATA)] 6 Gbps > phy 32:U:attached:[500093d23000a160:00 t(SATA)] 6 Gbps > phy 33:U:attached:[500093d23000a161:00 t(SATA)] 6 Gbps > phy 34:U:attached:[500093d23000a162:00 t(SATA)] 6 Gbps > phy 35:U:attached:[500093d23000a163:00 t(SATA)] 6 Gbps > phy 36:U:attached:[500093d23000a164:00 t(SATA)] 6 Gbps > phy 37:U:attached:[500093d23000a165:00 t(SATA)] 6 Gbps > phy 38:U:attached:[500093d23000a166:00 t(SATA)] 6 Gbps > phy 39:U:attached:[500093d23000a167:00 t(SATA)] 6 Gbps > phy 40:D:attached:[500093d23000a17d:00 V i(SMP) t(SSP)] 12 Gbps > <- This expander's SEP >=20 > -Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5124123C-B996-4DC2-A338-64ACC33A5C09>