Date: Mon, 23 Apr 2018 20:12:11 -0600 From: Alan Somers <asomers@freebsd.org> To: John <jwd@freebsd.org> Cc: SCSI List <freebsd-scsi@freebsd.org>, Alan Somers <asomers@freebsd.org> Subject: Re: /dev/enc@ SES doc? Message-ID: <CAOtMX2jKHjc%2B=0yJioFZWmpSnMZ%2B%2BkXqYnqm2o1sMzdMn%2B5Fkg@mail.gmail.com> In-Reply-To: <20180424013909.GA19649@FreeBSD.org> References: <20180413150704.GA77097@FreeBSD.org> <CAOtMX2hcSS5MwG1_iS05U1mNtGhopPiJkPorGfXfEpjC-ibWxQ@mail.gmail.com> <20180424013909.GA19649@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
If you have 3 shelves each with dual expanders, then you should have six
/dev/ses devices. The fact that you see 12 means that something is cabled
incorrectly. It sounds like each expander has two SAS ports (each of which
has 4 SAS lanes). Are there really only two LSI 9300-16e cards? It sounds
like there must be three. What you _should_ do is connect both of each
expanders' ports to the same HBA. That way it will form a wide port of 8
lanes. If you connect a single expander to two different HBAs, then the OS
sees them as separate expanders and doesn't realize that they're really the
same.
-Alan
On Mon, Apr 23, 2018 at 7:39 PM, John <jwd@freebsd.org> wrote:
> ----- Alan Somers's Original Message -----
> > The ses(4) man page describes the ioctls supported by the /dev/sesX
> nodes.
> > The /dev/enc@... nodes, however, are simply aliases for /dev/da* nodes
>
> I've been poking around in the code and am probably missing
> something obvious.
>
> I have 3 shelves, dual expanders each, 4 cables from a pair
> of LSI 9300-16e cards. I have one disk in each shelf, bay 1
> in the 1st shelf, bay 2 in the 2nd, bay 3 in the 3rd. One drive
> shows up on each cable/channel:
>
> WBN06J1R0000E8283APS - ses1-da0 ses9-da10 ses3-da4 ses7-da6
> WBN0414K0000E81627LM - ses11-da11 ses2-da2 ses5-da5 ses8-da8
> WBN05GME0000E8280DY5 - ses0-da1 ses4-da3 ses6-da7 ses10-da9
>
> And the readout of /dev/enc@* (trailing ses annotation added by hand):
>
> # find /dev/enc@* -name 'da*'
> /dev/enc@n500304801f25c63d/type@0/slot@3/elmdesc@Slot02/da3 (ses4)
> /dev/enc@n500304801f25c6bd/type@0/slot@3/elmdesc@Slot02/da1 (ses0)
> /dev/enc@n500304801f25c73d/type@0/slot@1/elmdesc@Slot00/da4 (ses3)
> /dev/enc@n500304801f25c7bd/type@0/slot@1/elmdesc@Slot00/da0 (ses1)
> /dev/enc@n500304801f25c83d/type@0/slot@2/elmdesc@Slot01/da5 (ses5)
> /dev/enc@n500304801f25c8bd/type@0/slot@2/elmdesc@Slot01/da2 (ses2)
>
> The 1st 6 ses units are shown - the 2nd 6 ses units are missing (ses6
> -> ses11).
>
> I think the following represents the issue, not sure how to fix it?
>
> # for i in /dev/ses*; do
> camcontrol smpphylist $i 2>/dev/null | grep ses
> done | awk '{print $2;}'
> 0x500304801f25c6bd
> 0x500304801f25c7bd
> 0x500304801f25c6bd
> 0x500304801f25c8bd
> 0x500304801f25c8bd
> 0x500304801f25c73d
> 0x500304801f25c63d
> 0x500304801f25c83d
> 0x500304801f25c63d
> 0x500304801f25c73d
> 0x500304801f25c83d
> 0x500304801f25c7bd
> # for i in /dev/ses*; do
> camcontrol smpphylist $i 2>/dev/null | grep ses
> done | awk '{print $2;}' | sort -u
> 0x500304801f25c63d
> 0x500304801f25c6bd
> 0x500304801f25c73d
> 0x500304801f25c7bd
> 0x500304801f25c83d
> 0x500304801f25c8bd
>
> The ses unit adds a level of uniqueness not provided by just the
> logical identifier?
>
> Thoughts?
>
> Thanks,
> John
>
> > On Fri, Apr 13, 2018 at 9:07 AM, John <jwd@freebsd.org> wrote:
> >
> > > Hi Folks,
> > >
> > > Just curious...
> > >
> > > Is there any doc for the /dev/enc@.... entries that map
> > > any attached SES enabled storage?
> > >
> > > cam/scsi/scsi_enc_ses.c does contain a good amount of info.
> > >
> > > Thanks!
> > > John
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2jKHjc%2B=0yJioFZWmpSnMZ%2B%2BkXqYnqm2o1sMzdMn%2B5Fkg>