Date: Fri, 10 Mar 2017 14:36:10 +0000 From: John <jwd@freebsd.org> To: Alan Somers <asomers@freebsd.org> Cc: Alexander Motin <mav@freebsd.org>, FreeBSD-SCSI <freebsd-scsi@freebsd.org> Subject: Re: sesutil not seeing disk in first bay Message-ID: <20170310143610.GA15302@FreeBSD.org> In-Reply-To: <CAOtMX2i=6CeqGyCZR9e5zwj036-RCPxboBz1nUE7ALoOPHCUDg@mail.gmail.com> References: <20170224102748.GA34792@FreeBSD.org> <0fec6ff8-ede3-d74d-0b81-6ed7e80dfc6d@FreeBSD.org> <20170224140002.GA77466@FreeBSD.org> <CAOtMX2i=6CeqGyCZR9e5zwj036-RCPxboBz1nUE7ALoOPHCUDg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Sorry for the slow response, system was being moved. ----- Alan Somers's Original Message ----- > Hard to say whether the problem is in sesutil or in your expander's > firmware. Could you please install sysutils/sg3_utils and run "sg_ses > -p2 /dev/ses0" and "sg_ses -p10 /dev/ses0" ? Requested outputs: sg_ses -p2 /dev/ses0 - https://people.freebsd.org/~jwd/ses0.sg_ses-p2.txt sg_ses -p10 /dev/ses0 - https://people.freebsd.org/~jwd/ses0.sg_ses-p10.txt For completeness: camcontrol smpphylist ses0 - https://people.freebsd.org/~jwd/ses0.camcontro= l-smpphylist.txt sesutil -u /dev/ses0 map - https://people.freebsd.org/~jwd/ses0.sesutil-map= =2Etxt Using a combination of smpphylist to get the phy & disk seems to always work in conjunction with sg_ses -p 10 to map the phy/disk to a slot number by matching the phy. sesutil seems like it's off by 1. Thanks, John > -Alan >=20 > On Fri, Feb 24, 2017 at 7:00 AM, John <jwd@freebsd.org> wrote: > > ----- Alexander Motin's Original Message ----- > >> Hi. > >> > >> I am not sure I completely understand from your description how exactly > >> detected disks match their enclosure slots, but make sure to take a lo= ok > >> on this change: > >> https://svnweb.freebsd.org/base?view=3Drevision&revision=3D308425 > > > > Yes - this patch is on my system. > > > > To phrase it differently - the shelf contains 25 drives. Disks da0 > > through da23 show up via sesutil: > > > > # sesutil map -u /dev/ses0 | grep da > > Device Names: da0,pass0 > > Device Names: da1,pass1 > > Device Names: da2,pass2 > > Device Names: da3,pass3 > > Device Names: da4,pass4 > > Device Names: da5,pass5 > > Device Names: da6,pass6 > > Device Names: da7,pass7 > > Device Names: da8,pass8 > > Device Names: da9,pass9 > > Device Names: da10,pass10 > > Device Names: da11,pass11 > > Device Names: da12,pass12 > > Device Names: da13,pass13 > > Device Names: da14,pass14 > > Device Names: da15,pass15 > > Device Names: da16,pass16 > > Device Names: da17,pass17 > > Device Names: da18,pass18 > > Device Names: da19,pass19 > > Device Names: da20,pass20 > > Device Names: da21,pass21 > > Device Names: da22,pass22 > > Device Names: da23,pass23 > > > > Disk da24 does not show up but is present in the first bay. da0 > > is in bay 2, etc. > > > > I hope this helps. > > > > Thanks, > > John > > > >> On 24.02.2017 12:27, John wrote: > >> > Hi Folks - > >> > > >> > Running fresh fbsd 11.0-STABLE #1 r314197: Fri Feb 24 05:05:34 ES= T 2017. > >> > > >> > I'm wondering if anyone using sesutil has seen the following issue > >> > with an HP D2700 shelf (25 x 600GB SAS drives): > >> > > >> > sesutil map -u /dev/ses0 > >> > ses0: > >> > Enclosure Name: HP D2700 SAS AJ941A 0149 > >> > Enclosure ID: 5001438023432c00 > >> > Element 0, Type: Array Device Slot > >> > Status: Unsupported (0x00 0x80 0x00 0x00) > >> > Element 1, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Element 2, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da0,pass0 > >> > Element 3, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da1,pass1 > >> > Element 4, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da2,pass2 > >> > ... > >> > ... full list below > >> > ... > >> > Element 25, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da23,pass23 > >> > > >> > > >> > Note Element 1 is an does not contain any device names. > >> > > >> > Sesutil is getting an ENODEV back from the ioctl requesting the name= s. > >> > > >> > In scsi_enc.c the error seems be coming from here: > >> > > >> > cam_periph_lock(periph); > >> > error =3D (*enc->enc_vec.get_elm_devnames)(enc, &elm= dn); > >> > cam_periph_unlock(periph); > >> > > >> > which eventually calls cam_periph_list() which "looks" ok to me. > >> > > >> > I'm wondering if there is some sort of off by 1 error somewhere in > >> > the code with respect to how this shelf reports devices and are > >> > assigned since da0 is not allocated to the first bay, but instead ba= y 2. > >> > > >> > Device da24 (the 25th device) is in the first bay. If I run a dd or > >> > iozone to da24 the drive works correctly and the activity light > >> > comes on as expected. > >> > > >> > This process works correctly on a set of supermicro shelves. > >> > > >> > Any thoughts or pointers? > >> > > >> > Thanks, > >> > John > >> > > >> > dmesg for da0/da1 > >> > > >> > (da0:mps0:0:8:0): UNMAPPED > >> > da0 at mps0 bus 0 scbus0 target 8 lun 0 > >> > da0: <HP EG0600FBDSR HPD6> Fixed Direct Access SPC-3 SCSI device > >> > da0: Serial Number EA01PC91LLBB1239 > >> > da0: 600.000MB/s transfers > >> > da0: Command Queueing enabled > >> > da0: 572325MB (1172123568 512 byte sectors) > >> > (da1:mps0:0:9:0): UNMAPPED > >> > da1 at mps0 bus 0 scbus0 target 9 lun 0 > >> > da1: <HP EG0600FBDSR HPD6> Fixed Direct Access SPC-3 SCSI device > >> > da1: Serial Number EA01PC91LPG91239 > >> > da1: 600.000MB/s transfers > >> > da1: Command Queueing enabled > >> > da1: 572325MB (1172123568 512 byte sectors) > >> > > >> > > >> > > >> > # sesutil map -u /dev/ses0 > >> > ses0: > >> > Enclosure Name: HP D2700 SAS AJ941A 0149 > >> > Enclosure ID: 5001438023432c00 > >> > Element 0, Type: Array Device Slot > >> > Status: Unsupported (0x00 0x80 0x00 0x00) > >> > Element 1, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Element 2, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da0,pass0 > >> > Element 3, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da1,pass1 > >> > Element 4, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da2,pass2 > >> > Element 5, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da3,pass3 > >> > Element 6, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da4,pass4 > >> > Element 7, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da5,pass5 > >> > Element 8, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da6,pass6 > >> > Element 9, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da7,pass7 > >> > Element 10, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da8,pass8 > >> > Element 11, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da9,pass9 > >> > Element 12, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da10,pass10 > >> > Element 13, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da11,pass11 > >> > Element 14, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da12,pass12 > >> > Element 15, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da13,pass13 > >> > Element 16, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da14,pass14 > >> > Element 17, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da15,pass15 > >> > Element 18, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da16,pass16 > >> > Element 19, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da17,pass17 > >> > Element 20, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da18,pass18 > >> > Element 21, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da19,pass19 > >> > Element 22, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da20,pass20 > >> > Element 23, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da21,pass21 > >> > Element 24, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da22,pass22 > >> > Element 25, Type: Array Device Slot > >> > Status: OK (0x01 0x80 0x00 0x00) > >> > Device Names: da23,pass23 > >> > Element 26, Type: SAS Connector > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 27, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 28, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 29, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 30, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 31, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 32, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 33, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 34, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 35, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 36, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 37, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 38, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 39, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 40, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 41, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 42, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 43, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 44, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 45, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 46, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 47, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 48, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 49, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 50, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 51, Type: SAS Connector > >> > Status: OK (0x01 0x20 0x00 0x00) > >> > Element 52, Type: <Type 0xff> > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 53, Type: <Type 0xff> > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Description: 5C7236P2L5 > >> > Element 54, Type: Power Supply > >> > Status: Unsupported (0x00 0x00 0x00 0x80) > >> > Element 55, Type: Power Supply > >> > Status: OK (0x01 0x00 0x00 0x80) > >> > Description: 20Delta 5ANLD0C4D3A0XN > >> > Element 56, Type: Power Supply > >> > Status: OK (0x01 0x00 0x00 0x80) > >> > Description: 20Delta 5ANLD0C4D3A0XS > >> > Element 57, Type: Cooling > >> > Status: OK (0x01 0x00 0x00 0x80) > >> > Description: B PAVCA0B9V3B0GH > >> > Extra status: > >> > - Speed: 0 rpm > >> > Element 58, Type: Cooling > >> > Status: Unsupported (0x00 0x02 0x8f 0x02) > >> > Extra status: > >> > - Speed: 6550 rpm > >> > Element 59, Type: Cooling > >> > Status: Unsupported (0x00 0x02 0x6b 0x02) > >> > Extra status: > >> > - Speed: 6190 rpm > >> > Element 60, Type: Cooling > >> > Status: OK (0x01 0x00 0x00 0x80) > >> > Description: B PAVCA0B9V3B0DL > >> > Extra status: > >> > - Speed: 0 rpm > >> > Element 61, Type: Cooling > >> > Status: Unsupported (0x00 0x02 0xa3 0x02) > >> > Extra status: > >> > - Speed: 6750 rpm > >> > Element 62, Type: Cooling > >> > Status: Unsupported (0x00 0x02 0x6b 0x02) > >> > Extra status: > >> > - Speed: 6190 rpm > >> > Element 63, Type: Enclosure Services Controller Electronics > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 64, Type: Enclosure Services Controller Electronics > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Description: 014901B0PAZAV0BTM3D3AX 0026 > >> > Element 65, Type: SAS Connector > >> > Status: Unsupported (0x00 0x02 0x00 0x00) > >> > Element 66, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x00 0x00) > >> > Element 67, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x01 0x00) > >> > Element 68, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x02 0x00) > >> > Element 69, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x03 0x00) > >> > Element 70, Type: SAS Connector > >> > Status: Unsupported (0x00 0x02 0x00 0x00) > >> > Element 71, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x00 0x00) > >> > Element 72, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x01 0x00) > >> > Element 73, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x02 0x00) > >> > Element 74, Type: SAS Connector > >> > Status: OK (0x01 0x02 0x03 0x00) > >> > Element 75, Type: SAS Expander > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 76, Type: SAS Expander > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Description: 900080050002 > >> > Element 77, Type: Display > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Description: 0.107SEGDISP C:03 > >> > Element 78, Type: Display > >> > Status: Unsupported (0x00 0x02 0x30 0x00) > >> > Element 79, Type: Display > >> > Status: Unsupported (0x00 0x02 0x31 0x00) > >> > Element 80, Type: Temperature Sensors > >> > Status: Unsupported (0x00 0x00 0x43 0x00) > >> > Extra status: > >> > - Temperature: 47 C > >> > Element 81, Type: Temperature Sensors > >> > Status: OK (0x01 0x00 0x43 0x00) > >> > Extra status: > >> > - Temperature: 47 C > >> > Element 82, Type: Enclosure Services Controller Electronics > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 83, Type: Enclosure Services Controller Electronics > >> > Status: Unknown (0x06 0x00 0x00 0x00) > >> > Description: 0149 0026 > >> > Element 84, Type: SAS Connector > >> > Status: Unsupported (0x00 0x02 0x00 0x00) > >> > Element 85, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x00 0x00) > >> > Element 86, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x01 0x00) > >> > Element 87, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x02 0x00) > >> > Element 88, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x03 0x00) > >> > Element 89, Type: SAS Connector > >> > Status: Unsupported (0x00 0x02 0x00 0x00) > >> > Element 90, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x00 0x00) > >> > Element 91, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x01 0x00) > >> > Element 92, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x02 0x00) > >> > Element 93, Type: SAS Connector > >> > Status: Unknown (0x06 0x02 0x03 0x00) > >> > Element 94, Type: SAS Expander > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 95, Type: SAS Expander > >> > Status: Unknown (0x06 0x00 0x00 0x00) > >> > Description: > >> > Element 96, Type: Display > >> > Status: Unknown (0x06 0x00 0x00 0x00) > >> > Description: > >> > Element 97, Type: Display > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 98, Type: Display > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 99, Type: Temperature Sensors > >> > Status: Unsupported (0x00 0x00 0x43 0x00) > >> > Extra status: > >> > - Temperature: 47 C > >> > Element 100, Type: Temperature Sensors > >> > Status: OK (0x01 0x00 0x43 0x00) > >> > Extra status: > >> > - Temperature: 47 C > >> > Element 101, Type: Enclosure Services Controller Electronics > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 102, Type: Enclosure Services Controller Electronics > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Description: 002803D0PARHV0DTM3B0FF > >> > Element 103, Type: Enclosure Services Controller Electronics > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 104, Type: Enclosure Services Controller Electronics > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Description: 1.02 > >> > Element 105, Type: Enclosure Services Controller Electronics > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Description: 1.02 > >> > Element 106, Type: Temperature Sensors > >> > Status: Unsupported (0x00 0x00 0x30 0x00) > >> > Extra status: > >> > - Temperature: 28 C > >> > Element 107, Type: Temperature Sensors > >> > Status: OK (0x01 0x00 0x31 0x00) > >> > Extra status: > >> > - Temperature: 29 C > >> > Element 108, Type: Temperature Sensors > >> > Status: OK (0x01 0x00 0x30 0x00) > >> > Extra status: > >> > - Temperature: 28 C > >> > Element 109, Type: Power Supply > >> > Status: Unsupported (0x00 0x00 0x00 0x00) > >> > Element 110, Type: Power Supply > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > Element 111, Type: Power Supply > >> > Status: OK (0x01 0x00 0x00 0x00) > >> > > >> > >> -- > >> Alexander Motin --FL5UXtIhxfXey3p5 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQF8BAEBCgBmBQJYwrnYXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQwNDBGOTgxNzM0NzQ3OEFBNDYyODNGQzVC NjI0OTlBMTQyNEY3RjgxAAoJELYkmaFCT3+B4fIIAIjdmBgVJPWfPeBNLenaNvw/ Fvwj++c/jD9V+ikZYZFUh9DaRL/QU265DJkxk4KcVcVGPRRFoj9t4eC8YjQpVuEY HpWGPxPqiOWZsFhj6NdnOoWS63fbprPdwDA37TSyrsVqToDNkvLnz1UiUuEQ+EZ3 vktNQcO07+fh+rEVAZufdozezRYQt/dLiP9Tm89dMwbYawyR5Km6bIVBjNVcn7DK SR0hp51tsbU1Gof2Qk3YgXMgSRm7bxVtvpMChHjAddgp9atuR4VD954fo9369vaS QHfX2mGMWQifuDaIP2rICuKFoM2ucg58Lw9wFuGMTvoIi/PG/Xp6+Pl/o54GsaU= =fSbj -----END PGP SIGNATURE----- --FL5UXtIhxfXey3p5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170310143610.GA15302>