Date: Tue, 16 Feb 2016 08:23:01 -0700 From: Alan Somers <asomers@freebsd.org> To: David Ford <david.ford@ouce.ox.ac.uk> Cc: "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org> Subject: Re: camcontrol sata affiliations Message-ID: <CAOtMX2jvVLUFHcpOs1r=Wcr_G3rKFStDQ92RatSsq-VAM9FDmQ@mail.gmail.com> In-Reply-To: <D64F8B312592434E801895942B9101163F9C7A61@MBX01.ad.oak.ox.ac.uk> References: <D64F8B312592434E801895942B9101163F9C7A61@MBX01.ad.oak.ox.ac.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Feb 16, 2016 at 4:45 AM, David Ford <david.ford@ouce.ox.ac.uk> wrote: > Hello, > > I have a number of dual homed SAS disk chasses, with a mixture of SAS and SATA drives. As expected, the SAS drives appear to both hosts, and the SATA drives appear on a single host, which gets the SAS affiliation. > > From the host with the SATA drive visible: > > [root@backup-san1 ~]# camcontrol smpphylist /dev/ses0 > 26 PHYs: > PHY Attached SAS Address > 0 0x0000000000000000 > 1 0x0000000000000000 > 2 0x50080e53c2b8f002 <ATA ST1000DM003-1ER1 CC45> (da33,pass36) > 3 0x5000cca01ab1a139 <IBM-ESXS HUS723030ALS64 J210> (pass0,da0) > 4 0x0000000000000000 > 5 0x0000000000000000 > 6 0x0000000000000000 > 7 0x5000c50041affc01 <IBM-ESXS ST33000650SS BC36> (pass2,da2) > 8 0x0000000000000000 > 9 0x0000000000000000 > 10 0x5000cca03ea41585 <IBM-ESXS HUS723030ALS64 J3K7> (pass1,da1) > 11 0x0000000000000000 > 12 0x500605b004f24f20 > 13 0x500605b004f24f20 > 14 0x500605b004f24f20 > 15 0x500605b004f24f20 > 16 0x0000000000000000 > 17 0x0000000000000000 > 18 0x0000000000000000 > 19 0x0000000000000000 > 20 0x0000000000000000 > 21 0x0000000000000000 > 22 0x0000000000000000 > 23 0x0000000000000000 > 24 0x50080e53c2b8f03d > 25 0x000000000000003e > > From the other host: > > root@backup-san-02:~ # camcontrol smpphylist /dev/ses0 > 26 PHYs: > PHY Attached SAS Address > 0 0x0000000000000000 > 1 0x0000000000000000 > 2 0x0000000000000000 > 3 0x5000cca01ab1a13a <IBM-ESXS HUS723030ALS64 J210> (pass2,da1) > 4 0x0000000000000000 > 5 0x0000000000000000 > 6 0x0000000000000000 > 7 0x5000c50041affc02 <IBM-ESXS ST33000650SS BC36> (pass1,da0) > 8 0x0000000000000000 > 9 0x0000000000000000 > 10 0x5000cca03ea41586 <IBM-ESXS HUS723030ALS64 J3K7> (pass3,da2) > 11 0x0000000000000000 > 12 0x500605b004f27920 > 13 0x500605b004f27920 > 14 0x500605b004f27920 > 15 0x500605b004f27920 > 16 0x0000000000000000 > 17 0x0000000000000000 > 18 0x0000000000000000 > 19 0x0000000000000000 > 20 0x0000000000000000 > 21 0x0000000000000000 > 22 0x0000000000000000 > 23 0x0000000000000000 > 24 0x50080e53c1e1803d > 25 0x000000000000003e > > > I can successfully clear the affiliation: > > [root@backup-san1 ~]# camcontrol smppc /dev/ses0 -p 2 -o clearaffiliation > [root@backup-san1 ~]# smp_rep_phy_sata --phy=2 /dev/ses0 > Report phy SATA response: > expander change count: 74 > phy identifier: 2 > STP I_T nexus loss occurred: 0 > affiliations supported: 1 > affiliation valid: 0 > STP SAS address: 0x50080e53c2b8f002 > register device to host FIS: > 34 00 50 01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 > affiliated STP initiator SAS address: 0x0 > STP I_T nexus loss SAS address: 0x0 > affiliation context: 0 > current affiliation contexts: 0 > maximum affiliation contexts: 1 > > However from the other host: > > root@backup-san-02:~ # camcontrol smppc /dev/ses0 -p 2 -o sataportsel > > appears to do nothing - the output of camcontrol smpphylist /dev/ses0 and it does not appear on a rescan, or if I attempt to hard reset it. > > root@backup-san-02:~ # smp_rep_phy_sata --phy=2 /dev/ses0 > Report phy SATA result: Phy does not support SATA > > The systems are running Freebsd 10.2, and I have tested with both the mps and the mpr driver on different systems, the behaviour is identical. > > Either I'm missing a crucial step in this process, or it's a bug. Does anyone have any suggestions. > > Thanks > > David > You aren't missing anything. This is just a difference between SATA and SAS. SAS drives have two ports, and SATA drives have only one. Most (all?) multipath JBODs like yours have two separate expander chips. They connect every slot's first port to the first expander and every slot's second port to the second expander. That results in a chassis with no SPOF. With such hardware, there's no way to connect a SATA drive to both servers. And with more complicated hardware that uses a single expander chip combined with SAS zoning to connect a SATA drive to two servers, you're stuck with a SPOF. -Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2jvVLUFHcpOs1r=Wcr_G3rKFStDQ92RatSsq-VAM9FDmQ>