From owner-freebsd-questions@freebsd.org Thu Feb 11 15:42:39 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EE43AA47D4 for ; Thu, 11 Feb 2016 15:42:39 +0000 (UTC) (envelope-from david.ford@ouce.ox.ac.uk) Received: from fallback2.mail.ox.ac.uk (fallback2.mail.ox.ac.uk [129.67.1.167]) by mx1.freebsd.org (Postfix) with ESMTP id 2E766FF8 for ; Thu, 11 Feb 2016 15:42:38 +0000 (UTC) (envelope-from david.ford@ouce.ox.ac.uk) Received: from relay13.mail.ox.ac.uk ([129.67.1.166]) by fallback2.mail.ox.ac.uk with esmtp (Exim 4.80) (envelope-from ) id 1aTt5q-0002WL-8o for freebsd-questions@freebsd.org; Thu, 11 Feb 2016 15:23:42 +0000 Received: from hub04.nexus.ox.ac.uk ([163.1.154.215] helo=HUB04.ad.oak.ox.ac.uk) by relay13.mail.ox.ac.uk with esmtp (Exim 4.80) (envelope-from ) id 1aTt5h-0005vB-hb for freebsd-questions@freebsd.org; Thu, 11 Feb 2016 15:23:33 +0000 Received: from MBX01.ad.oak.ox.ac.uk ([169.254.1.95]) by HUB04.ad.oak.ox.ac.uk ([169.254.211.233]) with mapi id 14.03.0248.002; Thu, 11 Feb 2016 15:23:33 +0000 From: David Ford To: "'freebsd-questions@freebsd.org'" Subject: multi-homed drive SATA drive affiliations Thread-Topic: multi-homed drive SATA drive affiliations Thread-Index: AdFk3ln/R2reH8aeT+aiRK6/GKLaJQ== Date: Thu, 11 Feb 2016 15:23:31 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.150.239] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Feb 2016 15:42:39 -0000 Hello, I suspect I'm missing something obvious here, but I can't quite think what.= I have a set of SAS disk shelves which are each dual homed to a pair of Fr= eeBSD 10.1 systems. These shelves contain a bunch of SAS drives, which as e= xpected work fine in this configuration, and can be accessed from whichever= host wishes to use them - the end workload is ZFS, so each disk will be as= signed to a single host, and the second host is for failover purposes. The problem is that for SATA drives (the idea is the use SSDs for L2ARC/ZIL= ) SATA affiliations don't appear to be having the desired effect, the disks= get assigned to one host when they are attached, and whilst I can clear th= e affiliations on that host, there seems no way to make the second host see= the drive once the affiliations have been cleared. On the host where the drive appears: [root@backup-san1 ~]# camcontrol devlist at scbus0 target 9 lun 0 (pass0,da0) at scbus0 target 10 lun 0 (pass1,da1) at scbus0 target 11 lun 0 (pass2,da2) at scbus0 target 12 lun 0 (pass3,ses0) at scbus0 target 13 lun 0 (da33,pass36) The last line is the ATA drive in question, On the other host: root@backup-san-02:~ # camcontrol devlist at scbus0 target 9 lun 0 (pass0,ses0) at scbus0 target 10 lun 0 (pass1,da0) at scbus0 target 11 lun 0 (pass2,da1) at scbus0 target 12 lun 0 (pass3,da2) I have tried using both: camcontrol --phy=3D2 --op=3Dca /dev/ses0 smp_phy_control --phy=3D2 --op=3Dca /dev/ses0 on the working host, to clear the affiliation, and smp_rep_phy_sata implies= this has had the desired effect: [root@backup-san1 ~]# smp_rep_phy_sata --phy=3D2 /dev/ses0 Report phy SATA response: expander change count: 71 phy identifier: 2 STP I_T nexus loss occurred: 0 affiliations supported: 1 affiliation valid: 0 STP SAS address: 0x50080e53c2b8f002 register device to host FIS: 34 00 50 01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 affiliated STP initiator SAS address: 0x0 STP I_T nexus loss SAS address: 0x0 affiliation context: 0 current affiliation contexts: 0 However, nothing I've tried on the second host then allows it to see it: root@backup-san-02:~ # smp_phy_control --phy=3D2 --op=3Dtspss /dev/ses0 root@backup-san-02:~ # smp_rep_phy_sata --phy=3D2 /dev/ses0 Report phy SATA result: Phy does not support SATA root@backup-san-02:~ # smp_phy_control --phy=3D2 --op=3Dlr /dev/ses0 root@backup-san-02:~ # camcontrol reset root@backup-san-02:~ # camcontrol rescan root@backup-san-02:~ # smp_rep_phy_sata --phy=3D2 /dev/ses0 Report phy SATA result: Phy does not support SATA I've even tried disabling the device from the first host, but it still beha= ves in an identical fashion. Does anyone happen to know what I've done wrong? Thanks David --=20 David Ford IT Manager, School of Geography and the Environment For general IT Support queries please contact itsupport@ouce.ox.ac.uk Telephone: +44 1865 285089