From owner-freebsd-geom@FreeBSD.ORG Mon Jul 16 15:45:45 2012 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C65C106564A for ; Mon, 16 Jul 2012 15:45:45 +0000 (UTC) (envelope-from trent@snakebite.org) Received: from exchange.liveoffice.com (exchla3.liveoffice.com [64.70.67.188]) by mx1.freebsd.org (Postfix) with ESMTP id 38C438FC17 for ; Mon, 16 Jul 2012 15:45:45 +0000 (UTC) Received: from EXHUB02.exchhosting.com (192.168.11.214) by exhub08.exchhosting.com (192.168.11.106) with Microsoft SMTP Server (TLS) id 8.3.213.0; Mon, 16 Jul 2012 08:44:34 -0700 Received: from EXMBX10.exchhosting.com ([fe80::9c37:32f6:a508:a44f]) by exhub02.exchhosting.com ([fe80::311c:a4c3:90a7:3e53%12]) with mapi; Mon, 16 Jul 2012 08:44:33 -0700 From: Trent Nelson To: "freebsd-geom@FreeBSD.org" Date: Mon, 16 Jul 2012 08:44:32 -0700 Thread-Topic: Odd gmultipath behaviour Thread-Index: Ac1jaeVE9T/XjGnwQQKVZTz7fN13OQ== Message-ID: <07BAC177-1227-44B1-83F0-AC998F61E885@snakebite.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "mj@feral.com Jacob" Subject: Odd gmultipath behaviour X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 15:45:45 -0000 This seems... odd: [root@s16/ttypts/1(../misc/isp/bin)#] gmultipath status = =20 Name Status Components multipath/292_61 BROKEN N/A multipath/292_39 BROKEN N/A multipath/292_43 BROKEN N/A multipath/292_54 BROKEN N/A multipath/292_39 OPTIMAL da0 (ACTIVE) da3 (PASSIVE) da8 (PASSIVE) da11 (PASSIVE) multipath/292_43 OPTIMAL da1 (ACTIVE) da5 (PASSIVE) da12 (PASSIVE) da15 (PASSIVE) multipath/292_61 OPTIMAL da2 (ACTIVE) da6 (PASSIVE) da9 (PASSIVE) da13 (PASSIVE) multipath/292_54 OPTIMAL da4 (ACTIVE) da7 (PASSIVE) da10 (PASSIVE) da14 (PASSIVE) I was playing around with isp's target mode, zfs and ctladm. I've got two = dual-port HBAs, I set one of each port to initiator and the other to target= and created a zfs pool (mirror 292_61 292_39 mirror 292_43 292_54). Everything was fine up to this point. Then I ran a few ctladm commands, th= e last one being `ctladm port -l`, then things started going a bit screwy. = I ran Matt's isp/tools/isp_getrole and all four HBAs were now purportedly = in target mode. That, understandably, hosed the gmultipath setup, which resulted in my zpoo= l hanging. I manually isp/tools/isp_setrole'd all the HBAs back to initiat= or, isp/tools/isp_reset them, then `camcontrol rescan all`'d, which brought= everything back. But, uh, what's going on with gmultipath? Why's it hanging on to the broke= n paths? Some more output: [root@s16/ttypts/1(../misc/isp/bin)#] gmultipath list =20 Geom name: 292_61 Type: AUTOMATIC Mode: Active/Passive UUID: 1ad76fac-cf53-11e1-903e-00118513ae7d State: BROKEN Providers: 1. Name: multipath/292_61 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r1w1e1 State: BROKEN Geom name: 292_39 Type: AUTOMATIC Mode: Active/Passive UUID: 65b3e0aa-cf53-11e1-903e-00118513ae7d State: BROKEN Providers: 1. Name: multipath/292_39 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r1w1e1 State: BROKEN Geom name: 292_43 Type: AUTOMATIC Mode: Active/Passive UUID: b5f58ec4-cf53-11e1-903e-00118513ae7d State: BROKEN Providers: 1. Name: multipath/292_43 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r1w1e1 State: BROKEN Geom name: 292_54 Type: AUTOMATIC Mode: Active/Passive UUID: ef54b03c-cf53-11e1-903e-00118513ae7d State: BROKEN Providers: 1. Name: multipath/292_54 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r1w1e1 State: BROKEN Geom name: 292_39 Type: AUTOMATIC Mode: Active/Passive UUID: 65b3e0aa-cf53-11e1-903e-00118513ae7d State: OPTIMAL Providers: 1. Name: multipath/292_39 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r0w0e0 State: OPTIMAL Consumers: 1. Name: da0 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE 2. Name: da3 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 3. Name: da8 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 4. Name: da11 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE Geom name: 292_43 Type: AUTOMATIC Mode: Active/Passive UUID: b5f58ec4-cf53-11e1-903e-00118513ae7d State: OPTIMAL Providers: 1. Name: multipath/292_43 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r0w0e0 State: OPTIMAL Consumers: 1. Name: da1 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE 2. Name: da5 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 3. Name: da12 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 4. Name: da15 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE Geom name: 292_61 Type: AUTOMATIC Mode: Active/Passive UUID: 1ad76fac-cf53-11e1-903e-00118513ae7d State: OPTIMAL Providers: 1. Name: multipath/292_61 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r0w0e0 State: OPTIMAL Consumers: 1. Name: da2 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE 2. Name: da6 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 3. Name: da9 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 4. Name: da13 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE Geom name: 292_54 Type: AUTOMATIC Mode: Active/Passive UUID: ef54b03c-cf53-11e1-903e-00118513ae7d State: OPTIMAL Providers: 1. Name: multipath/292_54 Mediasize: 146815737344 (136G) Sectorsize: 512 Mode: r0w0e0 State: OPTIMAL Consumers: 1. Name: da4 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE 2. Name: da7 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 3. Name: da10 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE 4. Name: da14 Mediasize: 146815737856 (136G) Sectorsize: 512 Mode: r1w1e1 State: PASSIVE Thoughts? Trent.