From owner-freebsd-fs@freebsd.org Sun Apr 22 21:00:39 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EB9FFBD4B8 for ; Sun, 22 Apr 2018 21:00:39 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B47FE75912 for ; Sun, 22 Apr 2018 21:00:38 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6F387FBD4B1; Sun, 22 Apr 2018 21:00:38 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30211FBD4AD for ; Sun, 22 Apr 2018 21:00:38 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BAD0F758E4 for ; Sun, 22 Apr 2018 21:00:37 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id E55352339B for ; Sun, 22 Apr 2018 21:00:36 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3ML0a2x033486 for ; Sun, 22 Apr 2018 21:00:36 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3ML0aYE033480 for fs@FreeBSD.org; Sun, 22 Apr 2018 21:00:36 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201804222100.w3ML0aYE033480@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 22 Apr 2018 21:00:36 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Apr 2018 21:00:39 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic New | 217062 | for file systems mounted with -o noexec, exec=off New | 221909 | [ZFS] Add a sysctl to toggle send_corrupt_data Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 211491 | System hangs after "Uptime" on reboot with ZFS 8 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Sun Apr 22 21:18:21 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54762F8B2E1 for ; Sun, 22 Apr 2018 21:18:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670063.outbound.protection.outlook.com [40.107.67.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4A2078EEA for ; Sun, 22 Apr 2018 21:18:20 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM (52.132.66.153) by YQBPR0101MB1489.CANPRD01.PROD.OUTLOOK.COM (52.132.69.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.696.12; Sun, 22 Apr 2018 21:18:19 +0000 Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a]) by YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a%13]) with mapi id 15.20.0696.017; Sun, 22 Apr 2018 21:18:19 +0000 From: Rick Macklem To: "freebsd-fs@freebsd.org" Subject: pNFS server completed Thread-Topic: pNFS server completed Thread-Index: AQHT2n6KBZhPilerAEebAsjWoa5XxQ== Date: Sun, 22 Apr 2018 21:18:19 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=none action=none header.from=uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQBPR0101MB1489; 7:sLYp2+Dk14JKfJjzHdCy9Vn/pxYSsna0079MpeThk9Rm1xpKQZYaE3mpUoLkEG989bay90FFojq8bvlcenjkW1TzqP3F1xT94wKDZArlRONVRDzFuFNWAl8bhk3yfEl+r1QxoQ1OYsRVqyOepPeSvHd3wpppAXE7RmrNRnLJiiSBHapQkOHQYaZk4SL/251LGzeD+4eUt3dnDWRNPeFH03uDDrNFYvz/tIOfbXQ00x/6SYXzm18obs/xjmfsZ/+T x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(2017052603328)(7153060)(7193020); SRVR:YQBPR0101MB1489; x-ms-traffictypediagnostic: YQBPR0101MB1489: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(5213294742642); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(3231232)(944501410)(52105095)(10201501046)(6041310)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(20161123560045)(6072148)(201708071742011); SRVR:YQBPR0101MB1489; BCL:0; PCL:0; RULEID:; SRVR:YQBPR0101MB1489; x-forefront-prvs: 0650714AAA x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(39850400004)(396003)(346002)(366004)(39380400002)(26005)(2351001)(9686003)(102836004)(6506007)(55016002)(6306002)(53936002)(33656002)(316002)(786003)(5640700003)(3660700001)(6436002)(186003)(7696005)(3280700002)(476003)(3480700004)(2501003)(25786009)(86362001)(8936002)(6916009)(2900100001)(7116003)(74316002)(5250100002)(305945005)(5660300001)(2906002)(478600001)(74482002)(966005)(8676002)(81166006); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB1489; H:YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; MLV:sfv; x-microsoft-antispam-message-info: kYV7gGAesbcxcjbsH2RUSJS9tSZWajMHJbYq8LJgLi35ghjFNu4+nq8MqBWMcnSCE4h1NyBVviZHLjcEqXnObvVseCfBpTv+aXQNvPMQ9UG3boysDNjifTdYZf1mq7QUdE/ArzJmi5ajuO615uPAo93LdFg1l1caWdBu2GU8CGmee6vXHjZXmzNYWAxtAnqn spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: b7822600-e6bb-43af-3f0f-08d5a896922c X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: b7822600-e6bb-43af-3f0f-08d5a896922c X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Apr 2018 21:18:19.2945 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB1489 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Apr 2018 21:18:21 -0000 Hi, I have now updated the projects/pnfs-planb-server subtree of svn with what I believe is a complete pNFS server. It now handles mirrored Data Servers (= DSs) and recovery of a failed mirrored DS while the service is operational. It handle non-pNFS clients (such as NFSv3 mounts) via having the MetaData Server act as a proxy for Read/Write operations. For pNFS capable NFSv4.1 clients, the Read/Write operations are done direct= ly on the appropriate DS(s), avoiding the MetaData Server (which act like a no= rmal NFS server except for pNFS capable clients). If you are interested in more information or doing testing, see: http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt I hope that this should be ready for head/current with some additional test= ing. I will post here asking the "collective" about this when I think it is read= y. If there are any volunteers on the doc side who would like to turn the abo= ve text file into a useful document, that would be greatly appreciated. Good luck with it, if you test it, rick= From owner-freebsd-fs@freebsd.org Mon Apr 23 08:25:16 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22517FA70EE for ; Mon, 23 Apr 2018 08:25:16 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9DE1980EB1 for ; Mon, 23 Apr 2018 08:25:14 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 4F09D4318E; Mon, 23 Apr 2018 10:19:09 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id QASEXy4AyV9m; Mon, 23 Apr 2018 10:19:08 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 049E743191; Mon, 23 Apr 2018 10:19:08 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 049E743191 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 849djm81hV28; Mon, 23 Apr 2018 10:19:07 +0200 (CEST) Received: from its-5cg60227mj (its-5cg60227mj.its.uu.se [130.238.131.18]) by mail.inparadise.se (Postfix) with ESMTPSA id 6F1F14318E; Mon, 23 Apr 2018 10:19:07 +0200 (CEST) Message-ID: <1524471526.9560.51.camel@inparadise.se> Subject: Re: pNFS server completed From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Rick Macklem , "freebsd-fs@freebsd.org" Date: Mon, 23 Apr 2018 10:18:46 +0200 In-Reply-To: References: Organization: InParadise Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-ll2EqRvoDmqtXbsQwuuy" X-Mailer: Evolution 3.26.1-1 Mime-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 08:25:16 -0000 --=-ll2EqRvoDmqtXbsQwuuy Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, 2018-04-22 at 21:18 +0000, Rick Macklem wrote: > Hi, >=20 > I have now updated the projects/pnfs-planb-server subtree of svn with > what > I believe is a complete pNFS server. It now handles mirrored Data > Servers (DSs) > and recovery of a failed mirrored DS while the service is > operational. > It handle non-pNFS clients (such as NFSv3 mounts) via having the > MetaData > Server act as a proxy for Read/Write operations. > For pNFS capable NFSv4.1 clients, the Read/Write operations are done > directly > on the appropriate DS(s), avoiding the MetaData Server (which act > like a normal > NFS server except for pNFS capable clients). >=20 > If you are interested in more information or doing testing, see: > http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt >=20 > I hope that this should be ready for head/current with some > additional testing. > I will post here asking the "collective" about this when I think it > is ready. >=20 > If there are any volunteers on the doc side who would like to turn > the above > text file into a useful document, that would be greatly appreciated. >=20 > Good luck with it, if you test it, rick > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" Hi Rick! I think this sounds amazing to have a distributed file server natively in FreeBSD, awesome! One question that comes to mind is about the MDS: wouldn=C2=B4t that be a SPOF? Maybe that has already been discussed and explained and I=C2=B4ve jus= t missed it, but would be nice to know. And if it is a SPOF, are there any plans of also mirroring MDS's as well? Perhaps a DS can be MDS to, something like that? /K --=-ll2EqRvoDmqtXbsQwuuy Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQEcBAABCAAGBQJa3ZbnAAoJEBpo164N2cuRW48H/i2LmHYkVF4eBqSlRyzzB60m lKbWtMYQVVcG9ZGiC+56ydPs/6fh68r+PDsZQw0B78AY4dmKqKJWU64bprwoGucy NxYvIw9FT2CZsVtUO0H4datZYlMKA0xIYTdbvlnWBd1B6zvn+iuBCAKdpxTQoik1 6Unl/coi+s65o3lhkOnKn4r0xS12smGrJ5M+t4DNiLah5u0fwTjyFly1+ATpeBfy Hr0QWsKvPJFIpKcmom3qCU0kb8a6PFv63cbjab+BcmnAEJYBdH8mnlD0UkfligeS PDcLkmP/b2fRud5FLQxpkiYdXisael5CS03uo4cEQ46Mus/56rtOyaeq4YBkXRU= =84Yq -----END PGP SIGNATURE----- --=-ll2EqRvoDmqtXbsQwuuy-- From owner-freebsd-fs@freebsd.org Mon Apr 23 15:59:31 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B67AFA5011 for ; Mon, 23 Apr 2018 15:59:31 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 45C4D7F1F0 for ; Mon, 23 Apr 2018 15:59:31 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w3NFxU38098746 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 23 Apr 2018 11:59:30 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.ca [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w3NFxSqd052784; Mon, 23 Apr 2018 11:59:29 -0400 (EDT) (envelope-from mike@sentex.net) To: "freebsd-fs@freebsd.org" From: Mike Tancsa Subject: ctl_isc_lun_sync: Received conflicting HA LUN Openpgp: preference=signencrypt Organization: Sentex Communications Message-ID: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> Date: Mon, 23 Apr 2018 11:59:28 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 15:59:31 -0000 Hi, I have been trying to test out a ctl HA setup as described in https://mezzantrop.files.wordpress.com/2017/04/approaching-online-zpool-switching-between-two-freebsd-machines_1-1.pdf However, when I start up the daemons, I get a ctl_isc_lun_sync: Received conflicting HA LUN 0 ctl_isc_lun_sync: Received conflicting HA LUN 0 ctl_isc_lun_sync: Received conflicting HA LUN 0 ctl_isc_lun_sync: Received conflicting HA LUN 0 ctl_isc_lun_sync: Received conflicting HA LUN 0 Plus, when I do a probe from the client, I only see the primary unit, so I am not able to create a multipath file system. Both servers are RELENG11 as of today (r332889) ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-fs@freebsd.org Mon Apr 23 17:11:24 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13934FA724F for ; Mon, 23 Apr 2018 17:11:24 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 55401704CD for ; Mon, 23 Apr 2018 17:11:23 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w3NHBLZl011636 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 23 Apr 2018 13:11:21 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.ca [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w3NHBJYV052974; Mon, 23 Apr 2018 13:11:19 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN To: Mikhail Zakharov Cc: "freebsd-fs@freebsd.org" References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> From: Mike Tancsa Openpgp: preference=signencrypt Organization: Sentex Communications Message-ID: <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> Date: Mon, 23 Apr 2018 13:11:19 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 17:11:24 -0000 On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: > Hello Mike, > > Thank you for your interest to my paper. I appreciate it very much! > Your error may be a consequence of the initial HA misconfiguration. What is in your /boot/loader.conf? Although the described config is quite simple, I can recheck the instruction in my paper in a couple of weeks only, unfortunately I’m on vacation right now. Hi, Config is pretty well copied from your article. Thanks for responding, and have a good holiday! a) kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" vfs.zfs.min_auto_ashift=12 zfs_load="YES" t5fw_cfg_load="YES" if_cxgbe_load="YES" vm.pmap.pti=0 vfs.zfs.arc_max=22161185792 ctl_load="YES" iscsi_load="YES" cfiscsi_load="YES" kern.cam.ctl.ha_id=1 kern.cam.ctl.ha_mode=2 kern.cam.ctl.ha_role=0 b) kern.geom.label.disk_ident.enable="0" kern.geom.label.gptid.enable="0" vfs.zfs.min_auto_ashift=12 zfs_load="YES" console="comconsole,vidconsole" comconsole_speed="115200" # Set the current serial console speed comconsole_port="0x2f8" ipmi_load="YES" ctl_load="YES" cfiscsi_load="YES" iscsi_load="YES" kern.cam.ctl.ha_id=2 kern.cam.ctl.ha_mode=2 kern.cam.ctl.ha_role=1 a) # cat /etc/ctl.conf debug 1 portal-group pg0 { discovery-auth-group no-authentication listen 192.168.242.6 listen 192.168.79.1 } target iqn.2018-01.ca.sentexcloud1:target0 { auth-group no-authentication portal-group pg0 lun 0 { backend block path /dev/zvol/tank1/v1 blocksize 4096 device-id MULTI serial 20180423 } } b) }# cat /etc/ctl.conf debug 1 portal-group pg0 { discovery-auth-group no-authentication listen 192.168.79.2 listen 192.168.242.11 } target iqn.2018-01.ca.sentexcloud1:target0 { auth-group no-authentication portal-group pg0 lun 0 { backend block path /dev/zvol/zroot/v1 blocksize 4096 device-id FBSDMULTI serial 20180423 } } a) # cat start.sh sysctl -w kern.cam.ctl.ha_peer="listen 192.168.79.1:7777" sysctl -w kern.iscsi.fail_on_disconnection=1 /etc/rc.d/ctld onestart b) # cat start.sh sysctl -w kern.cam.ctl.ha_peer="connect 192.168.79.1:7777" sysctl -w kern.iscsi.fail_on_disconnection=1 /etc/rc.d/ctld onestart From owner-freebsd-fs@freebsd.org Mon Apr 23 18:55:02 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E1AFFA95A2 for ; Mon, 23 Apr 2018 18:55:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D74968DA2 for ; Mon, 23 Apr 2018 18:55:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w3NIt1AD030173 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 23 Apr 2018 14:55:01 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.ca [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w3NIsxcW054940; Mon, 23 Apr 2018 14:54:59 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN To: Mikhail Zakharov Cc: "freebsd-fs@freebsd.org" References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <14CC6AF1-C1AB-48D2-925D-A46989AD7B3C@yahoo.com> From: Mike Tancsa Openpgp: preference=signencrypt Organization: Sentex Communications Message-ID: <1d4a2171-d083-9e37-954c-cf58ed2c2982@sentex.net> Date: Mon, 23 Apr 2018 14:54:59 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <14CC6AF1-C1AB-48D2-925D-A46989AD7B3C@yahoo.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 18:55:02 -0000 On 4/23/2018 1:41 PM, Mikhail Zakharov wrote: > > You have different Lun0 definitions in ctl.conf: check path > /dev/zvol/tank1/v1 vs path /dev/zvol/zroot/v1 as well as different > device-ids. These values should describe the same “drive” in both > configuration, are you sure if it’s the same disk? :) Hi, I guess I am misunderstanding it. The ctl.conf is on 2 different servers, so they are two different disks. What I am hoping to achieve is sort of like a HAST setup. So that if server-a dies, server-b takes over So on server a) I did zfs create -V 80G zroot/v1 and on server b) I did zfs create -V 80G tank1/v1 However, in order for the files to be identical, I used the slower disks on b) so that the path statement is identical. But should this matter ? a) # cat /etc/ctl.conf debug 1 portal-group pg0 { discovery-auth-group no-authentication listen 192.168.242.6 listen 192.168.79.1 } target iqn.2018-01.ca.sentexcloud1:target0 { auth-group no-authentication portal-group pg0 lun 1 { backend block path /dev/zvol/tank1/v2 blocksize 4096 device-id MULTI serial 20180423 } } b) # cat /etc/ctl.conf debug 1 portal-group pg0 { discovery-auth-group no-authentication listen 192.168.79.2 listen 192.168.242.11 } target iqn.2018-01.ca.sentexcloud1:target0 { auth-group no-authentication portal-group pg0 lun 1 { backend block path /dev/zvol/zroot/v2 blocksize 4096 device-id MULTI serial 20180423 } } > > WBR, > Mike > > 23 апр. 2018 г., в 20:11, Mike Tancsa > написал(а): > >> On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: >>> Hello Mike, >>> >>> Thank you for your interest to my paper. I appreciate it very much! >>> Your error may be a consequence of the initial HA misconfiguration. >>> What is in your /boot/loader.conf? Although the described config is >>> quite simple, I can recheck the instruction in my paper in a couple >>> of weeks only, unfortunately I’m on vacation right now. >> >> Hi, >>    Config is pretty well copied from your article. Thanks for responding, >> and have a good holiday! >> >> a) >> kern.geom.label.disk_ident.enable="0" >> kern.geom.label.gptid.enable="0" >> vfs.zfs.min_auto_ashift=12 >> zfs_load="YES" >> t5fw_cfg_load="YES" >> if_cxgbe_load="YES" >> vm.pmap.pti=0 >> vfs.zfs.arc_max=22161185792 >> ctl_load="YES" >> iscsi_load="YES" >> cfiscsi_load="YES" >> kern.cam.ctl.ha_id=1 >> kern.cam.ctl.ha_mode=2 >> kern.cam.ctl.ha_role=0 >> >> b) >> kern.geom.label.disk_ident.enable="0" >> kern.geom.label.gptid.enable="0" >> vfs.zfs.min_auto_ashift=12 >> zfs_load="YES" >> console="comconsole,vidconsole" >> comconsole_speed="115200"       # Set the current serial console speed >> comconsole_port="0x2f8" >> ipmi_load="YES" >> ctl_load="YES" >> cfiscsi_load="YES" >> iscsi_load="YES" >> kern.cam.ctl.ha_id=2 >> kern.cam.ctl.ha_mode=2 >> kern.cam.ctl.ha_role=1 >> >> >> >> >> >> a) >> # cat /etc/ctl.conf >> debug 1 >> portal-group pg0 { >>    discovery-auth-group no-authentication >>    listen 192.168.242.6 >>    listen 192.168.79.1 >> } >> >> target iqn.2018-01.ca.sentexcloud1:target0 { >>    auth-group no-authentication >>    portal-group pg0 >> >>    lun 0 { >>            backend block >>            path /dev/zvol/tank1/v1 >>            blocksize 4096 >>            device-id MULTI >>            serial 20180423 >>    } >> } >> >> b) >> }# cat /etc/ctl.conf >>    debug 1 >> portal-group pg0 { >>    discovery-auth-group no-authentication >>    listen 192.168.79.2 >>    listen 192.168.242.11 >> } >> >> target iqn.2018-01.ca.sentexcloud1:target0 { >>    auth-group no-authentication >>    portal-group pg0 >> >>    lun 0 { >>            backend block >>            path /dev/zvol/zroot/v1 >>            blocksize 4096 >>            device-id FBSDMULTI >>            serial 20180423 >>    } >> } >> >> a) >> # cat start.sh >> sysctl -w kern.cam.ctl.ha_peer="listen 192.168.79.1:7777" >> sysctl -w kern.iscsi.fail_on_disconnection=1 >> /etc/rc.d/ctld onestart >> >> b) >> # cat start.sh >> sysctl -w kern.cam.ctl.ha_peer="connect 192.168.79.1:7777" >> sysctl -w kern.iscsi.fail_on_disconnection=1 >> /etc/rc.d/ctld onestart >> >> -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-fs@freebsd.org Mon Apr 23 21:00:54 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A41F2FAB5C9 for ; Mon, 23 Apr 2018 21:00:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660052.outbound.protection.outlook.com [40.107.66.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3761B83EF7 for ; Mon, 23 Apr 2018 21:00:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM (52.132.66.153) by YQBPR0101MB1060.CANPRD01.PROD.OUTLOOK.COM (52.132.66.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Mon, 23 Apr 2018 21:00:52 +0000 Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a]) by YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a%13]) with mapi id 15.20.0696.017; Mon, 23 Apr 2018 21:00:52 +0000 From: Rick Macklem To: "freebsd-fs@freebsd.org" , "karli@inparadise.se" Subject: Re: pNFS server completed Thread-Topic: pNFS server completed Thread-Index: AQHT2n6KBZhPilerAEebAsjWoa5XxaQOAmgAgADSTVg= Date: Mon, 23 Apr 2018 21:00:52 +0000 Message-ID: References: , <1524471526.9560.51.camel@inparadise.se> In-Reply-To: <1524471526.9560.51.camel@inparadise.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=none action=none header.from=uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQBPR0101MB1060; 7:qz6J6/g11MwOFha86gcH3byZoLqDNM2PiOZTAMf+YSdmfocjS1fagY9QXlHsdZBKeXNXxby2uJcT/30w10e+YWLKWh6gKIuwvHApDBLHZL3aKCL/yVI+RVELTnHSLSPRnEMhqftWu2He1VtvvK9MgyBF5ekkA/Z0jdZXk4tlJ9GRiJ90WlsVisRkArmAKoZDMQquNMOiznmUQHN7m6zDkviwgL5tSlvu92t1sIMkD8Nd+Awlgb/19Cfb3hIdmhPQ x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(5600026)(2017052603328)(7153060)(7193020); SRVR:YQBPR0101MB1060; x-ms-traffictypediagnostic: YQBPR0101MB1060: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(5213294742642); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(3231232)(944501410)(52105095)(3002001)(6041310)(20161123560045)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011); SRVR:YQBPR0101MB1060; BCL:0; PCL:0; RULEID:; SRVR:YQBPR0101MB1060; x-forefront-prvs: 06515DA04B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(39380400002)(346002)(366004)(376002)(396003)(377424004)(229853002)(6306002)(9686003)(6506007)(33656002)(102836004)(55016002)(59450400001)(53936002)(26005)(7696005)(2906002)(6246003)(74316002)(3280700002)(7116003)(305945005)(76176011)(5660300001)(74482002)(5250100002)(2501003)(3660700001)(2900100001)(8676002)(8936002)(81166006)(110136005)(966005)(316002)(25786009)(3480700004)(478600001)(786003)(476003)(6436002)(11346002)(86362001)(186003)(446003); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB1060; H:YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; MLV:sfv; x-microsoft-antispam-message-info: zsDkGnVgUsnE6BHixmc7G9lnQfcVSWIZFCONHZ7PfjDLfhKaQel/MapuJy5zi2tiBF2o7rqNGZZn/kryBphnLU7ox1VOGsGLR3tUZoUiNjcKzfKQy4zQlNvz7vplXK8ZjODKtAFzXkyky3RzXuBCd2eca4/PQNMkSUdwzF8biJsIzNc715Ma6SWogLvkf2tf spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 77d6a201-f3d9-4ca5-59f2-08d5a95d4ca6 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 77d6a201-f3d9-4ca5-59f2-08d5a95d4ca6 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Apr 2018 21:00:52.5347 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB1060 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 21:00:55 -0000 Karli Sj=F6berg wrote: >On Sun, 2018-04-22 at 21:18 +0000, Rick Macklem wrote: >> Hi, >> >> I have now updated the projects/pnfs-planb-server subtree of svn with >> what >> I believe is a complete pNFS server. It now handles mirrored Data >> Servers (DSs) >> and recovery of a failed mirrored DS while the service is >> operational. >> It handle non-pNFS clients (such as NFSv3 mounts) via having the >> MetaData >> Server act as a proxy for Read/Write operations. >> For pNFS capable NFSv4.1 clients, the Read/Write operations are done >> directly >> on the appropriate DS(s), avoiding the MetaData Server (which act >> like a normal >> NFS server except for pNFS capable clients). >> >> If you are interested in more information or doing testing, see: >> http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt >> >> I hope that this should be ready for head/current with some >> additional testing. >> I will post here asking the "collective" about this when I think it >> is ready. >> >> If there are any volunteers on the doc side who would like to turn >> the above >> text file into a useful document, that would be greatly appreciated. >> >> Good luck with it, if you test it, rick > >Hi Rick! > >I think this sounds amazing to have a distributed file server natively >in FreeBSD, awesome! > >One question that comes to mind is about the MDS: wouldn=B4t that be a >SPOF?=20 Yes, the MDS is just like a regular NFS server on FreeBSD. It is a single p= oint of failure. >Maybe that has already been discussed and explained and I=B4ve just >missed it, but would be nice to know. And if it is a SPOF, are there >any plans of also mirroring MDS's as well? Perhaps a DS can be MDS to, >something like that? No plans w.r.t. mirroring the MDS. The pNFS client using Flexible File Layo= ut knows to write to both mirrors (and the MDS can do so acting as a proxy for a client). However, NFS clients don't know how to do NFS operations on multiple NFS se= rvers. As such, any mirroring would have to be implemented "behind the curtain" and I'll admit I don't know how to do that. I did try using a clustered file system (GlusterFS) as a backend when I fir= st implemented a pNFS server two years ago, but that was a failure, due to terrible performance. (Using a clustered file system as a storage backend w= ould make multiple NFS servers/MDSs more feasible, but it didn't work.) rick From owner-freebsd-fs@freebsd.org Mon Apr 23 18:56:14 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DAF2FA95F3 for ; Mon, 23 Apr 2018 18:56:14 +0000 (UTC) (envelope-from zmey20000@yahoo.com) Received: from sonic315-35.consmr.mail.gq1.yahoo.com (sonic315-35.consmr.mail.gq1.yahoo.com [98.137.65.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1244468E2F for ; Mon, 23 Apr 2018 18:56:13 +0000 (UTC) (envelope-from zmey20000@yahoo.com) X-YMail-OSG: 6Ca.hm8VM1nkFLbYHBwn7MgI3YCAj7jOWhH1bpm_wGZ3I6ZMFIgY8pYrzDsh67t Ngi9KICIXXIa00xKj_Sug42g4B2xYtO.gxh1iLSQYWND44R5.80IKsAFKTSb7bNJ.Gw6kQEOiNhr vKIFEkH8a169Jjw724dL6L5DSf.zR7BNUIoBGaA8kmHWZRRNiBoV3zJ4AgHRp3OC5LPrVJbNMQfk I0.0h81xaG7bOp_YqOSf3uEn95Mi7aae8PUjKtjXr0m2fjJbgf0wR9X_wCkF4E6UMeFJVoZwZZQv 5hqNUYkDK6adNt.CeWHWXIj2dW1aOSIxkxv2zibdpNid9RyqAw5O9PvNIzbrGrC_twyaBtITTfJi hVzT7Ut6Bg6pwpGtBsvVwjWoYVz8cBaS3hBhTHLwuzEmTcNUXMUee2YIEvG2IQGmU028k.hGJpfY xANVSCREeFS1clIhl6h0BUtGZ.EFGd6jE.fWsbDQhPzY8Cc5uq3_q.VGX8z_Blaliq14GMK6dcok OnPx05kNg42GxyzsDjM6LWXz1B_h.zT9lGxD5uCES4LTqN_WW4Q2qZzNR7SfGH0EdQpACk8oJhQ1 JHq3.L07vTOQ8IOeMpbd0Qw-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Mon, 23 Apr 2018 18:56:12 +0000 Received: from athe-hcsd-peer.customers.otenet.gr (EHLO [192.168.130.11]) ([195.167.103.234]) by smtp420.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID d4fa2d3c1d83e447501ea45c6dabe011; Mon, 23 Apr 2018 17:41:21 +0000 (UTC) Mime-Version: 1.0 (1.0) Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Mikhail Zakharov X-Mailer: iPad Mail (15E216) In-Reply-To: <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> Date: Mon, 23 Apr 2018 20:41:18 +0300 Cc: "freebsd-fs@freebsd.org" Message-Id: <14CC6AF1-C1AB-48D2-925D-A46989AD7B3C@yahoo.com> References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> To: Mike Tancsa X-Mailman-Approved-At: Mon, 23 Apr 2018 21:03:19 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 18:56:14 -0000 Hi,=20 and thank you! You have different Lun0 definitions in ctl.conf: check path /dev/zvol/tank1/= v1 vs path /dev/zvol/zroot/v1 as well as different device-ids. These values s= hould describe the same =E2=80=9Cdrive=E2=80=9D in both configuration, are y= ou sure if it=E2=80=99s the same disk? :) WBR, Mike > 23 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 20:11, Mike Tancsa =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >=20 >> On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: >> Hello Mike, >>=20 >> Thank you for your interest to my paper. I appreciate it very much! >> Your error may be a consequence of the initial HA misconfiguration. What i= s in your /boot/loader.conf? Although the described config is quite simple, I= can recheck the instruction in my paper in a couple of weeks only, unfortun= ately I=E2=80=99m on vacation right now. >=20 > Hi, > Config is pretty well copied from your article. Thanks for responding, > and have a good holiday! >=20 > a) > kern.geom.label.disk_ident.enable=3D"0" > kern.geom.label.gptid.enable=3D"0" > vfs.zfs.min_auto_ashift=3D12 > zfs_load=3D"YES" > t5fw_cfg_load=3D"YES" > if_cxgbe_load=3D"YES" > vm.pmap.pti=3D0 > vfs.zfs.arc_max=3D22161185792 > ctl_load=3D"YES" > iscsi_load=3D"YES" > cfiscsi_load=3D"YES" > kern.cam.ctl.ha_id=3D1 > kern.cam.ctl.ha_mode=3D2 > kern.cam.ctl.ha_role=3D0 >=20 > b) > kern.geom.label.disk_ident.enable=3D"0" > kern.geom.label.gptid.enable=3D"0" > vfs.zfs.min_auto_ashift=3D12 > zfs_load=3D"YES" > console=3D"comconsole,vidconsole" > comconsole_speed=3D"115200" # Set the current serial console speed > comconsole_port=3D"0x2f8" > ipmi_load=3D"YES" > ctl_load=3D"YES" > cfiscsi_load=3D"YES" > iscsi_load=3D"YES" > kern.cam.ctl.ha_id=3D2 > kern.cam.ctl.ha_mode=3D2 > kern.cam.ctl.ha_role=3D1 >=20 >=20 >=20 >=20 >=20 > a) > # cat /etc/ctl.conf > debug 1 > portal-group pg0 { > discovery-auth-group no-authentication > listen 192.168.242.6 > listen 192.168.79.1 > } >=20 > target iqn.2018-01.ca.sentexcloud1:target0 { > auth-group no-authentication > portal-group pg0 >=20 > lun 0 { > backend block > path /dev/zvol/tank1/v1 > blocksize 4096 > device-id MULTI > serial 20180423 > } > } >=20 > b) > }# cat /etc/ctl.conf > debug 1 > portal-group pg0 { > discovery-auth-group no-authentication > listen 192.168.79.2 > listen 192.168.242.11 > } >=20 > target iqn.2018-01.ca.sentexcloud1:target0 { > auth-group no-authentication > portal-group pg0 >=20 > lun 0 { > backend block > path /dev/zvol/zroot/v1 > blocksize 4096 > device-id FBSDMULTI > serial 20180423 > } > } >=20 > a) > # cat start.sh > sysctl -w kern.cam.ctl.ha_peer=3D"listen 192.168.79.1:7777" > sysctl -w kern.iscsi.fail_on_disconnection=3D1 > /etc/rc.d/ctld onestart >=20 > b) > # cat start.sh > sysctl -w kern.cam.ctl.ha_peer=3D"connect 192.168.79.1:7777" > sysctl -w kern.iscsi.fail_on_disconnection=3D1 > /etc/rc.d/ctld onestart >=20 >=20 From owner-freebsd-fs@freebsd.org Tue Apr 24 05:46:44 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C56CFBC2EE for ; Tue, 24 Apr 2018 05:46:44 +0000 (UTC) (envelope-from zmey20000@yahoo.com) Received: from sonic308-39.consmr.mail.ne1.yahoo.com (sonic308-39.consmr.mail.ne1.yahoo.com [66.163.187.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0818479B9E for ; Tue, 24 Apr 2018 05:46:43 +0000 (UTC) (envelope-from zmey20000@yahoo.com) X-YMail-OSG: HiVhAWAVM1ncfBdeGhTbSsrnq8_UxAfeOALA6pbVgMI6OAwb3cO6C_Osp1hr2FB OYku8M1zoDerWxgtwBoJT1sv4AbkmB2aF7Sm69nWAM6lzEWMOYIWCXaZ.r8mbnu3wkl2uPrP4Tp8 To1OkzZVJ3cp3KJS7GQfwWHeKM7oE6lQDDzrgmEnO7X_ub9f.t31Bf9Ngj.eluLK5vkl2ki.nlUr 7QvvmlGW9735YHfe2L9ewcGhfhF4BO8AUq7qHsyFkUVrGCNoNEFw0bZiov9U.N7vLHXMjQ_rT2.U kEDGjAtMs0wMGOOG1SHrYx_nhP.wQNDGjvqwrwxZnms.JnHJS8XJOPkPQ4ReWlnKqIVvtsU8wp45 UpYr8.M7APYWqs5UnUEQ8OFasOV406CANWDQivshe88DfrUWrlGUuoIu537cVK5twLVQ.fUoSGgP mM6kRn387zwxBJB9gUVBMW1PJxMBOAKL2dNlG3nUlB5DDMBfWCED9Ff.uZ7JXCthy60Zqb0hV1O6 OfHpe5gyo_ahRz.TA8_O5merWJmnXAErw56xc4XvcdaT4gYLJfxJ8.eNuhhpfWgpyNNXRgbwfntp e5qs- Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.ne1.yahoo.com with HTTP; Tue, 24 Apr 2018 05:46:42 +0000 Received: from athe-hcsd-peer.customers.otenet.gr (EHLO [192.168.140.13]) ([195.167.103.234]) by smtp417.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 5b02326fd0168e2da661101e424faf52; Tue, 24 Apr 2018 04:52:10 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Mikhail Zakharov X-Mailer: iPhone Mail (15E216) In-Reply-To: <1d4a2171-d083-9e37-954c-cf58ed2c2982@sentex.net> Date: Tue, 24 Apr 2018 07:52:07 +0300 Cc: "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <14CC6AF1-C1AB-48D2-925D-A46989AD7B3C@yahoo.com> <1d4a2171-d083-9e37-954c-cf58ed2c2982@sentex.net> To: Mike Tancsa X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 05:46:44 -0000 Yes, there is misunderstanding.=20 CTL HA setup works for 2 different servers that are accessing right the same= drive. It is not like HAST. So if you need replication to a different drive= , use HAST, but if you want simultaneous access to the single drive, use CTL= HA. Best regards, Mike=20 > 23 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 21:54, Mike Tancsa =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >=20 >> On 4/23/2018 1:41 PM, Mikhail Zakharov wrote: >>=20 >> You have different Lun0 definitions in ctl.conf: check path >> /dev/zvol/tank1/v1 vs path /dev/zvol/zroot/v1 as well as different >> device-ids. These values should describe the same =E2=80=9Cdrive=E2=80=9D= in both >> configuration, are you sure if it=E2=80=99s the same disk? :) >=20 > Hi, > I guess I am misunderstanding it. The ctl.conf is on 2 different > servers, so they are two different disks. >=20 > What I am hoping to achieve is sort of like a HAST setup. So that if > server-a dies, server-b takes over >=20 > So on server a) I did >=20 > zfs create -V 80G zroot/v1 >=20 > and on server b) I did >=20 > zfs create -V 80G tank1/v1 >=20 >=20 > However, in order for the files to be identical, I used the slower disks > on b) so that the path statement is identical. But should this matter ? >=20 >=20 > a) > # cat /etc/ctl.conf > debug 1 > portal-group pg0 { > discovery-auth-group no-authentication > listen 192.168.242.6 > listen 192.168.79.1 > } >=20 > target iqn.2018-01.ca.sentexcloud1:target0 { > auth-group no-authentication > portal-group pg0 >=20 > lun 1 { > backend block > path /dev/zvol/tank1/v2 > blocksize 4096 > device-id MULTI > serial 20180423 > } > } >=20 > b) >=20 > # cat /etc/ctl.conf > debug 1 > portal-group pg0 { > discovery-auth-group no-authentication > listen 192.168.79.2 > listen 192.168.242.11 > } >=20 > target iqn.2018-01.ca.sentexcloud1:target0 { > auth-group no-authentication > portal-group pg0 >=20 > lun 1 { > backend block > path /dev/zvol/zroot/v2 > blocksize 4096 > device-id MULTI > serial 20180423 > } > } >=20 >=20 >=20 >>=20 >> WBR, >> Mike >>=20 >> 23 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 20:11, Mike Tancsa > > =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0= ): >>=20 >>>> On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: >>>> Hello Mike, >>>>=20 >>>> Thank you for your interest to my paper. I appreciate it very much! >>>> Your error may be a consequence of the initial HA misconfiguration. >>>> What is in your /boot/loader.conf? Although the described config is >>>> quite simple, I can recheck the instruction in my paper in a couple >>>> of weeks only, unfortunately I=E2=80=99m on vacation right now. >>>=20 >>> Hi, >>> Config is pretty well copied from your article. Thanks for responding= , >>> and have a good holiday! >>>=20 >>> a) >>> kern.geom.label.disk_ident.enable=3D"0" >>> kern.geom.label.gptid.enable=3D"0" >>> vfs.zfs.min_auto_ashift=3D12 >>> zfs_load=3D"YES" >>> t5fw_cfg_load=3D"YES" >>> if_cxgbe_load=3D"YES" >>> vm.pmap.pti=3D0 >>> vfs.zfs.arc_max=3D22161185792 >>> ctl_load=3D"YES" >>> iscsi_load=3D"YES" >>> cfiscsi_load=3D"YES" >>> kern.cam.ctl.ha_id=3D1 >>> kern.cam.ctl.ha_mode=3D2 >>> kern.cam.ctl.ha_role=3D0 >>>=20 >>> b) >>> kern.geom.label.disk_ident.enable=3D"0" >>> kern.geom.label.gptid.enable=3D"0" >>> vfs.zfs.min_auto_ashift=3D12 >>> zfs_load=3D"YES" >>> console=3D"comconsole,vidconsole" >>> comconsole_speed=3D"115200" # Set the current serial console speed= >>> comconsole_port=3D"0x2f8" >>> ipmi_load=3D"YES" >>> ctl_load=3D"YES" >>> cfiscsi_load=3D"YES" >>> iscsi_load=3D"YES" >>> kern.cam.ctl.ha_id=3D2 >>> kern.cam.ctl.ha_mode=3D2 >>> kern.cam.ctl.ha_role=3D1 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>> a) >>> # cat /etc/ctl.conf >>> debug 1 >>> portal-group pg0 { >>> discovery-auth-group no-authentication >>> listen 192.168.242.6 >>> listen 192.168.79.1 >>> } >>>=20 >>> target iqn.2018-01.ca.sentexcloud1:target0 { >>> auth-group no-authentication >>> portal-group pg0 >>>=20 >>> lun 0 { >>> backend block >>> path /dev/zvol/tank1/v1 >>> blocksize 4096 >>> device-id MULTI >>> serial 20180423 >>> } >>> } >>>=20 >>> b) >>> }# cat /etc/ctl.conf >>> debug 1 >>> portal-group pg0 { >>> discovery-auth-group no-authentication >>> listen 192.168.79.2 >>> listen 192.168.242.11 >>> } >>>=20 >>> target iqn.2018-01.ca.sentexcloud1:target0 { >>> auth-group no-authentication >>> portal-group pg0 >>>=20 >>> lun 0 { >>> backend block >>> path /dev/zvol/zroot/v1 >>> blocksize 4096 >>> device-id FBSDMULTI >>> serial 20180423 >>> } >>> } >>>=20 >>> a) >>> # cat start.sh >>> sysctl -w kern.cam.ctl.ha_peer=3D"listen 192.168.79.1:7777" >>> sysctl -w kern.iscsi.fail_on_disconnection=3D1 >>> /etc/rc.d/ctld onestart >>>=20 >>> b) >>> # cat start.sh >>> sysctl -w kern.cam.ctl.ha_peer=3D"connect 192.168.79.1:7777" >>> sysctl -w kern.iscsi.fail_on_disconnection=3D1 >>> /etc/rc.d/ctld onestart >>>=20 >>>=20 >=20 >=20 > --=20 > ------------------- > Mike Tancsa, tel +1 519 651 3400 x203 > Sentex Communications, mike@sentex.net > Providing Internet services since 1994 www.sentex.net > Cambridge, Ontario Canada From owner-freebsd-fs@freebsd.org Tue Apr 24 06:09:26 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80CD9FBCCA3 for ; Tue, 24 Apr 2018 06:09:26 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 115CC7F85A for ; Tue, 24 Apr 2018 06:09:25 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 1BD44431E2; Tue, 24 Apr 2018 08:09:23 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id WPhW7o4V1jhs; Tue, 24 Apr 2018 08:09:22 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id E7446431E8; Tue, 24 Apr 2018 08:09:21 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se E7446431E8 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id MAfU5NJZgSEH; Tue, 24 Apr 2018 08:09:21 +0200 (CEST) Received: from emp181-188.eduroam.uu.se (emp181-188.eduroam.uu.se [130.238.181.188]) by mail.inparadise.se (Postfix) with ESMTPSA id AEA84431E2; Tue, 24 Apr 2018 08:09:21 +0200 (CEST) Message-ID: <1524550160.1130.6.camel@inparadise.se> Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Mikhail Zakharov Cc: "freebsd-fs@freebsd.org" , Mike Tancsa Date: Tue, 24 Apr 2018 08:09:20 +0200 In-Reply-To: <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> Organization: InParadise Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.18.5.2-0ubuntu3.2 Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 06:09:26 -0000 On Mon, 2018-04-23 at 13:11 -0400, Mike Tancsa wrote: > On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: > >=20 > > Hello Mike, > >=20 > > Thank you for your interest to my paper. I appreciate it very much! > > Your error may be a consequence of the initial HA misconfiguration. > > What is in your /boot/loader.conf? Although the described config is > > quite simple, I can recheck the instruction in my paper in a couple > > of weeks only, unfortunately I=E2=80=99m on vacation right now. [snip] I read your articles on CTL HA, BQ and BeaST, and just wanted to say they are amazing, good job! One thing I=C2=B4m wondering about though is if you can claim HA with jus= t two nodes, usually you need at least three, where the third is a tie- breaker. Otherwise with your current setup, both systems may loose contact with each other while both still being powered on, leading to potential split brain situations. What are your thoughts about that? /K From owner-freebsd-fs@freebsd.org Tue Apr 24 06:14:37 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC059FBCF79 for ; Tue, 24 Apr 2018 06:14:36 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4832E800D3 for ; Tue, 24 Apr 2018 06:14:35 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 8921B431FC; Tue, 24 Apr 2018 08:14:34 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id VqJOTCZlWsjT; Tue, 24 Apr 2018 08:14:33 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 54FB2431F2; Tue, 24 Apr 2018 08:14:33 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 54FB2431F2 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id vbgYk-2Z_rdd; Tue, 24 Apr 2018 08:14:33 +0200 (CEST) Received: from emp181-188.eduroam.uu.se (emp181-188.eduroam.uu.se [130.238.181.188]) by mail.inparadise.se (Postfix) with ESMTPSA id 31D5B431E9; Tue, 24 Apr 2018 08:14:33 +0200 (CEST) Message-ID: <1524550472.1130.10.camel@inparadise.se> Subject: Re: pNFS server completed From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Rick Macklem , "freebsd-fs@freebsd.org" Date: Tue, 24 Apr 2018 08:14:32 +0200 In-Reply-To: References: ,<1524471526.9560.51.camel@inparadise.se> Organization: InParadise Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.18.5.2-0ubuntu3.2 Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 06:14:37 -0000 On Mon, 2018-04-23 at 21:00 +0000, Rick Macklem wrote: > Karli Sj=C3=B6berg wrote: > >=20 > > On Sun, 2018-04-22 at 21:18 +0000, Rick Macklem wrote: > > >=20 > > > Hi, > > >=20 > > > I have now updated the projects/pnfs-planb-server subtree of svn > > > with > > > what > > > I believe is a complete pNFS server. It now handles mirrored Data > > > Servers (DSs) > > > and recovery of a failed mirrored DS while the service is > > > operational. > > > It handle non-pNFS clients (such as NFSv3 mounts) via having the > > > MetaData > > > Server act as a proxy for Read/Write operations. > > > For pNFS capable NFSv4.1 clients, the Read/Write operations are > > > done > > > directly > > > on the appropriate DS(s), avoiding the MetaData Server (which act > > > like a normal > > > NFS server except for pNFS capable clients). > > >=20 > > > If you are interested in more information or doing testing, see: > > > http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt > > >=20 > > > I hope that this should be ready for head/current with some > > > additional testing. > > > I will post here asking the "collective" about this when I think > > > it > > > is ready. > > >=20 > > > If there are any volunteers on the doc side=C2=A0=C2=A0who would li= ke to > > > turn > > > the above > > > text file into a useful document, that would be greatly > > > appreciated. > > >=20 > > > Good luck with it, if you test it, rick > > Hi Rick! > >=20 > > I think this sounds amazing to have a distributed file server > > natively > > in FreeBSD, awesome! > >=20 > > One question that comes to mind is about the MDS: wouldn=C2=B4t that = be > > a > > SPOF?=C2=A0 > Yes, the MDS is just like a regular NFS server on FreeBSD. It is a > single point > of failure. >=20 > >=20 > > Maybe that has already been discussed and explained and I=C2=B4ve jus= t > > missed it, but would be nice to know. And if it is a SPOF, are > > there > > any plans of also mirroring MDS's as well? Perhaps a DS can be MDS > > to, > > something like that? > No plans w.r.t. mirroring the MDS. The pNFS client using Flexible > File Layout > knows to write to both mirrors (and the MDS can do so acting as a > proxy > for a client). >=20 > However, NFS clients don't know how to do NFS operations on multiple > NFS servers. As such, any mirroring would have to be implemented > "behind the > curtain" and I'll admit I don't know how to do that. >=20 > I did try using a clustered file system (GlusterFS) as a backend when > I first > implemented a pNFS server two years ago, but that was a failure, due > to > terrible performance. (Using a clustered file system as a storage > backend would > make multiple NFS servers/MDSs more feasible, but it didn't work.) >=20 > rick OK, I think I understand, thank you for the explanation! So you could say that this enhancement is more about the performance benifit of scaling out, rather than resiliance/tolerance? /K From owner-freebsd-fs@freebsd.org Tue Apr 24 09:46:30 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A49EFC1BAE for ; Tue, 24 Apr 2018 09:46:30 +0000 (UTC) (envelope-from zmey20000@yahoo.com) Received: from sonic307-37.consmr.mail.ne1.yahoo.com (sonic307-37.consmr.mail.ne1.yahoo.com [66.163.190.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14BAB737B3 for ; Tue, 24 Apr 2018 09:46:29 +0000 (UTC) (envelope-from zmey20000@yahoo.com) X-YMail-OSG: jf5NAQYVM1nGkUh2DXAmZdR0If.LRhhyykpmCJdKiYwdUXLcji.U9DG0xdSMVFo AtRX6RVSg0X1TKWa90Lf8FL0IhpYwQNlhmBqI6LCZy7zbcwyRnJsCSOcnEtZqhAiJGmSEqh1BeyK IHdpG08BD0xnr3XCjTNPdMYvv_IgNU1vqsRwBNtaFKYMxhGHkQKWm8uc6q_FDxAEI3Xu8x3xGf2y 4_4oquI7now5_e5TT2QtVZTRf3yTgSICmmJ4BMj73PS5XJ0HyNzvOmoLy5VoW_J6xy0K.iABUPOT S.EpQKSPp.zNvNP2phf4YECASMmY2lG.KNEyhDwxVrQa0svrGV9ESAkqC6PH4eQXw2rui_ShaO3G bJE.TLir1tnANTBokt0XzzUQftSOY5Es_ckcXeA.mlrVXU0fs7jrsrlW8tTUfmMgvMffCh0Fkeaf f2.ndymv4KOu1mPl5mDr.mAmX9Jc3A8wgnxRHqodji7DF5Xs.3ldfrn_XRQ5mI07qhatwStNr2jS extfTnpYaOeVGBxkgZv8m5BQHACS2rJFkVYzwWjzElDDouo32Q3lxxqfuJv3ayxlCR38KinzprWB 2mohhPzKVRvmB4Sz0QevCSw_Y Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.ne1.yahoo.com with HTTP; Tue, 24 Apr 2018 09:46:23 +0000 Received: from athe-hcsd-peer.customers.otenet.gr (EHLO [192.168.130.11]) ([195.167.103.234]) by smtp410.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 22f7da5d83d72fd219e5448977ac8fb2; Tue, 24 Apr 2018 09:32:16 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Mikhail Zakharov X-Mailer: iPad Mail (15E216) In-Reply-To: <1524550160.1130.6.camel@inparadise.se> Date: Tue, 24 Apr 2018 12:32:12 +0300 Cc: "freebsd-fs@freebsd.org" , Mike Tancsa Content-Transfer-Encoding: quoted-printable Message-Id: <615DFFBB-239A-4350-B961-FD10D0C9A8DD@yahoo.com> References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <1524550160.1130.6.camel@inparadise.se> To: karli@inparadise.se X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 09:46:30 -0000 Hi Karli, Thank you, I=E2=80=99m just exploring the storage abilities of my preferred O= S - FreeBSD.=20 Three nodes are preferable to choose the quorum for sure, but my idea was no= t to establish contacts between nodes. Instead of it, BQ uses a small partit= ion for the =E2=80=9Cquorum=E2=80=9D on the same space where data volume is l= ocated. And if a node looses access to the quorum it means, it looses access= to the data volume too. Now, BQ runs on both nodes and both BQ instances wr= ite stamps to the quorum partition. If for any reason BQ on one node detects= , the other node stops updating it=E2=80=99s stamps, it performs failover pr= ocedure. It=E2=80=99s quite a questionable, rude way, I can agree, and that=E2= =80=99s why I always write a warning to use the BeaST for testing only purpo= ses.=20 Best regards, Mike > 24 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 9:09, Karli Sj=C3=B6berg =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >=20 >> On Mon, 2018-04-23 at 13:11 -0400, Mike Tancsa wrote: >>> On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: >>>=20 >>> Hello Mike, >>>=20 >>> Thank you for your interest to my paper. I appreciate it very much! >>> Your error may be a consequence of the initial HA misconfiguration. >>> What is in your /boot/loader.conf? Although the described config is >>> quite simple, I can recheck the instruction in my paper in a couple >>> of weeks only, unfortunately I=E2=80=99m on vacation right now. >=20 > [snip] >=20 > I read your articles on CTL HA, BQ and BeaST, and just wanted to say > they are amazing, good job! >=20 > One thing I=C2=B4m wondering about though is if you can claim HA with just= > two nodes, usually you need at least three, where the third is a tie- > breaker. Otherwise with your current setup, both systems may loose > contact with each other while both still being powered on, leading to > potential split brain situations. What are your thoughts about that? >=20 > /K From owner-freebsd-fs@freebsd.org Tue Apr 24 11:00:50 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D54BF89BEE for ; Tue, 24 Apr 2018 11:00:50 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A3FC82550 for ; Tue, 24 Apr 2018 11:00:49 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 0E9A0431FE; Tue, 24 Apr 2018 13:00:41 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id j-8-TTwkc0WK; Tue, 24 Apr 2018 13:00:39 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 98A0C431FF; Tue, 24 Apr 2018 13:00:39 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 98A0C431FF X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id o7HiSVf3uA5S; Tue, 24 Apr 2018 13:00:39 +0200 (CEST) Received: from its-5cg60227mj (its-5cg60227mj.its.uu.se [130.238.131.18]) by mail.inparadise.se (Postfix) with ESMTPSA id 6C689431FE; Tue, 24 Apr 2018 13:00:39 +0200 (CEST) Message-ID: <1524567621.9560.65.camel@inparadise.se> Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Mikhail Zakharov Cc: "freebsd-fs@freebsd.org" , Mike Tancsa Date: Tue, 24 Apr 2018 13:00:21 +0200 In-Reply-To: <615DFFBB-239A-4350-B961-FD10D0C9A8DD@yahoo.com> References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <1524550160.1130.6.camel@inparadise.se> <615DFFBB-239A-4350-B961-FD10D0C9A8DD@yahoo.com> Organization: InParadise Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-x4uETt1Aeoun59RYDAqh" X-Mailer: Evolution 3.26.1-1 Mime-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 11:00:50 -0000 --=-x4uETt1Aeoun59RYDAqh Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2018-04-24 at 12:32 +0300, Mikhail Zakharov wrote: > Hi Karli, >=20 > Thank you, I=E2=80=99m just exploring the storage abilities of my preferr= ed > OS - FreeBSD.=20 >=20 > Three nodes are preferable to choose the quorum for sure, but my idea > was not to establish contacts between nodes. Instead of it, BQ uses a > small partition for the =E2=80=9Cquorum=E2=80=9D on the same space where = data volume > is located.=20 Yes, of course. But there=C2=B4s nothing you from having three nodes connected to the same partition and being able to make more accurate choices on when to take over? If one node stops updating stamps, take over. If two nodes stops updating, then the problem is likely network-related and _must not_ take over to avoid split brain. Something like that? /K > And if a node looses access to the quorum it means, it looses access > to the data volume too. Now, BQ runs on both nodes and both BQ > instances write stamps to the quorum partition. If for any reason BQ > on one node detects, the other node stops updating it=E2=80=99s stamps, i= t > performs failover procedure. It=E2=80=99s quite a questionable, rude way,= I > can agree, and that=E2=80=99s why I always write a warning to use the Bea= ST > for testing only purposes.=20 >=20 > Best regards, > Mike >=20 > > 24 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 9:09, Karli Sj=C3=B6berg > > =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): > >=20 > > > On Mon, 2018-04-23 at 13:11 -0400, Mike Tancsa wrote: > > > > On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: > > > >=20 > > > > Hello Mike, > > > >=20 > > > > Thank you for your interest to my paper. I appreciate it very > > > > much! > > > > Your error may be a consequence of the initial HA > > > > misconfiguration. > > > > What is in your /boot/loader.conf? Although the described > > > > config is > > > > quite simple, I can recheck the instruction in my paper in a > > > > couple > > > > of weeks only, unfortunately I=E2=80=99m on vacation right now. > >=20 > > [snip] > >=20 > > I read your articles on CTL HA, BQ and BeaST, and just wanted to > > say > > they are amazing, good job! > >=20 > > One thing I=C2=B4m wondering about though is if you can claim HA with > > just > > two nodes, usually you need at least three, where the third is a > > tie- > > breaker. Otherwise with your current setup, both systems may loose > > contact with each other while both still being powered on, leading > > to > > potential split brain situations. What are your thoughts about > > that? > >=20 > > /K >=20 >=20 --=-x4uETt1Aeoun59RYDAqh Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQEcBAABCAAGBQJa3w5FAAoJEBpo164N2cuRxRYH/A77xGcrCRgnaB0dvNAgifU2 kBkrQsFSkOpN5SP+JGsdWo4chYrNL7xdaDUy4VNCa1joElP8bgC+PIeUhgqdlOHd F2iolSy7IpXoC7R77kAxqXo4B0zny6rxLhFjwf5gWWAlkiXhswlqZPioPNGScNKc mwpfLzb2m4L9b0n8qwNy2aKb1yDMGP19cCILgPpTVL/yLoiLMI9TloPvh2q+gHP1 wzLI3rKFy6apGLERbAKEgQTVOHPQPbFAmVXlSigsWmOLDfvswwT9UlP4lszhf0sI 7/TFEsjtFAi987W32FecIhauE1+Dx3kNKuNcUX5HMYTsuRbU8iHe/rbWPyayM+8= =wDyl -----END PGP SIGNATURE----- --=-x4uETt1Aeoun59RYDAqh-- From owner-freebsd-fs@freebsd.org Tue Apr 24 11:04:35 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B66FF89D50 for ; Tue, 24 Apr 2018 11:04:35 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC2298276E for ; Tue, 24 Apr 2018 11:04:34 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 28D72431E9; Tue, 24 Apr 2018 13:04:33 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id MSd7xfvmqKGg; Tue, 24 Apr 2018 13:04:32 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 09C3B431F0; Tue, 24 Apr 2018 13:04:32 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 09C3B431F0 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id bGAlNz5nfUFW; Tue, 24 Apr 2018 13:04:31 +0200 (CEST) Received: from its-5cg60227mj (its-5cg60227mj.its.uu.se [130.238.131.18]) by mail.inparadise.se (Postfix) with ESMTPSA id E83CB431E9; Tue, 24 Apr 2018 13:04:30 +0200 (CEST) Message-ID: <1524567842.9560.66.camel@inparadise.se> Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Karli =?ISO-8859-1?Q?Sj=F6berg?= Reply-To: karli@inparadise.se To: Mikhail Zakharov Cc: "freebsd-fs@freebsd.org" Date: Tue, 24 Apr 2018 13:04:02 +0200 In-Reply-To: <1524567621.9560.65.camel@inparadise.se> References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <1524550160.1130.6.camel@inparadise.se> <615DFFBB-239A-4350-B961-FD10D0C9A8DD@yahoo.com> <1524567621.9560.65.camel@inparadise.se> Organization: InParadise Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-PWWqx37QHYO2cefLF1q8" X-Mailer: Evolution 3.26.1-1 Mime-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 11:04:35 -0000 --=-PWWqx37QHYO2cefLF1q8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2018-04-24 at 13:00 +0200, Karli Sj=C3=B6berg via freebsd-fs wrote: > On Tue, 2018-04-24 at 12:32 +0300, Mikhail Zakharov wrote: > > Hi Karli, > >=20 > > Thank you, I=E2=80=99m just exploring the storage abilities of my prefe= rred > > OS - FreeBSD.=20 > >=20 > > Three nodes are preferable to choose the quorum for sure, but my > > idea > > was not to establish contacts between nodes. Instead of it, BQ uses > > a > > small partition for the =E2=80=9Cquorum=E2=80=9D on the same space wher= e data > > volume > > is located.=20 >=20 > Yes, of course. But there=C2=B4s nothing you from having three nodes 's/nothing you/nothing stopping you/' > connected to the same partition and being able to make more accurate > choices on when to take over? >=20 > If one node stops updating stamps, take over. If two nodes stops > updating, then the problem is likely network-related and _must not_ > take over to avoid split brain. Something like that? >=20 > /K >=20 > > And if a node looses access to the quorum it means, it looses > > access > > to the data volume too. Now, BQ runs on both nodes and both BQ > > instances write stamps to the quorum partition. If for any reason > > BQ > > on one node detects, the other node stops updating it=E2=80=99s stamps,= it > > performs failover procedure. It=E2=80=99s quite a questionable, rude wa= y, I > > can agree, and that=E2=80=99s why I always write a warning to use the B= eaST > > for testing only purposes.=20 > >=20 > > Best regards, > > Mike > >=20 > > > 24 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 9:09, Karli Sj=C3=B6berg = > > > =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): > > >=20 > > > > On Mon, 2018-04-23 at 13:11 -0400, Mike Tancsa wrote: > > > > > On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: > > > > >=20 > > > > > Hello Mike, > > > > >=20 > > > > > Thank you for your interest to my paper. I appreciate it very > > > > > much! > > > > > Your error may be a consequence of the initial HA > > > > > misconfiguration. > > > > > What is in your /boot/loader.conf? Although the described > > > > > config is > > > > > quite simple, I can recheck the instruction in my paper in a > > > > > couple > > > > > of weeks only, unfortunately I=E2=80=99m on vacation right now. > > >=20 > > > [snip] > > >=20 > > > I read your articles on CTL HA, BQ and BeaST, and just wanted to > > > say > > > they are amazing, good job! > > >=20 > > > One thing I=C2=B4m wondering about though is if you can claim HA with > > > just > > > two nodes, usually you need at least three, where the third is a > > > tie- > > > breaker. Otherwise with your current setup, both systems may > > > loose > > > contact with each other while both still being powered on, > > > leading > > > to > > > potential split brain situations. What are your thoughts about > > > that? > > >=20 > > > /K --=-PWWqx37QHYO2cefLF1q8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQEcBAABCAAGBQJa3w8iAAoJEBpo164N2cuRlWUIAIBoZTRb3CxKi9My7V+3+2Ca Wac9y1RvOpKAyUo77s+bvUhkfxMeHyFXUkRsvlVbW9rZqWSPYO4GfLAi4EBYMxYL O1PVt7Ks+0ASGbaL2ztGtoaqOuXCRkUl81YYdtO8loEGC1ipKs41FZjSdhV80z9K o1PvQ2T8/Jgq+inpzc1fNfvadgMO69VjiVCJo/lDk/FJhTQUwyADOUsXOiSdcP2z dxp8PtJPmm/h73MvctOgwT1JmkcpG6UDecPqF2ZINH2usxAdIrzuAVTnsQb+KxLg r3WP9/uay4KonceUiesErTLBfA3slTCjJN3dR82ujKw2mQXs9Y5czZOy15W23VM= =vuBA -----END PGP SIGNATURE----- --=-PWWqx37QHYO2cefLF1q8-- From owner-freebsd-fs@freebsd.org Tue Apr 24 11:21:11 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F1EAF8B957 for ; Tue, 24 Apr 2018 11:21:11 +0000 (UTC) (envelope-from zmey20000@yahoo.com) Received: from sonic307-37.consmr.mail.ne1.yahoo.com (sonic307-37.consmr.mail.ne1.yahoo.com [66.163.190.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28D0886C76 for ; Tue, 24 Apr 2018 11:21:11 +0000 (UTC) (envelope-from zmey20000@yahoo.com) X-YMail-OSG: Tk8URDUVM1mTy_u8s7MJ23Ydn0mBy.KTvKzWClcJslyv7ifgHeY3TAmb_u4mh40 tjt3ZY4BBYExvBUdWWuvxv8YukY9mRF8OIIE6CpSC2j7Ioj2diq.sz5v6L_R5BuWePTVeZ376WYa tTjoKccWLVhaLT1ROY5c4Q34p263hUb1UFrqyfI9XASi.xh7VGFhFS2NRxr8ZcQlxeWBcsMb0iaR tCOqgXsQ7WJ7gvvUDGYgfkL0LOIQ6bTpXIVipDr4IQ_xwU19GsqNl6zI_6MCPJAS22vbujhyrzxe HRWlqGkJIesdzaYpVbbQmEz4ksTKUlkmNSPjXFG20XcDxQ.IQpgpNV3XOdK0niW59antv_8QSv_U N1TrX7dqGebLGQc9h_..Zw3d.Zn3Zc3Rx0J0ooXWaxRL2sWM.WQPcJ4NfMbAOp3tF7Fl2Yzgw2jP 0_4bErwrXI3yZx.qwtiqjUh12Q6OtE1zuhXBrfcLagWakUo1_SzPolvNHlx8Rjl_0edQ.WGEThyZ 8vHY8_LPa_WZanXmoTwUsWvk6xSdjVRLu_nhtz9yuXIqpDzg0Ye_GcNqovDd2LrGQXYu26bZAk_P Yv8lOC4ZR3juliO7GXzH9zIs9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.ne1.yahoo.com with HTTP; Tue, 24 Apr 2018 11:21:10 +0000 Received: from athe-hcsd-peer.customers.otenet.gr (EHLO [192.168.140.13]) ([195.167.103.234]) by smtp425.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 6e0247872a84857e5fb2b1e9a0b67144; Tue, 24 Apr 2018 11:17:09 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Mikhail Zakharov X-Mailer: iPhone Mail (15E216) In-Reply-To: <1524567621.9560.65.camel@inparadise.se> Date: Tue, 24 Apr 2018 14:17:07 +0300 Cc: "freebsd-fs@freebsd.org" , Mike Tancsa Content-Transfer-Encoding: quoted-printable Message-Id: References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <1524550160.1130.6.camel@inparadise.se> <615DFFBB-239A-4350-B961-FD10D0C9A8DD@yahoo.com> <1524567621.9560.65.camel@inparadise.se> To: karli@inparadise.se X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 11:21:11 -0000 I don=E2=80=99t think, adding a third node will add accuracy to BQ, and ther= e is no network connection involved with BQ usage. Also if both nodes have s= toped updating stamps, the system is dead. But the third node may be configu= red to handle all other issues related to any interconnections and death of B= Q itself :) WBR, Mike > 24 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 14:00, Karli Sj=C3=B6berg =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >=20 >> On Tue, 2018-04-24 at 12:32 +0300, Mikhail Zakharov wrote: >> Hi Karli, >>=20 >> Thank you, I=E2=80=99m just exploring the storage abilities of my preferr= ed >> OS - FreeBSD.=20 >>=20 >> Three nodes are preferable to choose the quorum for sure, but my idea >> was not to establish contacts between nodes. Instead of it, BQ uses a >> small partition for the =E2=80=9Cquorum=E2=80=9D on the same space where d= ata volume >> is located.=20 >=20 > Yes, of course. But there=C2=B4s nothing you from having three nodes > connected to the same partition and being able to make more accurate > choices on when to take over? >=20 > If one node stops updating stamps, take over. If two nodes stops > updating, then the problem is likely network-related and _must not_ > take over to avoid split brain. Something like that? >=20 > /K >=20 >> And if a node looses access to the quorum it means, it looses access >> to the data volume too. Now, BQ runs on both nodes and both BQ >> instances write stamps to the quorum partition. If for any reason BQ >> on one node detects, the other node stops updating it=E2=80=99s stamps, i= t >> performs failover procedure. It=E2=80=99s quite a questionable, rude way,= I >> can agree, and that=E2=80=99s why I always write a warning to use the Bea= ST >> for testing only purposes.=20 >>=20 >> Best regards, >> Mike >>=20 >>> 24 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 9:09, Karli Sj=C3=B6berg >>> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >>>=20 >>>>> On Mon, 2018-04-23 at 13:11 -0400, Mike Tancsa wrote: >>>>> On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: >>>>>=20 >>>>> Hello Mike, >>>>>=20 >>>>> Thank you for your interest to my paper. I appreciate it very >>>>> much! >>>>> Your error may be a consequence of the initial HA >>>>> misconfiguration. >>>>> What is in your /boot/loader.conf? Although the described >>>>> config is >>>>> quite simple, I can recheck the instruction in my paper in a >>>>> couple >>>>> of weeks only, unfortunately I=E2=80=99m on vacation right now. >>>=20 >>> [snip] >>>=20 >>> I read your articles on CTL HA, BQ and BeaST, and just wanted to >>> say >>> they are amazing, good job! >>>=20 >>> One thing I=C2=B4m wondering about though is if you can claim HA with >>> just >>> two nodes, usually you need at least three, where the third is a >>> tie- >>> breaker. Otherwise with your current setup, both systems may loose >>> contact with each other while both still being powered on, leading >>> to >>> potential split brain situations. What are your thoughts about >>> that? >>>=20 >>> /K >>=20 From owner-freebsd-fs@freebsd.org Tue Apr 24 11:41:49 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3584DF914F0 for ; Tue, 24 Apr 2018 11:41:49 +0000 (UTC) (envelope-from zmey20000@yahoo.com) Received: from sonic305-47.consmr.mail.ne1.yahoo.com (sonic305-47.consmr.mail.ne1.yahoo.com [66.163.185.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB5C46C2B4 for ; Tue, 24 Apr 2018 11:41:48 +0000 (UTC) (envelope-from zmey20000@yahoo.com) X-YMail-OSG: y_Ci1_YVM1k7RWheLZRLJ_8BL6GaxXz.JKsWvjsPcJJctNfXKLCV3fu_roYTLGP kNdwlsBeWYfiBjLRUFf4tw1a3xdOGinbXTDHdz9LlRMg6ilaQmPHGGYIQUbdac85X72Lu.lGkl9t 3GCC_KVxceJVBjZMyOPniOu58GluhCYAhBXVoqyWggbKk8WnuzpC7i.zR5yZlStvbi_S1FfmQx1a B6M2mShtCmm5oE9_Iv07zhtMXVN5pLIA509bOQvyiBx8tBB5jQ1NktZ1ncqJwIxlIgZJwOoge9ws Juxxb11jIaRM9Q.wo2LqBRvZe3KGHWIVRS1.qWKurcMwN5.kcBxmpRf0m4kZ6S7UVTjfuLcp3_Tt vRz5g7mLhZ7p5ErhF4Y7SnIo1vqUTD35PJ6qPkBkqcxt6jftbb2Zm67OaMMHI563ruQL1lNEBkqu A3j18VfEXBnisMkVxg7w_2wtv9U537IT4kBqCwvZfC_8Oemg.m.o2389m_lpfBeyu0AkfkmHeuH4 BaXqxerolNW0ivRSBjEBInU8qmRq28Co8XAVmeoG_4x7O0_cA7.uWIibbwDpRfHt18WTcr8_XwAx p9olip2.1Yndd0JCwuKVTt0Az Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.ne1.yahoo.com with HTTP; Tue, 24 Apr 2018 11:41:47 +0000 Received: from athe-hcsd-peer.customers.otenet.gr (EHLO [192.168.130.11]) ([195.167.103.234]) by smtp425.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 1c83dc9916ea31e6b245869d552b8206; Tue, 24 Apr 2018 11:27:40 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN From: Mikhail Zakharov X-Mailer: iPad Mail (15E216) In-Reply-To: <1524567842.9560.66.camel@inparadise.se> Date: Tue, 24 Apr 2018 14:27:38 +0300 Cc: "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <56E4773F-4EAD-47EB-A803-38BFCD8C63F8@yahoo.com> References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <1524550160.1130.6.camel@inparadise.se> <615DFFBB-239A-4350-B961-FD10D0C9A8DD@yahoo.com> <1524567621.9560.65.camel@inparadise.se> <1524567842.9560.66.camel@inparadise.se> To: karli@inparadise.se X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 11:41:49 -0000 Ah, and unfortunately CTL HA is two-node cluster, as I remember, there is no= possibility to add the third one. So the third node is an external arbiter i= n that case. > 24 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 14:04, Karli Sj=C3=B6berg =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >=20 >> On Tue, 2018-04-24 at 13:00 +0200, Karli Sj=C3=B6berg via freebsd-fs wrot= e: >>> On Tue, 2018-04-24 at 12:32 +0300, Mikhail Zakharov wrote: >>> Hi Karli, >>>=20 >>> Thank you, I=E2=80=99m just exploring the storage abilities of my prefer= red >>> OS - FreeBSD.=20 >>>=20 >>> Three nodes are preferable to choose the quorum for sure, but my >>> idea >>> was not to establish contacts between nodes. Instead of it, BQ uses >>> a >>> small partition for the =E2=80=9Cquorum=E2=80=9D on the same space where= data >>> volume >>> is located.=20 >>=20 >> Yes, of course. But there=C2=B4s nothing you from having three nodes >=20 > 's/nothing you/nothing stopping you/' >=20 >> connected to the same partition and being able to make more accurate >> choices on when to take over? >>=20 >> If one node stops updating stamps, take over. If two nodes stops >> updating, then the problem is likely network-related and _must not_ >> take over to avoid split brain. Something like that? >>=20 >> /K >>=20 >>> And if a node looses access to the quorum it means, it looses >>> access >>> to the data volume too. Now, BQ runs on both nodes and both BQ >>> instances write stamps to the quorum partition. If for any reason >>> BQ >>> on one node detects, the other node stops updating it=E2=80=99s stamps, i= t >>> performs failover procedure. It=E2=80=99s quite a questionable, rude way= , I >>> can agree, and that=E2=80=99s why I always write a warning to use the Be= aST >>> for testing only purposes.=20 >>>=20 >>> Best regards, >>> Mike >>>=20 >>>> 24 =D0=B0=D0=BF=D1=80. 2018 =D0=B3., =D0=B2 9:09, Karli Sj=C3=B6berg >>>> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >>>>=20 >>>>>> On Mon, 2018-04-23 at 13:11 -0400, Mike Tancsa wrote: >>>>>> On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: >>>>>>=20 >>>>>> Hello Mike, >>>>>>=20 >>>>>> Thank you for your interest to my paper. I appreciate it very >>>>>> much! >>>>>> Your error may be a consequence of the initial HA >>>>>> misconfiguration. >>>>>> What is in your /boot/loader.conf? Although the described >>>>>> config is >>>>>> quite simple, I can recheck the instruction in my paper in a >>>>>> couple >>>>>> of weeks only, unfortunately I=E2=80=99m on vacation right now. >>>>=20 >>>> [snip] >>>>=20 >>>> I read your articles on CTL HA, BQ and BeaST, and just wanted to >>>> say >>>> they are amazing, good job! >>>>=20 >>>> One thing I=C2=B4m wondering about though is if you can claim HA with >>>> just >>>> two nodes, usually you need at least three, where the third is a >>>> tie- >>>> breaker. Otherwise with your current setup, both systems may >>>> loose >>>> contact with each other while both still being powered on, >>>> leading >>>> to >>>> potential split brain situations. What are your thoughts about >>>> that? >>>>=20 >>>> /K From owner-freebsd-fs@freebsd.org Tue Apr 24 12:36:27 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1816CFA2C43 for ; Tue, 24 Apr 2018 12:36:27 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670063.outbound.protection.outlook.com [40.107.67.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8034277D5B for ; Tue, 24 Apr 2018 12:36:26 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM (52.132.66.153) by YQBPR0101MB1587.CANPRD01.PROD.OUTLOOK.COM (52.132.69.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.12; Tue, 24 Apr 2018 12:36:25 +0000 Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a]) by YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a%13]) with mapi id 15.20.0696.019; Tue, 24 Apr 2018 12:36:25 +0000 From: Rick Macklem To: "freebsd-fs@freebsd.org" , "karli@inparadise.se" Subject: Re: pNFS server completed Thread-Topic: pNFS server completed Thread-Index: AQHT2n6KBZhPilerAEebAsjWoa5XxaQOAmgAgADSTViAAJ1SAIAAZzU1 Date: Tue, 24 Apr 2018 12:36:24 +0000 Message-ID: References: ,<1524471526.9560.51.camel@inparadise.se> , <1524550472.1130.10.camel@inparadise.se> In-Reply-To: <1524550472.1130.10.camel@inparadise.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQBPR0101MB1587; 7:hfsihZCSVFxSqc3X7nqu7TMN/zINJaR+kjDKwldl9MzhSfUnhDGvJifV0ESAjwfEnHoU4eQHj7RYPJqQNFkqZetWQOEXWKH+SJFzmNoQBHVwYCv0VwZbTZ6rxxZlnnB6/Dqu6dLGLAvqO5FyCdDT3LUAcCU8PkQ4feg0p/PpshtT023lNjNmpVqQN1BV9INx26/HxTKp9e7yoH9o36RHA4lBEsBj4KIQnIekmQD+wTpJ2fYZfO+4KvTvJZ6b9PTU x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(2017052603328)(7153060)(7193020); SRVR:YQBPR0101MB1587; x-ms-traffictypediagnostic: YQBPR0101MB1587: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(10201501046)(3231232)(944501410)(52105095)(3002001)(93006095)(93001095)(6041310)(20161123564045)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(6072148)(201708071742011); SRVR:YQBPR0101MB1587; BCL:0; PCL:0; RULEID:; SRVR:YQBPR0101MB1587; x-forefront-prvs: 0652EA5565 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(366004)(376002)(346002)(39860400002)(39380400002)(189003)(199004)(6436002)(446003)(11346002)(53936002)(9686003)(2900100001)(5250100002)(99286004)(81156014)(81166006)(8936002)(106356001)(186003)(476003)(8676002)(86362001)(68736007)(55016002)(3480700004)(26005)(74482002)(6246003)(5890100001)(229853002)(345774005)(93886005)(97736004)(478600001)(3660700001)(105586002)(305945005)(3280700002)(2906002)(74316002)(14454004)(59450400001)(102836004)(6506007)(486006)(110136005)(76176011)(33656002)(316002)(786003)(7696005)(2501003)(25786009)(5660300001)(7116003); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB1587; H:YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-microsoft-antispam-message-info: Ky9WAGFJWkc+2I9OchMHsz9roT7Jgtbs8IVe/aeVLcF9+ewWZqcAbHBn68No+2uri9dZE/ncxpx86IibmHoGST1PSc+PfKa+vYvG6qzAh+F0q42hfQ7BR252/GBhS4cLDwhJrkjaWCJvWukvw2q80tFKg9sb8DMvX0gbxCtHG8bGS4Y0c2MryOELwOUehb9Z spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: a9e0bffa-c041-4066-fc30-08d5a9dffe3d X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: a9e0bffa-c041-4066-fc30-08d5a9dffe3d X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Apr 2018 12:36:24.9975 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB1587 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 12:36:27 -0000 Karli Sj=F6berg wrote: [stuff snipped] >OK, I think I understand, thank you for the explanation! > >So you could say that this enhancement is more about the performance >benifit of scaling out, rather than resiliance/tolerance? Yes, although without the DS mirroring (which is optional and doesn't have = to be done), you would go from one SPOF to N SPOFs. To be honest, this pNFS service will not be useful for many. Only people wh= o need to scale out beyond the limits of one NFS server and find it more conv= enient that creating a second NFS server. Improving resiliance/tolerance for an NFS server is a separate topic (and w= ould apply to the MDS of this pNFS service as well). I don't have any expertise w.r.t. this, but something like HA (which I just= learned about from the email thread) might be appropriate. If you can build a stora= ge subsystem with internal redundancy (raid at the drive level) and then have two servers "attached" to the storage subsystem so that it can fail over from the 1st to 2nd server, that would be a good start. With that, you would just need to have something that would tell the nfs servers to start/stop when the HA failover happens, I think? This is a really interesting problem, but not one for me. Of course if some= one gets to the point where it needs an nfsd tweak to make the fail over work, = I could help with that. Thanks for your interest, rick From owner-freebsd-fs@freebsd.org Tue Apr 24 15:45:28 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E49A2FA9B45 for ; Tue, 24 Apr 2018 15:45:27 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 62B8F856EF for ; Tue, 24 Apr 2018 15:45:26 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 6FE644320B; Tue, 24 Apr 2018 17:45:24 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id cfGMu9tnwUXe; Tue, 24 Apr 2018 17:45:22 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 9F623431FF; Tue, 24 Apr 2018 17:45:22 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 9F623431FF X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id SAh3HJQalEu8; Tue, 24 Apr 2018 17:45:22 +0200 (CEST) Received: from nexus6p.inparadise.se (nexus6p.inparadise.se [172.16.1.122]) by mail.inparadise.se (Postfix) with ESMTPSA id 709E9431FA; Tue, 24 Apr 2018 17:45:22 +0200 (CEST) Date: Tue, 24 Apr 2018 17:45:21 +0200 User-Agent: K-9 Mail for Android In-Reply-To: <56E4773F-4EAD-47EB-A803-38BFCD8C63F8@yahoo.com> References: <4cb4aa83-bd49-0c20-4e41-c11c682b0570@sentex.net> <1e1e7cd5-0797-c168-fbce-a36edc6a432e@sentex.net> <1524550160.1130.6.camel@inparadise.se> <615DFFBB-239A-4350-B961-FD10D0C9A8DD@yahoo.com> <1524567621.9560.65.camel@inparadise.se> <1524567842.9560.66.camel@inparadise.se> <56E4773F-4EAD-47EB-A803-38BFCD8C63F8@yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: ctl_isc_lun_sync: Received conflicting HA LUN To: Mikhail Zakharov CC: "freebsd-fs@freebsd.org" From: =?ISO-8859-1?Q?Karli_Sj=F6berg?= Message-ID: X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 15:45:28 -0000 Mikhail Zakharov skrev: (24 april 2018 13:27:38 CE= ST) >Ah, and unfortunately CTL HA is two-node cluster, as I remember, there >is no possibility to add the third one=2E So the third node is an >external arbiter in that case=2E Yes, Arbiter, exactly the term I was looking for:) So it would be up to th= e Arbiter to decide whether it really is a node problem or a network proble= m- a tie-breaker=2E Would that be feasible in your software? /K > > >> 24 =D0=B0=D0=BF=D1=80=2E 2018 =D0=B3=2E, =D0=B2 14:04, Karli Sj=C3=B6be= rg >=D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >>=20 >>> On Tue, 2018-04-24 at 13:00 +0200, Karli Sj=C3=B6berg via freebsd-fs >wrote: >>>> On Tue, 2018-04-24 at 12:32 +0300, Mikhail Zakharov wrote: >>>> Hi Karli, >>>>=20 >>>> Thank you, I=E2=80=99m just exploring the storage abilities of my pre= ferred >>>> OS - FreeBSD=2E=20 >>>>=20 >>>> Three nodes are preferable to choose the quorum for sure, but my >>>> idea >>>> was not to establish contacts between nodes=2E Instead of it, BQ uses >>>> a >>>> small partition for the =E2=80=9Cquorum=E2=80=9D on the same space wh= ere data >>>> volume >>>> is located=2E=20 >>>=20 >>> Yes, of course=2E But there=C2=B4s nothing you from having three nodes >>=20 >> 's/nothing you/nothing stopping you/' >>=20 >>> connected to the same partition and being able to make more accurate >>> choices on when to take over? >>>=20 >>> If one node stops updating stamps, take over=2E If two nodes stops >>> updating, then the problem is likely network-related and _must not_ >>> take over to avoid split brain=2E Something like that? >>>=20 >>> /K >>>=20 >>>> And if a node looses access to the quorum it means, it looses >>>> access >>>> to the data volume too=2E Now, BQ runs on both nodes and both BQ >>>> instances write stamps to the quorum partition=2E If for any reason >>>> BQ >>>> on one node detects, the other node stops updating it=E2=80=99s stamp= s, it >>>> performs failover procedure=2E It=E2=80=99s quite a questionable, rud= e way, I >>>> can agree, and that=E2=80=99s why I always write a warning to use the= BeaST >>>> for testing only purposes=2E=20 >>>>=20 >>>> Best regards, >>>> Mike >>>>=20 >>>>> 24 =D0=B0=D0=BF=D1=80=2E 2018 =D0=B3=2E, =D0=B2 9:09, Karli Sj=C3=B6= berg >>>>> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >>>>>=20 >>>>>>> On Mon, 2018-04-23 at 13:11 -0400, Mike Tancsa wrote: >>>>>>> On 4/23/2018 12:59 PM, Mikhail Zakharov wrote: >>>>>>>=20 >>>>>>> Hello Mike, >>>>>>>=20 >>>>>>> Thank you for your interest to my paper=2E I appreciate it very >>>>>>> much! >>>>>>> Your error may be a consequence of the initial HA >>>>>>> misconfiguration=2E >>>>>>> What is in your /boot/loader=2Econf? Although the described >>>>>>> config is >>>>>>> quite simple, I can recheck the instruction in my paper in a >>>>>>> couple >>>>>>> of weeks only, unfortunately I=E2=80=99m on vacation right now=2E >>>>>=20 >>>>> [snip] >>>>>=20 >>>>> I read your articles on CTL HA, BQ and BeaST, and just wanted to >>>>> say >>>>> they are amazing, good job! >>>>>=20 >>>>> One thing I=C2=B4m wondering about though is if you can claim HA wit= h >>>>> just >>>>> two nodes, usually you need at least three, where the third is a >>>>> tie- >>>>> breaker=2E Otherwise with your current setup, both systems may >>>>> loose >>>>> contact with each other while both still being powered on, >>>>> leading >>>>> to >>>>> potential split brain situations=2E What are your thoughts about >>>>> that? >>>>>=20 >>>>> /K --=20 Skickat fr=C3=A5n min Android-enhet med K-9 Mail=2E Urs=C3=A4kta min f=C3= =A5ordighet=2E From owner-freebsd-fs@freebsd.org Tue Apr 24 15:47:02 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F981FA9C0D for ; Tue, 24 Apr 2018 15:47:02 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D60C3857F8 for ; Tue, 24 Apr 2018 15:47:01 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 36069431FA; Tue, 24 Apr 2018 17:47:00 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id L47suSj99TAW; Tue, 24 Apr 2018 17:46:58 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 3EC16431FF; Tue, 24 Apr 2018 17:46:58 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 3EC16431FF X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id tFTECs-RLnH8; Tue, 24 Apr 2018 17:46:58 +0200 (CEST) Received: from nexus6p.inparadise.se (nexus6p.inparadise.se [172.16.1.122]) by mail.inparadise.se (Postfix) with ESMTPSA id 1A1E6431FA; Tue, 24 Apr 2018 17:46:58 +0200 (CEST) Date: Tue, 24 Apr 2018 17:46:57 +0200 User-Agent: K-9 Mail for Android In-Reply-To: References: , <1524471526.9560.51.camel@inparadise.se> , <1524550472.1130.10.camel@inparadise.se> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: pNFS server completed To: Rick Macklem , "freebsd-fs@freebsd.org" From: =?ISO-8859-1?Q?Karli_Sj=F6berg?= Message-ID: <7A931945-0783-48EF-8743-B7F7261E9343@inparadise.se> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 15:47:02 -0000 Rick Macklem skrev: (24 april 2018 14:36:24 CEST) >Karli Sj=C3=B6berg wrote: >[stuff snipped] >>OK, I think I understand, thank you for the explanation! >> >>So you could say that this enhancement is more about the performance >>benifit of scaling out, rather than resiliance/tolerance? >Yes, although without the DS mirroring (which is optional and doesn't >have to >be done), you would go from one SPOF to N SPOFs=2E > >To be honest, this pNFS service will not be useful for many=2E Only >people who >need to scale out beyond the limits of one NFS server and find it more >convenient >that creating a second NFS server=2E > >Improving resiliance/tolerance for an NFS server is a separate topic >(and would >apply to the MDS of this pNFS service as well)=2E >I don't have any expertise w=2Er=2Et=2E this, but something like HA (whic= h I >just learned >about from the email thread) might be appropriate=2E If you can build a >storage >subsystem with internal redundancy (raid at the drive level) and then >have >two servers "attached" to the storage subsystem so that it can fail >over >from the 1st to 2nd server, that would be a good start=2E >With that, you would just need to have something that would tell the >nfs >servers to start/stop when the HA failover happens, I think? >This is a really interesting problem, but not one for me=2E Of course if >someone >gets to the point where it needs an nfsd tweak to make the fail over >work, I >could help with that=2E > >Thanks for your interest, rick And thank you for working on this! /K --=20 Skickat fr=C3=A5n min Android-enhet med K-9 Mail=2E Urs=C3=A4kta min f=C3= =A5ordighet=2E From owner-freebsd-fs@freebsd.org Wed Apr 25 04:54:36 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66C43FBCD29 for ; Wed, 25 Apr 2018 04:54:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id F035477F68 for ; Wed, 25 Apr 2018 04:54:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id AA8D0FBCD26; Wed, 25 Apr 2018 04:54:35 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9957AFBCD25 for ; Wed, 25 Apr 2018 04:54:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3176077F43 for ; Wed, 25 Apr 2018 04:54:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 66FB122DB7 for ; Wed, 25 Apr 2018 04:54:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3P4sYtC019526 for ; Wed, 25 Apr 2018 04:54:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3P4sYMd019525 for fs@FreeBSD.org; Wed, 25 Apr 2018 04:54:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 227740] concurrent zfs management operations may lead to a race/subsystem locking Date: Wed, 25 Apr 2018 04:54:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 04:54:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227740 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Apr 25 05:56:29 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C134FBDAF2 for ; Wed, 25 Apr 2018 05:56:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D190885D91 for ; Wed, 25 Apr 2018 05:56:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8F5E3FBDAF1; Wed, 25 Apr 2018 05:56:28 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C273FBDAF0 for ; Wed, 25 Apr 2018 05:56:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B53485D6F for ; Wed, 25 Apr 2018 05:56:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 370FE235A7 for ; Wed, 25 Apr 2018 05:56:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3P5uRsH077479 for ; Wed, 25 Apr 2018 05:56:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3P5uR3F077478 for fs@FreeBSD.org; Wed, 25 Apr 2018 05:56:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 222929] ZFS ARC stats have wrong count Date: Wed, 25 Apr 2018 05:56:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: naito.yuichiro@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 05:56:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222929 --- Comment #9 from Yuichiro NAITO --- Created attachment 192787 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D192787&action= =3Dedit zfs-stat-resolved.log --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Apr 25 05:56:47 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77DB1FBDB0A for ; Wed, 25 Apr 2018 05:56:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 112FB86065 for ; Wed, 25 Apr 2018 05:56:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id C660EFBDB09; Wed, 25 Apr 2018 05:56:46 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2AA2FBDB08 for ; Wed, 25 Apr 2018 05:56:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B7DC8603F for ; Wed, 25 Apr 2018 05:56:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 961B6235AC for ; Wed, 25 Apr 2018 05:56:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3P5uj1o077892 for ; Wed, 25 Apr 2018 05:56:45 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3P5ujei077891 for fs@FreeBSD.org; Wed, 25 Apr 2018 05:56:45 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 222929] ZFS ARC stats have wrong count Date: Wed, 25 Apr 2018 05:56:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: naito.yuichiro@gmail.com X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 05:56:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222929 Yuichiro NAITO changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|New |Closed --- Comment #10 from Yuichiro NAITO --- I checked that r332523 has fixed this problem. After upgrading r332641, I ran my script described in comment:3 again. It shows as 'zfs-stat-resolved.log'. Cache Hit Ratio increases in each loop as I expected. I will close this PR. Thank you! --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Apr 26 07:28:16 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AEE9AFA3F1C; Thu, 26 Apr 2018 07:28:16 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 00D2875191; Thu, 26 Apr 2018 07:28:15 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from bsdrookie.norma.com. (asterisk.enaza.ru [91.237.76.254]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id w3Q7S9dF027301 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 26 Apr 2018 12:28:09 +0500 (YEKT) (envelope-from eugene@zhegan.in) To: freebsd-stable@freebsd.org Cc: freebsd-fs@freebsd.org From: "Eugene M. Zheganin" Subject: clear old pools remains from active vdevs Message-ID: Date: Thu, 26 Apr 2018 12:28:09 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Spamd-Result: default: False [-998.10 / 25.00] WHITELISTED_IPS(-999.00)[91.237.76.254] MIME_GOOD(-0.10)[text/plain] AUTH_NA(1.00)[] R_DKIM_NA(0.00)[] ASN(0.00)[asn:57973, ipnet:91.237.76.0/24, country:RU] RCVD_TLS_ALL(0.00)[] DMARC_NA(0.00)[zhegan.in] TO_MATCH_ENVRCPT_ALL(0.00)[] TO_DN_NONE(0.00)[] RCPT_COUNT_TWO(0.00)[2] RCVD_COUNT_TWO(0.00)[2] MID_RHS_MATCH_FROM(0.00)[] R_SPF_NA(0.00)[] FROM_HAS_DN(0.00)[] ARC_NA(0.00)[] FROM_EQ_ENVFROM(0.00)[] IP_SCORE(0.00)[ip: (-9.89), ipnet: 91.237.76.0/24(-7.83), asn: 57973(-4.90), country: RU(0.13)] X-Rspamd-Server: localhost X-Rspamd-Scan-Time: 1.79 X-Rspamd-Queue-ID: w3Q7S9dF027301 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 07:28:16 -0000 Hello, I have some active vdev disk members that used to be in pool that clearly have not beed destroyed properly, so I'm seeing in a "zpool import" output something like # zpool import pool: zroot id: 14767697319309030904 state: UNAVAIL status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. see: http://illumos.org/msg/ZFS-8000-EY config: zroot UNAVAIL insufficient replicas mirror-0 UNAVAIL insufficient replicas 5291726022575795110 UNAVAIL cannot open 2933754417879630350 UNAVAIL cannot open pool: esx id: 8314148521324214892 state: UNAVAIL status: The pool was last accessed by another system. action: The pool cannot be imported due to damaged devices or data. see: http://illumos.org/msg/ZFS-8000-EY config: esx UNAVAIL insufficient replicas mirror-0 UNAVAIL insufficient replicas 10170732803757341731 UNAVAIL cannot open 9207269511643803468 UNAVAIL cannot open is there any _safe_ way to get rid of this ? I'm asking because a gptzfsboot loader in recent -STABLE stumbles upon this and refuses to boot the system (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227772). The workaround is to use the 11.1 loader, but I'm afraid this behavior will now be the intended one. Eugene. From owner-freebsd-fs@freebsd.org Thu Apr 26 07:50:28 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05CABFA47ED; Thu, 26 Apr 2018 07:50:28 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f48.google.com (mail-lf0-f48.google.com [209.85.215.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 549F17AAA1; Thu, 26 Apr 2018 07:50:27 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f48.google.com with SMTP id h197-v6so1878363lfg.11; Thu, 26 Apr 2018 00:50:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=gge1vlSNmEqNc7dBcPyGaa0iUGie1vC2NsgndxPkosw=; b=FU5fBfZALOg4iHk0I3LEmYsDOlx8sLsTxFm2zl40I128+ifjg4PGfJkLfyx2+JtsLA Y6VFTrtb75DhOSRTqi5I/pto6EpRY0axJQrTCN8mORFgwTZlWBs2HAWgA7DzXppE401P zSHOx6OTiHLUfnFV6mbdeV/Bn0akIPV878C6zkq/aNM0eMVeX/K0YIEGU59DzmuS9rTI V+5KwH0LXgDeBuZ08DNC6qSeCaP5KIjbPKki8xMEr4EdzQaem4ehulLSfDdQj9euSmrW 7vxlUsCdWvAeMIGNqQmKgCM0r5oyrbEnQvYyOyc3WT0USK6QA31fQIe6+0o6m08pQlS/ Lj1A== X-Gm-Message-State: ALQs6tD4ZFXmEeWifZGJCWUL7C2m7cyvBtoq/+IU0za6iKIomJVgeTZd C01drf2sVxw1/OlNruAOliBG1NWr X-Google-Smtp-Source: AB8JxZoEZ3koAm3M0cNi/9bX/PVjdHVFKu727h0GYifFKwYb7MogVF0sP3eSTP1wqVUOEoGYjynKFQ== X-Received: by 10.46.5.200 with SMTP id 191mr1843847ljf.136.1524729019823; Thu, 26 Apr 2018 00:50:19 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id e23-v6sm4308156lfi.58.2018.04.26.00.50.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Apr 2018 00:50:19 -0700 (PDT) Subject: Re: clear old pools remains from active vdevs To: "Eugene M. Zheganin" , freebsd-stable@freebsd.org Cc: freebsd-fs@freebsd.org References: From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABzR5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz7CwZQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryM7BTQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAcLBfAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <92b45b63-48c0-e9fe-c7c8-3ff24f37bf1f@FreeBSD.org> Date: Thu, 26 Apr 2018 10:50:18 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 07:50:28 -0000 On 26/04/2018 10:28, Eugene M. Zheganin wrote: > Hello, > > > I have some active vdev disk members that used to be in pool that clearly have > not beed destroyed properly, so I'm seeing in a "zpool import" output something > like > > > # zpool import >    pool: zroot >      id: 14767697319309030904 >   state: UNAVAIL >  status: The pool was last accessed by another system. >  action: The pool cannot be imported due to damaged devices or data. >    see: http://illumos.org/msg/ZFS-8000-EY >  config: > >         zroot                    UNAVAIL  insufficient replicas >           mirror-0               UNAVAIL  insufficient replicas >             5291726022575795110  UNAVAIL  cannot open >             2933754417879630350  UNAVAIL  cannot open > >    pool: esx >      id: 8314148521324214892 >   state: UNAVAIL >  status: The pool was last accessed by another system. >  action: The pool cannot be imported due to damaged devices or data. >    see: http://illumos.org/msg/ZFS-8000-EY >  config: > >         esx                       UNAVAIL  insufficient replicas >           mirror-0                UNAVAIL  insufficient replicas >             10170732803757341731  UNAVAIL  cannot open >             9207269511643803468   UNAVAIL  cannot open > > > is there any _safe_ way to get rid of this ? I'm asking because a gptzfsboot > loader in recent -STABLE stumbles upon this and refuses to boot the system > (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227772). The workaround is to > use the 11.1 loader, but I'm afraid this behavior will now be the intended one. You can try to use zdb -l to find the stale labels. And then zpool labelclear to clear them. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Thu Apr 26 14:37:24 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 873DBFAEE2C; Thu, 26 Apr 2018 14:37:24 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (unknown [IPv6:2a01:4f8:d12:604::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 05D5F72D64; Thu, 26 Apr 2018 14:37:23 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w3QEbH4Y035767 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 26 Apr 2018 16:37:17 +0200 (CEST) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: avg@FreeBSD.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w3QEbCvn027163 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 26 Apr 2018 21:37:12 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: clear old pools remains from active vdevs To: Andriy Gapon , "Eugene M. Zheganin" , freebsd-stable@freebsd.org References: <92b45b63-48c0-e9fe-c7c8-3ff24f37bf1f@FreeBSD.org> Cc: freebsd-fs@freebsd.org From: Eugene Grosbein Message-ID: <5AE1E414.4010408@grosbein.net> Date: Thu, 26 Apr 2018 21:37:08 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <92b45b63-48c0-e9fe-c7c8-3ff24f37bf1f@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -0.0 SPF_PASS SPF: sender matches SPF record * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 14:37:24 -0000 26.04.2018 14:50, Andriy Gapon wrote: > You can try to use zdb -l to find the stale labels. > And then zpool labelclear to clear them. Our "zpool labelclear" implementation destroys everything (literally). Have you really tried it? From owner-freebsd-fs@freebsd.org Thu Apr 26 15:14:44 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB4B7FB0263; Thu, 26 Apr 2018 15:14:44 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x231.google.com (mail-lf0-x231.google.com [IPv6:2a00:1450:4010:c07::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B1387AD67; Thu, 26 Apr 2018 15:14:44 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x231.google.com with SMTP id h4-v6so4517190lfc.1; Thu, 26 Apr 2018 08:14:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=kQtFSmmDhGQuT9AJNRiFWh/sF/yxnJO516WILVhjTyc=; b=RWURpwN3CiYaHLE1odKgd8Ok6xrXcTKRyZ92lf7vfmrGIVq04zcEpZgd0wnsO/CHlt lAJzcxQ1D1B26GUm1GdzPHQz0I736RPn/hV6EfVFKB2U5LTIJ3z14L9kzXTmiPsYfpMo w/tk6OySTp/oHLKpqJGvk1cealqFII2gRSeZk6whTe0G/ZIIIuDWaxMUIdCwxkxW2dKC R4CdQBZCL6ds5Ga85Ens5eBj3naUJMiF+2AodnZMPvSxzgvYVvhPma+buiQh/1wxTgVM PHsvsbwFVsaQnC9OblaImpiAGWOn9WDGpGDGOO6owDn3TbnKbAVdl9VxDXQDlK9A7nTF rg7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=kQtFSmmDhGQuT9AJNRiFWh/sF/yxnJO516WILVhjTyc=; b=M8O8+FVDUAfpDrA3n5CO3U82Nshx5Nu91Q2UtL2p5e0IlwU1d4diDlS72CYMv4IM+a NB0WopGV4gNXSWtTzMo9z6S/Zb6jGg20tOyZnDrif5aDGJhguBKigh/appvhqUgfPIPx nvfcegmF3KtjlLGi1fAwQa8XLvFn+/kwbni+j3ubal8PGg1fLUWtRVIUIkMvubUXCiX7 Iw6Xzwvn3+8BFLdUAUWaNEY0U7pz1J69+iZ70dChNIEt4YlnQl2UCVvSQgyFHmQUCRwh ZeDLC/44IpWXJiC2VREm480yewsAp+C0tCErC7p8Rb2YkYdaf2IiyjucefOwFD2NiQaj Ldow== X-Gm-Message-State: ALQs6tDUklrkPlO4FRHoRtKWoePZd4N9+QkIkXVxgwTwH0aFF1B33rns vHW0VntzPlV6pZruSIt/gTKTBlPS7iF07cJdYRo= X-Google-Smtp-Source: AIpwx49OYhLJB2qzmfNo/7DPGgSU1pldERiZxhxE0HC0/Nhdr2FDL/iHuLsJ7hoSXOlGqYBl3I4lVkr738G9Lq4Z0d0= X-Received: by 10.46.2.90 with SMTP id 87mr23872898ljc.46.1524755682549; Thu, 26 Apr 2018 08:14:42 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.6.21 with HTTP; Thu, 26 Apr 2018 08:14:41 -0700 (PDT) In-Reply-To: <5AE1E414.4010408@grosbein.net> References: <92b45b63-48c0-e9fe-c7c8-3ff24f37bf1f@FreeBSD.org> <5AE1E414.4010408@grosbein.net> From: Alan Somers Date: Thu, 26 Apr 2018 09:14:41 -0600 X-Google-Sender-Auth: aff5XLq8b6Zs3Za6jknqFbazjdw Message-ID: Subject: Re: clear old pools remains from active vdevs To: Eugene Grosbein Cc: Andriy Gapon , "Eugene M. Zheganin" , FreeBSD , freebsd-fs Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 15:14:45 -0000 On Thu, Apr 26, 2018 at 8:37 AM, Eugene Grosbein wrote: > 26.04.2018 14:50, Andriy Gapon wrote: > > > You can try to use zdb -l to find the stale labels. > > And then zpool labelclear to clear them. > > Our "zpool labelclear" implementation destroys everything (literally). > Have you really tried it? > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > "zpool labelclear" won't help in this case, because you have literally no devices with active labels. The problem is that the pool is still in your /boot/zfs/zpool.cache file. I think that plain "zpool destroy esx" will work in this case. From owner-freebsd-fs@freebsd.org Thu Apr 26 15:50:54 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A819FB0F60; Thu, 26 Apr 2018 15:50:54 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f47.google.com (mail-lf0-f47.google.com [209.85.215.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE6A382750; Thu, 26 Apr 2018 15:50:53 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f47.google.com with SMTP id b23-v6so31508755lfg.4; Thu, 26 Apr 2018 08:50:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=KsGtaSeguFOFTT7e03cSHgODnc3OrUsYW5/Sijm2FDU=; b=BW/NXRD7/ca6Px0q6lzob9jWugjuLdrECldU5RnhPPgMOXkHHrWgGGEk3rslqdZHr+ N7xKCeX4498N89DZsTn7pvdb4t4zljwGCDKZpn+p57FBbkU9dMSg/dfQboEGzqug3kfU EVuBnXdIORXBI9ozxGzVHqCZ9mHvc+xeyoe1T30ctFl1qfIWdn2KJWZg/cJ9SC3anPPh kE+LUwn+aQvto0rZw2EIrPAn/i1UD+6y7suftul/RAKh9LVihhDeYXUYjWA0wAWk7UZU DzUIvscGAZgmYhL1PDlnzgkmJMsJ2IMDPii+0bbHlEuUH7S2uu/N736KzUI/0LZLE7Ej jB4Q== X-Gm-Message-State: ALQs6tD4iJ7dDg6oc2ViDA9a4J+JG1a1gRRrZMviFLgu/2pIr0PRyOFr DwNAgZg9VGUSW2YEywbeWjW86Ji+ X-Google-Smtp-Source: AB8JxZrEMmMBbfjrocUQqA8UUye56QempJbZpj6tuwyy+OA01OzhQpgxadFg1AmMD1xoGEo+/qD63Q== X-Received: by 2002:a19:b587:: with SMTP id g7-v6mr18057226lfk.90.1524757851545; Thu, 26 Apr 2018 08:50:51 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id l12sm2633594ljh.78.2018.04.26.08.50.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Apr 2018 08:50:50 -0700 (PDT) Subject: Re: clear old pools remains from active vdevs To: Eugene Grosbein , "Eugene M. Zheganin" , freebsd-stable@FreeBSD.org Cc: freebsd-fs@FreeBSD.org References: <92b45b63-48c0-e9fe-c7c8-3ff24f37bf1f@FreeBSD.org> <5AE1E414.4010408@grosbein.net> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABzR5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz7CwZQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryM7BTQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAcLBfAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <5811ffb0-9afd-d6f3-6c04-febb0fab6be3@FreeBSD.org> Date: Thu, 26 Apr 2018 18:50:49 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <5AE1E414.4010408@grosbein.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 15:50:54 -0000 On 26/04/2018 17:37, Eugene Grosbein wrote: > 26.04.2018 14:50, Andriy Gapon wrote: > >> You can try to use zdb -l to find the stale labels. >> And then zpool labelclear to clear them. > > Our "zpool labelclear" implementation destroys everything (literally). > Have you really tried it? > I have never needed it, so I have never used it. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Thu Apr 26 15:59:09 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07C80FB1442; Thu, 26 Apr 2018 15:59:09 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f47.google.com (mail-lf0-f47.google.com [209.85.215.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A5CF848F1; Thu, 26 Apr 2018 15:59:08 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f47.google.com with SMTP id v85-v6so9939503lfa.13; Thu, 26 Apr 2018 08:59:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=7TRQZCwirU1oPkVKbX2Dy2RRmAgr0Sz5QA2VielJaaY=; b=giJjjpCqvcOsK9qi2RjmhoiPLIbI+FDENFGPmGQTnGXXrBMvcoD0Jw+nlb0ZziGxbI NAOxu+ntLjuc6qEk4coIcyTVJUAhTFGFOZ3AtG+hQ3tVx0uMUzKGq3yWWG8llv1Jt1JP oluM1Su/H0Xar1J5CYaTamctJqN/2Bvvg7C8yAJxSiv6S5GwK8DAX18I472BjFRLQZyj Qpi3gUZQQ/5MuwuTEYFgt+Phw1Lis3zINQfhQe8c+qABHRjK//7NjFQ2tbg0K7mX3aft XmYF6noBF4DOhKX+sTI+lzGtv0plC37xFbP/vfSa8lenoGOlc4ifZijXo+ITOHeh6MxV Tsdw== X-Gm-Message-State: ALQs6tCJ8j1xDCMT2wYBrwDaGrv1PGxmgCBR/iKKoJ0sYcna9mt1GG3S IrzgdBikYoZZJI/d7TvKRAqbn/Wj X-Google-Smtp-Source: AIpwx49T51sY8EObqnNvyVJGXAjPVRF6decsA2SPrzH2jJuNLRJAr39DVRzg35R4w9FV5FHJyAYyaQ== X-Received: by 10.46.146.204 with SMTP id k12mr17245078ljh.101.1524757969242; Thu, 26 Apr 2018 08:52:49 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id u5sm403398ljk.64.2018.04.26.08.52.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Apr 2018 08:52:48 -0700 (PDT) Subject: Re: clear old pools remains from active vdevs To: Alan Somers , Eugene Grosbein Cc: "Eugene M. Zheganin" , FreeBSD , freebsd-fs References: <92b45b63-48c0-e9fe-c7c8-3ff24f37bf1f@FreeBSD.org> <5AE1E414.4010408@grosbein.net> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABzR5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz7CwZQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryM7BTQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAcLBfAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: Date: Thu, 26 Apr 2018 18:52:47 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 15:59:09 -0000 On 26/04/2018 18:14, Alan Somers wrote: > On Thu, Apr 26, 2018 at 8:37 AM, Eugene Grosbein > wrote: > > 26.04.2018 14:50, Andriy Gapon wrote: > > > You can try to use zdb -l to find the stale labels. > > And then zpool labelclear to clear them. > > Our "zpool labelclear" implementation destroys everything (literally). > Have you really tried it? > > > "zpool labelclear" won't help in this case, because you have literally no > devices with active labels.  The problem is that the pool is still in your > /boot/zfs/zpool.cache file.  I think that plain "zpool destroy esx" will work in > this case. In the original message Eugene (Zheganin) mentioned that those phantom pools confused the boot chain. I do not think that anything in zpool.cache can do that. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Thu Apr 26 22:26:58 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3EF9FB9E0A for ; Thu, 26 Apr 2018 22:26:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6128078E98 for ; Thu, 26 Apr 2018 22:26:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 20693FB9E04; Thu, 26 Apr 2018 22:26:58 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E407FB9E03 for ; Thu, 26 Apr 2018 22:26:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A74078E72 for ; Thu, 26 Apr 2018 22:26:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id DCED01A199 for ; Thu, 26 Apr 2018 22:26:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3QMQuxj026790 for ; Thu, 26 Apr 2018 22:26:56 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3QMQumi026788 for fs@FreeBSD.org; Thu, 26 Apr 2018 22:26:56 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 227784] zfs: Fatal trap 9: general protection fault while in kernel mode on shutdown Date: Thu, 26 Apr 2018 22:26:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Apr 2018 22:26:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227784 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Apr 28 06:34:40 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6538FC3163 for ; Sat, 28 Apr 2018 06:34:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6D3546E7F7 for ; Sat, 28 Apr 2018 06:34:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 1DB90FC3162; Sat, 28 Apr 2018 06:34:40 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F06AAFC3161 for ; Sat, 28 Apr 2018 06:34:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 793FF6E7CF for ; Sat, 28 Apr 2018 06:34:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 91EF62C5C for ; Sat, 28 Apr 2018 06:34:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3S6YcAR088605 for ; Sat, 28 Apr 2018 06:34:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3S6Yc6L088604 for fs@FreeBSD.org; Sat, 28 Apr 2018 06:34:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 223846] msdosfs does not reflect READONLY to user Date: Sat, 28 Apr 2018 06:34:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 06:34:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223846 Damjan Jovanovic changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |damjan.jov@gmail.com --- Comment #4 from Damjan Jovanovic --- Created attachment 192866 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D192866&action= =3Dedit deny writing to READONLY files in msdosfs Hi. Here is my patch to deny writing to files with the READONLY attribute on msdosfs filesystems. It just affects the file_mode variable in msdosfs_access(). My tests show writing and appending to READONLY files is successfully denied the same way on ufs, zfs, and msdosfs. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Apr 28 07:46:39 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85ED1FC629C for ; Sat, 28 Apr 2018 07:46:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1C9F27D110 for ; Sat, 28 Apr 2018 07:46:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id D4CFEFC629B; Sat, 28 Apr 2018 07:46:38 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98BFEFC6299 for ; Sat, 28 Apr 2018 07:46:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 29E377D0EE for ; Sat, 28 Apr 2018 07:46:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 491873988 for ; Sat, 28 Apr 2018 07:46:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w3S7kbZ9079330 for ; Sat, 28 Apr 2018 07:46:37 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w3S7kbnK079329 for fs@FreeBSD.org; Sat, 28 Apr 2018 07:46:37 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 205154] newfs_msdos issue Date: Sat, 28 Apr 2018 07:46:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Rejected X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 07:46:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205154 Damjan Jovanovic changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |damjan.jov@gmail.com --- Comment #4 from Damjan Jovanovic --- Both 80GiB and 150GiB files seem to work within FreeBSD CURRENT: # dd if=3D/dev/zero of=3D80G seek=3D$((80*1024*1024*1024-1)) count=3D1 bs= =3D1 # newfs_msdos -F 32 -O android -c 8 ./80G newfs_msdos: warning, ./80G is not a character device ./80G: 167445080 sectors in 20930635 FAT32 clusters (4096 bytes/cluster) BytesPerSec=3D512 SecPerClust=3D8 ResSectors=3D32 FATs=3D2 Media=3D0xf0 Sec= PerTrack=3D63 Heads=3D255 HiddenSecs=3D0 HugeSectors=3D167772160 FATsecs=3D163521 RootClu= ster=3D2 FSInfo=3D1 Backup=3D2 # file 80G 80G: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "android ", sectors/cluster 8, sectors/track 63, heads 255, FAT (32 bit), sectors/FAT 163521, serial number 0x432c1018, unlabeled # fsck_msdosfs ./80G=20 ** ./80G ** Phase 1 - Read and Compare FATs ** Phase 2 - Check Cluster Chains ** Phase 3 - Checking Directories ** Phase 4 - Checking for Lost Files 1 files, 4030760 free (20930634 clusters) # mdconfig -a -f 80G md0 # mount -t msdosfs /dev/md0 /mnt/test (seems to work, can write and read a file) # dd if=3D/dev/zero of=3D150G seek=3D$((150*1024*1024*1024-1)) count=3D1 bs= =3D1 # newfs_msdos -F 32 -O android -c 8 ./150G=20 newfs_msdos: warning, ./150G is not a character device ./150G: 313959560 sectors in 39244945 FAT32 clusters (4096 bytes/cluster) BytesPerSec=3D512 SecPerClust=3D8 ResSectors=3D32 FATs=3D2 Media=3D0xf0 Sec= PerTrack=3D63 Heads=3D255 HiddenSecs=3D0 HugeSectors=3D314572800 FATsecs=3D306602 RootClu= ster=3D2 FSInfo=3D1 Backup=3D2 # file ./150G=20 ./150G: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "android ", sectors/cluster 8, sectors/track 63, heads 255, FAT (32 bit), sectors/FAT 306602, serial number 0x3b941109, unlabeled # fsck_msdosfs ./150G ** ./150G ** Phase 1 - Read and Compare FATs ** Phase 2 - Check Cluster Chains ** Phase 3 - Checking Directories ** Phase 4 - Checking for Lost Files 1 files, 1790528 free (39244944 clusters) # mdconfig -a -f 150G=20 md0 # mount -t msdosfs /dev/md0 /mnt/test (seems to work, can write and read a file) Where didn't this storage work? In your Android device? Maybe forcing secto= rs per cluster to be 8 is an issue for other FAT32 implementations? When I lea= ve the "-c 8" out, sectors per cluster goes up to 64: # newfs_msdos -F 32 -O android ./150G=20 newfs_msdos: warning, ./150G is not a character device ./150G: 314495936 sectors in 4913999 FAT32 clusters (32768 bytes/cluster) BytesPerSec=3D512 SecPerClust=3D64 ResSectors=3D32 FATs=3D2 Media=3D0xf0 Se= cPerTrack=3D63 Heads=3D255 HiddenSecs=3D0 HugeSectors=3D314572800 FATsecs=3D38391 RootClus= ter=3D2 FSInfo=3D1 Backup=3D2 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Apr 28 15:42:47 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 850BAFB27E2; Sat, 28 Apr 2018 15:42:47 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1523F7ABC0; Sat, 28 Apr 2018 15:42:47 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id CCD99178D6; Sat, 28 Apr 2018 17:42:44 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n8TaGzT5ttP8; Sat, 28 Apr 2018 17:42:44 +0200 (CEST) Received: from [192.168.10.67] (opteron [192.168.10.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 34938178D5; Sat, 28 Apr 2018 17:42:44 +0200 (CEST) To: FreeBSD Hackers , FreeBSD Filesystems From: Willem Jan Withagen Subject: Getting ZFS pools back. Message-ID: <5f836c79-b379-f066-689b-1645e393c5e9@digiware.nl> Date: Sat, 28 Apr 2018 17:42:41 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 15:42:47 -0000 Hi, I had this server crash on me, and first it just complained about not being able to boot because it could not find the guid. Now I cannot even import the pools any longer. Althoug zdb -l /dev/ada0 still gives me data that indicates that there should be a ZFS pool on that partition. Any suggestions on how to get the pools/data back online? Help would be highly appreciated, since restoring it from backups is going to quite some work. Thanx, --WjW From owner-freebsd-fs@freebsd.org Sat Apr 28 16:28:12 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CC7FFB3D29 for ; Sat, 28 Apr 2018 16:28:12 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wr0-x22a.google.com (mail-wr0-x22a.google.com [IPv6:2a00:1450:400c:c0c::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9BB88602F for ; Sat, 28 Apr 2018 16:28:11 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wr0-x22a.google.com with SMTP id i14-v6so1242439wre.2 for ; Sat, 28 Apr 2018 09:28:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language; bh=wDyfjfgv9mNXHs5QlAB07w7P2aUeT/zgvbQmyKL3iOw=; b=jVDCJq7ehP+yUp6Whs20XLwZMPCEPiWTuJAtlqkeVqoTltsI2wtFMYtcyK0zcTtnFj Ec95pl4PBFkLufFAUqGprAn6cMWTot9bgsuzM7npmG3JgYJUj4iAWoagdtqJQkDe72Xt p1VdqjJhtgJORCNMKuxqp+4LhtvY75nw2MJvuJzGAFe8pORENDPfahs6JWRpeFgcDk45 pLGMFdM+40JgGTulawJ4/bWPiCCDxJDL/aXHM6ERAcamp/c80+bytse0U9pLaMrFr3pd 2tZ83EYxuDFfp4dklWQ0NgzUirrPAUv8qI2wbWqIO+6gFDn4n4RowtVfEuv6jfSfob9+ RWSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=wDyfjfgv9mNXHs5QlAB07w7P2aUeT/zgvbQmyKL3iOw=; b=dE+DX5RQXeL6QfojqgihB7a+w3qWhIPuQkNQxXgFOGyaO6LaTzFQXLYB8kIwdUIoHk 9jCcucPynG9EWac4K3n7tqISwULWJuLv4i+OlRTn0tHzzJ5wThv9CHbl8zqX6+lyun4s k6JjCy+tbjdjHDoDSe/9Rrlm2j0M0izsRsOTiqcMJFG4LfsxYcWDoxZWKHe5CyJ+TZIr 6sjerQGC0IRQq+KHnYsY/7nUFVBk/AQK84Gq9vVG8cvxM1Rttr8wxeZwKPgI6XAEeUOr ewdD6qeMFouFf4rk6IdqcA3Pljsveqdwxjo9bdvTk4F4bwq61Hz8238S1WGnbYCLgZ5z bGoQ== X-Gm-Message-State: ALQs6tC8J3hdvFTHEab5xhckS3aLgQQSSlXJmgkZt3Jc/5mfriWao9nV /5Ax3p65AKnKxyqLX5gv9PnOn677YtI= X-Google-Smtp-Source: AB8JxZqHD/gqlj7HXMCzeXT2OUedD76N/o2nBBRgGHegb6vW4/NTxfVG2yevIGQmhwm8XbjHp4gc3Q== X-Received: by 2002:adf:99b5:: with SMTP id y50-v6mr4406613wrb.168.1524932889937; Sat, 28 Apr 2018 09:28:09 -0700 (PDT) Received: from [10.10.1.111] ([185.97.61.1]) by smtp.gmail.com with ESMTPSA id h12sm4929581wmc.7.2018.04.28.09.28.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 28 Apr 2018 09:28:08 -0700 (PDT) Subject: Re: Getting ZFS pools back. To: freebsd-fs@freebsd.org References: <5f836c79-b379-f066-689b-1645e393c5e9@digiware.nl> From: Steven Hartland Message-ID: <7607fc94-6fbf-07e9-91a3-b5053b60f1c4@multiplay.co.uk> Date: Sat, 28 Apr 2018 17:28:08 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <5f836c79-b379-f066-689b-1645e393c5e9@digiware.nl> Content-Language: en-US Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 16:28:12 -0000 Silly question have you tried booting up from the old kernel? On 28/04/2018 16:42, Willem Jan Withagen wrote: > > Hi, > > I had this server crash on me, and first it just complained about not > being able to boot because it could not find the guid. > > Now I cannot even import the pools any longer. > Althoug zdb -l /dev/ada0 still gives me data that indicates that there > should be a ZFS pool on that partition. > > Any suggestions on how to get the pools/data back online? > > Help would be highly appreciated, since restoring it from backups is > going to quite some work. > > Thanx, > --WjW > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Sat Apr 28 18:43:42 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2A7FFB6A39; Sat, 28 Apr 2018 18:43:41 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 949DE827BB; Sat, 28 Apr 2018 18:43:41 +0000 (UTC) (envelope-from ryao@gentoo.org) Received: from [192.168.5.108] (pool-96-232-204-110.nycmny.fios.verizon.net [96.232.204.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: ryao) by smtp.gentoo.org (Postfix) with ESMTPSA id 520D4335C5A; Sat, 28 Apr 2018 18:43:34 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: Getting ZFS pools back. From: Richard Yao X-Mailer: iPad Mail (15E302) In-Reply-To: <5f836c79-b379-f066-689b-1645e393c5e9@digiware.nl> Date: Sat, 28 Apr 2018 14:43:31 -0400 Cc: FreeBSD Hackers , FreeBSD Filesystems Content-Transfer-Encoding: quoted-printable Message-Id: References: <5f836c79-b379-f066-689b-1645e393c5e9@digiware.nl> To: Willem Jan Withagen X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Apr 2018 18:43:42 -0000 What is the output of =E2=80=98zpool import=E2=80=98 with no arguments? > On Apr 28, 2018, at 11:42 AM, Willem Jan Withagen wrote:= >=20 >=20 > Hi, >=20 > I had this server crash on me, and first it just complained about not bein= g able to boot because it could not find the guid. >=20 > Now I cannot even import the pools any longer. > Althoug zdb -l /dev/ada0 still gives me data that indicates that there sho= uld be a ZFS pool on that partition. >=20 > Any suggestions on how to get the pools/data back online? >=20 > Help would be highly appreciated, since restoring it from backups is going= to quite some work. >=20 > Thanx, > --WjW > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"=