From owner-freebsd-fs@freebsd.org Sun Jun 26 15:03:40 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94B41B82E3E for ; Sun, 26 Jun 2016 15:03:40 +0000 (UTC) (envelope-from cobble2@galaxy.lunarpages.com) Received: from galaxy.lunarpages.com (galaxy.lunarpages.com [64.50.186.104]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7375F2564 for ; Sun, 26 Jun 2016 15:03:40 +0000 (UTC) (envelope-from cobble2@galaxy.lunarpages.com) Received: from cobble2 by galaxy.lunarpages.com with local (Exim 4.87) (envelope-from ) id 1bHArM-0008BM-9S for freebsd-fs@freebsd.org; Sun, 26 Jun 2016 07:16:28 -0700 To: freebsd-fs@freebsd.org Subject: We could not deliver your parcel, #0000971665 Date: Sun, 26 Jun 2016 07:16:28 -0700 From: "FedEx International Next Flight" Reply-To: "FedEx International Next Flight" Message-ID: X-Priority: 3 MIME-Version: 1.0 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - galaxy.lunarpages.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [32431 928] / [47 12] X-AntiAbuse: Sender Address Domain - galaxy.lunarpages.com X-Get-Message-Sender-Via: galaxy.lunarpages.com: authenticated_id: cobble2/from_h X-Authenticated-Sender: galaxy.lunarpages.com: rene.fink@cobblestoneconsulting.com Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Jun 2016 15:03:40 -0000 Dear Customer, We could not deliver your parcel. Shipment Label is attached to this email. Warm regards, Rene Fink, Station Agent. From owner-freebsd-fs@freebsd.org Sun Jun 26 17:15:17 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60571B81EC6 for ; Sun, 26 Jun 2016 17:15:17 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 27B672508 for ; Sun, 26 Jun 2016 17:15:17 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1bHDPh-0004ET-RE for freebsd-fs@freebsd.org; Sun, 26 Jun 2016 19:00:06 +0200 Received: from 89.248.140.17 ([89.248.140.17]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 26 Jun 2016 19:00:05 +0200 Received: from holger by 89.248.140.17 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 26 Jun 2016 19:00:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Holger Freyther Subject: Deadlock in zpool import with degraded pool Date: Sun, 26 Jun 2016 16:52:30 +0000 (UTC) Lines: 35 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: sea.gmane.org User-Agent: Loom/3.14 (http://gmane.org/) X-Loom-IP: 89.248.140.17 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Jun 2016 17:15:17 -0000 Hi, Everytime I run zpool import tank, I can see disk activity with gstat, with top I see the ARC growing and the free memory going down and then the system seems to lock-up. Adding a swap partition doesn't help. I have tried the zpool import with 10.1, 10.3 and 11.0-CURRENT. My pool consists out of two disks and on the last start the second GELI partition was not attached. When manually doing geli attach + zpool online it started to resilver. The system still had a broken disk inside so I took this opportunity to halt and remove it. On reboot the disk changed from ada2 to ada1. With that the pool was degraded and the only thing I found was zpool replace to replace the physical disk with itself. As it was resilvering before this didn't look like a big loss in terms of time. GELI is not the fastest and I didn't want to wait too long so I started zfs destroy of some volumes I will not use anymore. This should have freed 300GB from 2TB of a ~3TB pool. Before the zfs destroy completing the system locked-up. On reboot the system unlocked both GELI partitions but got stuck on mounting the rootfs. Booting the memstick version of FreeBSD 10.3 and 11.0-CURRENT I do see the deadlock on import (top not updating time, not responding to switches of the VTY, USB keyboard plug/unplug still generating messages though). The system is a AMD64 with 12GB RAM and running FreeBSD 10.1. It has two 3TB disks, a boot pool and the main pool on top of GELI. The main pool runs with deduplication enabled Does this sound familiar? Is there anything I can try before saying good bye to mybacked up data? If the pool itself can not be repaired, is there a way to access the volumes with the data? thanks for your help. From owner-freebsd-fs@freebsd.org Sun Jun 26 17:43:31 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE03AB81AE2 for ; Sun, 26 Jun 2016 17:43:31 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from Exch2-2.slu.se (pop.slu.se [77.235.224.122]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 689EF2254 for ; Sun, 26 Jun 2016 17:43:30 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by Exch2-2.slu.se (77.235.224.122) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Sun, 26 Jun 2016 19:28:18 +0200 Received: from exch2-4.slu.se ([fe80::f0cf:a4:cef1:ba2]) by exch2-4.slu.se ([fe80::f0cf:a4:cef1:ba2%22]) with mapi id 15.00.1156.000; Sun, 26 Jun 2016 19:28:18 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Holger Freyther CC: "freebsd-fs@freebsd.org" Subject: Re: Deadlock in zpool import with degraded pool Thread-Topic: Deadlock in zpool import with degraded pool Thread-Index: AQHRz9Ag7kkebCRlmEmmvTPK77TiTA== Date: Sun, 26 Jun 2016 17:28:17 +0000 Message-ID: <8a4cb87252c04ebfbf71451c5dc1a41e@exch2-4.slu.se> Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Jun 2016 17:43:32 -0000 DQpEZW4gMjYganVuIDIwMTYgNzoxNSBlbSBza3JldiBIb2xnZXIgRnJleXRoZXIgPGhvbGdlckBm cmV5dGhlci5kZT46DQo+DQo+IEhpLA0KPg0KPiBFdmVyeXRpbWUgSSBydW4genBvb2wgaW1wb3J0 IHRhbmssIEkgY2FuIHNlZSBkaXNrIGFjdGl2aXR5IHdpdGggZ3N0YXQsDQo+IHdpdGggdG9wIEkg c2VlIHRoZSBBUkMgZ3Jvd2luZyBhbmQgdGhlIGZyZWUgbWVtb3J5IGdvaW5nIGRvd24gYW5kIHRo ZW4NCj4gdGhlIHN5c3RlbSBzZWVtcyB0byBsb2NrLXVwLiBBZGRpbmcgYSBzd2FwIHBhcnRpdGlv biBkb2Vzbid0IGhlbHAuIEkNCj4gaGF2ZSB0cmllZCB0aGUgenBvb2wgaW1wb3J0IHdpdGggMTAu MSwgMTAuMyBhbmQgMTEuMC1DVVJSRU5ULg0KPg0KPiBNeSBwb29sIGNvbnNpc3RzIG91dCBvZiB0 d28gZGlza3MgYW5kIG9uIHRoZSBsYXN0IHN0YXJ0IHRoZSBzZWNvbmQgR0VMSQ0KPiBwYXJ0aXRp b24gd2FzIG5vdCBhdHRhY2hlZC4gV2hlbiBtYW51YWxseSBkb2luZyBnZWxpIGF0dGFjaCArIHpw b29sDQo+IG9ubGluZSBpdCBzdGFydGVkIHRvIHJlc2lsdmVyLiBUaGUgc3lzdGVtIHN0aWxsIGhh ZCBhIGJyb2tlbiBkaXNrIGluc2lkZQ0KPiBzbyBJIHRvb2sgdGhpcyBvcHBvcnR1bml0eSB0byBo YWx0IGFuZCByZW1vdmUgaXQuIE9uIHJlYm9vdCB0aGUgZGlzaw0KPiBjaGFuZ2VkIGZyb20gYWRh MiB0byBhZGExLiBXaXRoIHRoYXQgdGhlIHBvb2wgd2FzIGRlZ3JhZGVkIGFuZCB0aGUgb25seQ0K PiB0aGluZyBJIGZvdW5kIHdhcyB6cG9vbCAgcmVwbGFjZSB0byByZXBsYWNlIHRoZSBwaHlzaWNh bCBkaXNrIHdpdGggaXRzZWxmLg0KPiBBcyBpdCB3YXMgcmVzaWx2ZXJpbmcgYmVmb3JlIHRoaXMg ZGlkbid0IGxvb2sgbGlrZSBhIGJpZyBsb3NzIGluIHRlcm1zDQo+IG9mIHRpbWUuIEdFTEkgaXMg bm90IHRoZSBmYXN0ZXN0IGFuZCBJIGRpZG4ndCB3YW50IHRvIHdhaXQgdG9vIGxvbmcgc28gSQ0K PiBzdGFydGVkIHpmcyBkZXN0cm95IG9mIHNvbWUgdm9sdW1lcyBJIHdpbGwgbm90IHVzZSBhbnlt b3JlLiBUaGlzIHNob3VsZA0KPiBoYXZlIGZyZWVkIDMwMEdCIGZyb20gMlRCIG9mIGEgfjNUQiBw b29sLiBCZWZvcmUgdGhlIHpmcyBkZXN0cm95DQo+IGNvbXBsZXRpbmcgdGhlIHN5c3RlbSBsb2Nr ZWQtdXAuDQo+DQo+IE9uIHJlYm9vdCB0aGUgc3lzdGVtIHVubG9ja2VkIGJvdGggR0VMSSBwYXJ0 aXRpb25zIGJ1dCBnb3Qgc3R1Y2sgb24NCj4gbW91bnRpbmcgdGhlIHJvb3Rmcy4gQm9vdGluZyB0 aGUgbWVtc3RpY2sgdmVyc2lvbiBvZiBGcmVlQlNEIDEwLjMgYW5kDQo+IDExLjAtQ1VSUkVOVCBJ IGRvIHNlZSB0aGUgZGVhZGxvY2sgb24gaW1wb3J0ICh0b3Agbm90IHVwZGF0aW5nIHRpbWUsIG5v dA0KPiByZXNwb25kaW5nIHRvIHN3aXRjaGVzIG9mIHRoZSBWVFksIFVTQiBrZXlib2FyZCBwbHVn L3VucGx1ZyBzdGlsbA0KPiBnZW5lcmF0aW5nIG1lc3NhZ2VzIHRob3VnaCkuDQo+DQo+IFRoZSBz eXN0ZW0gaXMgYSBBTUQ2NCB3aXRoIDEyR0IgUkFNIGFuZCBydW5uaW5nIEZyZWVCU0QgMTAuMS4g SXQgaGFzDQo+IHR3byAzVEIgZGlza3MsIGEgYm9vdCBwb29sIGFuZCB0aGUgbWFpbiBwb29sIG9u IHRvcCBvZiBHRUxJLiBUaGUgbWFpbg0KPiBwb29sIHJ1bnMgd2l0aCBkZWR1cGxpY2F0aW9uIGVu YWJsZWQNCg0KVGhhdCdzIHlvdXIgcHJvYmxlbSByaWdodCB0aGVyZTsgZGVkdXAhIFlvdSBuZWVk IHRvIHRocm93IG1vcmUgUkFNIGludG8gaXQgdW50aWwgdGhlIGRlc3Ryb3kgY2FuIGNvbXBsZXRl LiBJZiB0aGUgbW9ibyBpcyAnZnVsbCcsIHlvdSBuZWVkIG5ldy9vdGhlciBodyB0byBjcmFtIG1v cmUgUkFNIGludG8gb3IgeW91IGNhbiBraXNzIHlvdXIgZGF0YSBnb29kYnllLiBJJ3ZlIGJlZW4g aW4gdGhlIGV4YWN0IHNhbWUgc2l0dWF0aW9uIGFzIHlvdSBhcmUgbm93IHNvIEkgc3ltcGF0aGl6 ZTooDQoNCi9LDQoNCj4NCj4gRG9lcyB0aGlzIHNvdW5kIGZhbWlsaWFyPyBJcyB0aGVyZSBhbnl0 aGluZyBJIGNhbiB0cnkgYmVmb3JlIHNheWluZw0KPiBnb29kIGJ5ZSB0byBteWJhY2tlZCB1cCBk YXRhPyBJZiB0aGUgcG9vbCBpdHNlbGYgY2FuIG5vdCBiZSByZXBhaXJlZCwNCj4gaXMgdGhlcmUg YSB3YXkgdG8gYWNjZXNzIHRoZSB2b2x1bWVzIHdpdGggdGhlIGRhdGE/DQo+DQo+IHRoYW5rcyBm b3IgeW91ciBoZWxwLg0KPg0KPg0KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fXw0KPiBmcmVlYnNkLWZzQGZyZWVic2Qub3JnIG1haWxpbmcgbGlzdA0KPiBo dHRwczovL2xpc3RzLmZyZWVic2Qub3JnL21haWxtYW4vbGlzdGluZm8vZnJlZWJzZC1mcw0KPiBU byB1bnN1YnNjcmliZSwgc2VuZCBhbnkgbWFpbCB0byAiZnJlZWJzZC1mcy11bnN1YnNjcmliZUBm cmVlYnNkLm9yZyINCg== From owner-freebsd-fs@freebsd.org Mon Jun 27 08:15:17 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7B3FB84611 for ; Mon, 27 Jun 2016 08:15:17 +0000 (UTC) (envelope-from honzhan@microsoft.com) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0132.outbound.protection.outlook.com [157.56.110.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 687B624A8 for ; Mon, 27 Jun 2016 08:15:16 +0000 (UTC) (envelope-from honzhan@microsoft.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=FhyTZUM9LeHi3KQDV32ooaMCUG1+Ma4uau08+CXphRI=; b=RTIWl5h+1ZCdWE0cybdLD1HN1sMZAWvzkDfj+GuoevCtaQevpsBthtGRFkLNCXTpHJ5d4fY708Zl3ZjrLmX/OlS5sW3zHGsNWCeDJ5MyRSc1LWOrAnPpsONVxE9fzZnfG9z6W0QOif4vrpIfxPpU/anRocWhENX+VzEQQ16YETk= Received: from CO2PR03MB2215.namprd03.prod.outlook.com (10.166.92.26) by CO2PR03MB2215.namprd03.prod.outlook.com (10.166.92.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.523.12; Mon, 27 Jun 2016 08:15:09 +0000 Received: from CO2PR03MB2215.namprd03.prod.outlook.com ([10.166.92.26]) by CO2PR03MB2215.namprd03.prod.outlook.com ([10.166.92.26]) with mapi id 15.01.0523.019; Mon, 27 Jun 2016 08:15:09 +0000 From: Hongjiang Zhang To: "freebsd-fs@freebsd.org" Subject: ufs freeze does not work Thread-Topic: ufs freeze does not work Thread-Index: AdHQS4KXvmVyW6W2TjqUQxMwTLuTxQ== Date: Mon, 27 Jun 2016 08:15:09 +0000 Message-ID: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=honzhan@microsoft.com; x-originating-ip: [167.220.255.25] x-ms-office365-filtering-correlation-id: 24666eb5-2ba2-4d22-4af8-08d39e6327c2 x-microsoft-exchange-diagnostics: 1; CO2PR03MB2215; 6:5UUtfmsKlwAecWYhpD8XNqG9qr1MJjPzTXx7vfmmRGDGVCYqwoF+eXlIy+ogVD7Cf5HErtwfctGH/xKZQyHIdP8ZsZgLWtewQnEiexvd3kda73kB/WWbfPJuyxKlMwD9DPlYrAcaP+NMrrR3Hv1CeWj06YsdDIB7ZTDLwOG35uxe15yNbzAU72Ng4GaIWLhMxuG7Sq/WDVhauUOpkHRiwsXshHoy4wgBjb5gq2BX3cswJ//MWILKzDolVKl3QlCNZVy2xbrkdzSOfVtX9r+t4gLS7GsksmcBdC/LiBMEGW74c/rLGfYGbEPsmXw4yfjKbrSyzc1U4YJNFfkwsXzqOa1lqvEXddg+Ku3o9n7MbKs=; 5:h5/Y/1PZLU1n35iS6swKmGwKa2/Uwh+AZkBiYQnJ1P3mVvkFfSC8T/59Du7ewUmwsECQ7OyrzTjDfptyxdya3OAazaXVRWrYOaH9LV7JhypXZLIkYuUiLOn/3LAfiNfo; 24:HimixcBCyKLclrDu4SCUSW86jaoQ0SnMdLNsecwcNpWhES4raa8OOAD+UMxjbsShTaQ7VNwx1ohnr6DItQ+aZt86K2x3rfG3EDV3jKVei44=; 7:gV+/0KCW9HVMof5NpszqVsvzDlzbmw2kFPwV39pfqG1DhApa/1TweD0TSwV070LmKYsTXAnMcycUhre/KQZepOxYe62fqHa8NEIjhuVE8AQwDE5oHsBYLNnExyCBW8Ud788LfWSbpG/MjrPuQrIk7pB0gr4XMLhfs9hRO4vkUQ5IWTcdJjuROfSRnSCHKFPFUkSqAVCatUD6fZtVmBFV3bSA194OMlqrjf4xudJNgA9nH+/WW41yHXU8YTrZljBvpRaiPgwuDzF8nxWyi2k16w== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CO2PR03MB2215; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(21748063052155); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(102415321)(61425038)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026)(61426038)(61427038); SRVR:CO2PR03MB2215; BCL:0; PCL:0; RULEID:; SRVR:CO2PR03MB2215; x-forefront-prvs: 09860C2161 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(199003)(189002)(53754006)(54356999)(101416001)(50986999)(77096005)(76576001)(229853001)(11100500001)(2351001)(19300405004)(86362001)(2906002)(19625215002)(105586002)(92566002)(189998001)(2501003)(97736004)(110136002)(107886002)(5002640100001)(5640700001)(5630700001)(16236675004)(3280700002)(8936002)(3846002)(102836003)(6116002)(68736007)(106356001)(586003)(66066001)(33656002)(450100001)(87936001)(99286002)(3660700001)(8990500004)(2900100001)(7696003)(7736002)(86612001)(19580395003)(15975445007)(9686002)(81156014)(81166006)(10090500001)(10290500002)(122556002)(8676002)(5005710100001)(74316001)(790700001)(5003600100003)(7846002)(99936001)(10400500002); DIR:OUT; SFP:1102; SCL:1; SRVR:CO2PR03MB2215; H:CO2PR03MB2215.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Jun 2016 08:15:09.3595 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR03MB2215 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jun 2016 08:15:18 -0000 Hi all, I wrote a test to freeze ufs, but it does not work even if the ioctl return= s successful. What is the problem? #define _PATH_UFSSUSPEND "/dev/ufssuspend" static void freeze_allmountpoints() { struct statfs *mntbuf, *statfsp; int mntsize; int fso; int error; int i; fso =3D open(_PATH_UFSSUSPEND, O_RDWR); if (fso =3D=3D -1) err(1, "unable to open %s", _PATH_UFSSUSPEND); /* * First check the mounted filesystems. */ mntsize =3D getmntinfo(&mntbuf, MNT_NOWAIT); if (mntsize =3D=3D 0) return; printf("mnt size: %d\n", mntsize); for(i =3D mntsize - 1; i >=3D 0; --i) { statfsp =3D &mntbuf[i]; if (strcmp("/", statfsp->f_mntonname) =3D=3D 0 || strcmp("ufs", statfsp->f_fstypename) =3D=3D 0) { printf("begin to suspend on '%s' from '%s'\n", statfsp->f_mntonname, statfsp->f_mntfromnam= e); error =3D ioctl(fso, UFSSUSPEND, &statfsp->f_fsid); if (error !=3D 0) { //err(1, "UFSSUSPEND"); printf("error: %d\n",errno); } else { printf("Successfully suspend filesystem\n")= ; } break; } } close(fso); } Thanks Hongjiang Zhang From owner-freebsd-fs@freebsd.org Mon Jun 27 15:21:12 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8D68B84957 for ; Mon, 27 Jun 2016 15:21:12 +0000 (UTC) (envelope-from holger@freyther.de) Received: from gandharva.secretlabs.de (gandharva.secretlabs.de [IPv6:2a01:4f8:161:8201::2:4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8564624D8 for ; Mon, 27 Jun 2016 15:21:12 +0000 (UTC) (envelope-from holger@freyther.de) Received: from [192.168.61.98] (p57B9824C.dip0.t-ipconnect.de [87.185.130.76]) by gandharva.secretlabs.de (Postfix) with ESMTPSA id 0F803814BE; Mon, 27 Jun 2016 15:21:08 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Deadlock in zpool import with degraded pool From: Holger Freyther In-Reply-To: <8a4cb87252c04ebfbf71451c5dc1a41e@exch2-4.slu.se> Date: Mon, 27 Jun 2016 17:21:07 +0200 Cc: "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <9DF3E719-5184-419E-B81A-599D5ECCD969@freyther.de> References: <8a4cb87252c04ebfbf71451c5dc1a41e@exch2-4.slu.se> To: =?utf-8?Q?Karli_Sj=C3=B6berg?= X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jun 2016 15:21:12 -0000 > On 26 Jun 2016, at 19:28, Karli Sj=C3=B6berg = wrote: >=20 Hi, > That's your problem right there; dedup! You need to throw more RAM = into it until the destroy can complete. If the mobo is 'full', you need = new/other hw to cram more RAM into or you can kiss your data goodbye. = I've been in the exact same situation as you are now so I sympathize:( did you look at it further? * Why does it only start after I zfs destroyed something? The dedup = hash/table/??? grows by that? * Why a plain dead-lock and no panic? * Is there an easy way to see how much RAM is needed? (In the end I can = use Linux/KVM with RAM backed in a file/disk and just wait...) * Would you know if zpool import -o readonly avoids loading/building = that big table? =46rom common sense this block table would only be = needed on write to map from checksum to block? kind regards holger= From owner-freebsd-fs@freebsd.org Mon Jun 27 15:57:25 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0357BB841B6 for ; Mon, 27 Jun 2016 15:57:25 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qt0-x232.google.com (mail-qt0-x232.google.com [IPv6:2607:f8b0:400d:c0d::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B0B8928C4 for ; Mon, 27 Jun 2016 15:57:24 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-qt0-x232.google.com with SMTP id f89so24124006qtd.2 for ; Mon, 27 Jun 2016 08:57:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=QY93Sg3rnyvq2dF+wr45XkOiTsJglLxdyWtikHS9qQk=; b=Au0/ykQVHaPsi5bhjbPfhsJNb8LWkgpJ2bco1qqBUpMvFmkMO7iXFvzZI7SEvwBxym i8Rl+PUxE102IYFysIEYOF+1z9wweuziWR5hYkvVKPjF8zaKbYRO3TAKtI6Fffh2ITG4 9Cp6d4WnWYFbMFh/acZZbVFd2VYSuklOdaomtA9g6ZnnPUZobLxvvAVWGKBz/End+ap9 oLSLGsa+r/orRYvk5xnlhLNfSIvQxrf2UYiMARXENSbssej5hNk2XCDapRLRcp1oPTyb 5oGCERlur2rNKR70yxvo+whlyoEJluZPjgDzazlJ3LemyvVm2nXqgIJx9WizGKaNF9zw kT/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=QY93Sg3rnyvq2dF+wr45XkOiTsJglLxdyWtikHS9qQk=; b=D9s1F+wvVWphwsBlOroqlvEgSUUJCbA/YruUrdo3nJPpk/t28v8Oyls7BJG7T3S+3s Ej0diRyfnULQfdmJTiedF0qwHyKtOtaW8YhxgxI7hfHNNGPfa9kE2lbLIqfaG12svMwq YeJ8pxTfrDqsKEe4g1zV7Zg372NLeCHHRQTrOOMOgtXmAEL24IQdqjAjls8Lc4lxcIP1 qpyuLh5flg90JqkCGE+bM/zI30+rB9ezr6CzZh61QsW1Sv+RDIUk9mrTQVjUzjU56JHT qXyE9OseQ+og5fted+TsJGO/AN8p7OZs8YfDcXeVmHUFnEZvNZBNUFEdTpgwQAD6pnlm jLqA== X-Gm-Message-State: ALyK8tItYYF7L7jgJfASONhGmlvakOVuEbCbiKjVChIH5C4ZB9b441nbV8m1Ie9sjjVxd2OjNV4VVieUf4YAtA== MIME-Version: 1.0 X-Received: by 10.237.54.5 with SMTP id e5mr23980153qtb.41.1467043034265; Mon, 27 Jun 2016 08:57:14 -0700 (PDT) Received: by 10.200.56.93 with HTTP; Mon, 27 Jun 2016 08:57:14 -0700 (PDT) Received: by 10.200.56.93 with HTTP; Mon, 27 Jun 2016 08:57:14 -0700 (PDT) In-Reply-To: <9DF3E719-5184-419E-B81A-599D5ECCD969@freyther.de> References: <8a4cb87252c04ebfbf71451c5dc1a41e@exch2-4.slu.se> <9DF3E719-5184-419E-B81A-599D5ECCD969@freyther.de> Date: Mon, 27 Jun 2016 08:57:14 -0700 Message-ID: Subject: Re: Deadlock in zpool import with degraded pool From: Freddie Cash To: Holger Freyther Cc: =?UTF-8?Q?Karli_Sj=C3=B6berg?= , FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jun 2016 15:57:25 -0000 On Jun 27, 2016 8:21 AM, "Holger Freyther" wrote: > > > > On 26 Jun 2016, at 19:28, Karli Sj=C3=B6berg wro= te: > > > > Hi, > > > > > That's your problem right there; dedup! You need to throw more RAM into it until the destroy can complete. If the mobo is 'full', you need new/other hw to cram more RAM into or you can kiss your data goodbye. I've been in the exact same situation as you are now so I sympathize:( > > did you look at it further? > > * Why does it only start after I zfs destroyed something? The dedup hash/table/??? grows by that? Because every reference to every deleted block needs to be updated (decremented) in the DDT (dedupe table), which means the DDT needs to be pulled into ARC first. It's the pathological case for RAM use with dedupe enabled. :( > * Why a plain dead-lock and no panic? It's stuck trying to free RAM for ARC to load the DDT. > * Is there an easy way to see how much RAM is needed? (In the end I can use Linux/KVM with RAM backed in a file/disk and just wait...) There's a zdb command (-S or something like that) that will show the block distribution in the DDT, along with how many unique data blocks there are. You need approx 1 GB of ARC per TB of unique data, over and above any other RAM requirements for normal operation. And then double that for deleting snapshots. :( > * Would you know if zpool import -o readonly avoids loading/building that big table? From common sense this block table would only be needed on write to map from checksum to block? If you are in the "hang on import due to out-of-memory" situation, the only solution is to add more RAM (if possible) and just keep rebooting the server. Every import process will delete a little more data from the pool, update a little more of the DDT, and eventually the destroy process will complete, and the pool will be imported. The longest one for me took a little over a week of rebooting the server multiple times per day. :( We've since moved away from using dedupe. It was a great feature to have when we could only afford 400 GB drives and could get 3-5x convinced compress + dedupe ratios. Now that we can get 4-8 TB drives, it's not worth it. Cheers, Freddie From owner-freebsd-fs@freebsd.org Mon Jun 27 16:53:42 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E21C4B81157 for ; Mon, 27 Jun 2016 16:53:42 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (imap.slu.se [77.235.224.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "webmail.slu.se", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78E6C2B4D for ; Mon, 27 Jun 2016 16:53:41 +0000 (UTC) (envelope-from karli.sjoberg@slu.se) Received: from exch2-4.slu.se (77.235.224.124) by exch2-4.slu.se (77.235.224.124) with Microsoft SMTP Server (TLS) id 15.0.1156.6; Mon, 27 Jun 2016 18:38:26 +0200 Received: from exch2-4.slu.se ([fe80::f0cf:a4:cef1:ba2]) by exch2-4.slu.se ([fe80::f0cf:a4:cef1:ba2%22]) with mapi id 15.00.1156.000; Mon, 27 Jun 2016 18:38:25 +0200 From: =?utf-8?B?S2FybGkgU2rDtmJlcmc=?= To: Holger Freyther CC: "freebsd-fs@freebsd.org" Subject: Re: Deadlock in zpool import with degraded pool Thread-Topic: Deadlock in zpool import with degraded pool Thread-Index: AQHR0JJT7kkebCRlmEmmvTPK77TiTA== Date: Mon, 27 Jun 2016 16:38:25 +0000 Message-ID: Accept-Language: sv-SE, en-US Content-Language: sv-SE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jun 2016 16:53:43 -0000 DQpEZW4gMjcganVuIDIwMTYgNToyOCBlbSBza3JldiBIb2xnZXIgRnJleXRoZXIgPGhvbGdlckBm cmV5dGhlci5kZT46DQo+DQo+DQo+ID4gT24gMjYgSnVuIDIwMTYsIGF0IDE5OjI4LCBLYXJsaSBT asO2YmVyZyA8a2FybGkuc2pvYmVyZ0BzbHUuc2U+IHdyb3RlOg0KPiA+DQo+DQo+IEhpLA0KPg0K Pg0KPg0KPiA+IFRoYXQncyB5b3VyIHByb2JsZW0gcmlnaHQgdGhlcmU7IGRlZHVwISBZb3UgbmVl ZCB0byB0aHJvdyBtb3JlIFJBTSBpbnRvIGl0IHVudGlsIHRoZSBkZXN0cm95IGNhbiBjb21wbGV0 ZS4gSWYgdGhlIG1vYm8gaXMgJ2Z1bGwnLCB5b3UgbmVlZCBuZXcvb3RoZXIgaHcgdG8gY3JhbSBt b3JlIFJBTSBpbnRvIG9yIHlvdSBjYW4ga2lzcyB5b3VyIGRhdGEgZ29vZGJ5ZS4gSSd2ZSBiZWVu IGluIHRoZSBleGFjdCBzYW1lIHNpdHVhdGlvbiBhcyB5b3UgYXJlIG5vdyBzbyBJIHN5bXBhdGhp emU6KA0KPg0KPiBkaWQgeW91IGxvb2sgYXQgaXQgZnVydGhlcj8NCj4NCj4gKiBXaHkgZG9lcyBp dCBvbmx5IHN0YXJ0IGFmdGVyIEkgemZzIGRlc3Ryb3llZCBzb21ldGhpbmc/IFRoZSBkZWR1cCBo YXNoL3RhYmxlLz8/PyBncm93cyBieSB0aGF0Pw0KDQpZZXMsIGl0IG5lZWRzIHRvIGxvYWQgYWxs IHVuaXF1ZSBwb3N0cyBvZiB0aGUgZGVzdHJveSBpbnRvIFJBTSBhbmQgaXQgY2FuJ3QgYmUgc3dh cHBlZCBlaXRoZXIsIG1lYW5pbmcgdGhhdCBvbmNlIHRoZSBSQU0gaXMgZnVsbCwgdGhlIHN5c3Rl bSBsb2NrcyB1cC4NCg0KPiAqIFdoeSBhIHBsYWluIGRlYWQtbG9jayBhbmQgbm8gcGFuaWM/DQoN Ck5vIGlkZWEuIEJ1dCBJIGtub3cgaG93IHJlbHVjdGFudCBkZXZlbG9wZXJzIGhhdmUgYmVlbiBn b2luZyB0aHJvdWdoIFpGUydzIGRlZHVwIGNvZGUuIE1vc3QgaXMgbGlrZSAidHdlbnR5IGZvb3Qg cG9sZSIgYmVjYXVzZSBvZiBpdCdzIGNvbXBsZXhpdHksIEkgZ3Vlc3MsIG5vdCBhIGRldmVsb3Bl ciBteSBzZWxmLg0KDQo+ICogSXMgdGhlcmUgYW4gZWFzeSB3YXkgdG8gc2VlIGhvdyBtdWNoIFJB TSBpcyBuZWVkZWQ/DQoNCk5vLg0KDQo+IChJbiB0aGUgZW5kIEkgY2FuIHVzZSBMaW51eC9LVk0g d2l0aCBSQU0gYmFja2VkIGluIGEgZmlsZS9kaXNrIGFuZCBqdXN0IHdhaXQuLi4pDQoNCkdvb2Qg dGhpbmtpbmchIFdpc2ggSSBoYWQgdGhvdWdodCBvZiB0aGF0Li4uIEFmdGVyIEkgdHJpZWQgaW1w b3J0aW5nIHdpdGggYSBtYWNoaW5lIGVxdWlwcGVkIHdpdGggNjRHQiBJIGp1c3QgZ2F2ZSB1cC4N Cg0KPiAqIFdvdWxkIHlvdSBrbm93IGlmIHpwb29sIGltcG9ydCAtbyByZWFkb25seSBhdm9pZHMg bG9hZGluZy9idWlsZGluZyB0aGF0IGJpZyB0YWJsZT8NCg0KWWVzLCBJIHRyaWVkIHRoYXQgdG8u IE1hZGUgbm8gZGlmZmVyZW5jZSB3aGF0IHNvIGV2ZXIgdW5mb3J0dW5hdGVseS4NCg0KPiBGcm9t IGNvbW1vbiBzZW5zZSB0aGlzIGJsb2NrIHRhYmxlIHdvdWxkIG9ubHkgYmUgbmVlZGVkIG9uIHdy aXRlIHRvIG1hcCBmcm9tIGNoZWNrc3VtIHRvIGJsb2NrPw0KDQpDb21tb24gc2Vuc2U/ISBJIHdp c2ggdGhlcmUgd2FzIG1vcmUgb2YgdGhhdCBpbiBaRlMncyBkZWR1cCBjb2RlOikNCg0KL0sNCg0K Pg0KPiBraW5kIHJlZ2FyZHMNCj4gICAgICAgICBob2xnZXINCg== From owner-freebsd-fs@freebsd.org Tue Jun 28 07:17:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59C47B8556B for ; Tue, 28 Jun 2016 07:17:30 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCED12111 for ; Tue, 28 Jun 2016 07:17:29 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm0-x231.google.com with SMTP id r201so13216504wme.1 for ; Tue, 28 Jun 2016 00:17:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=YjRHhwpV7Gaksw1Y4r/Ij7+DeA9pVxpO6yjMsS/q1RE=; b=KkIFudrUzWRyt5AwQM9ZWdpkDhMr2fSOam/z7P94l2zZrrkMvLm0DQZqpe9ABqWNPV n+zZOLLDUXUxLKJsOXFYdjszE+f0x6wYj/BMRW2w7UbG5T5wBineGVPj40RN/MJvEBcB uFljysboCQBj8YpqsvAKeH9Ks5RdL2jRLRPQkyvgkZSMqlWKuyQPfOHFbMSoM7lX0cGm SJ4OGho6dgm/750ZYElpW/7tgZW1ZCZA4lPVp0ChYxHER+XjzbOZZtO9EP1WW3AZCOto 2Vp2kBDGm0IpfDWHmPl3kCydV3WVjbpKYhKkNvROT/CgO6Ye48qZ+fxnvrV3EVBM1VyG 0JGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=YjRHhwpV7Gaksw1Y4r/Ij7+DeA9pVxpO6yjMsS/q1RE=; b=PCP/ZuSTEwJ5/11m3FjIysPxY1r6XG8wfa6CfBgXlA6YuyFp3B1tWmPgIWetqZ3ktN lelOqDYVp04qm+I0sYOQLJC89zDxOxhIK5IthfRU0JD+iH8HEO1DVDX691loR1EPEOO/ dkFwd5uqSS7WK5Mc+A+mJSKKavQ997W/OJLGhIefk5jXia22qy924tDBbdVLYwWZddoh o/5ju5DOipkkNzxFqISsdDY/y5mfP9IBJMlgVWFS1k/zwPgO3PM6c3xSSFBZDR0VpRgv Eyzi9NhGVrSFz9T8lW8F//yb58go/WenEVZousOqb4rZqLXm6pxMbQuU3xtQLapAFAOV YSqw== X-Gm-Message-State: ALyK8tLv7LI2W6ymERHBTHbdEaubQ1tUmLwqeJ+ko4cAc+mVFlX5o/cQvtZeOTxACRkWNw== X-Received: by 10.194.96.208 with SMTP id du16mr1647633wjb.0.1467098247374; Tue, 28 Jun 2016 00:17:27 -0700 (PDT) Received: from brick (eud22.neoplus.adsl.tpnet.pl. [83.20.175.22]) by smtp.gmail.com with ESMTPSA id i14sm317138wmg.7.2016.06.28.00.17.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Jun 2016 00:17:26 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Tue, 28 Jun 2016 08:54:32 +0200 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Hongjiang Zhang Cc: "freebsd-fs@freebsd.org" Subject: Re: ufs freeze does not work Message-ID: <20160628065432.GA20716@brick> Mail-Followup-To: Hongjiang Zhang , "freebsd-fs@freebsd.org" References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 07:17:30 -0000 On 0627T0815, Hongjiang Zhang via freebsd-fs wrote: > Hi all, > > I wrote a test to freeze ufs, but it does not work even if the ioctl returns successful. What is the problem? What do you mean by 'does not work'? What happens, and what did you expect to happen? Regarding your example - remember that the filesystem gets automatically unsuspended as soon as you close the /dev/ufssuspend file descriptor. From owner-freebsd-fs@freebsd.org Tue Jun 28 08:06:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE930B8549C for ; Tue, 28 Jun 2016 08:06:54 +0000 (UTC) (envelope-from honzhan@microsoft.com) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2on0111.outbound.protection.outlook.com [65.55.169.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 93CDA2CF9; Tue, 28 Jun 2016 08:06:53 +0000 (UTC) (envelope-from honzhan@microsoft.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=1rmFYydiYDfP4h6UPVyVTe2Fh78a2JLBith68q4kcvU=; b=UDzp0Sd1sRNyBjwVHrlzbQUM5wPWIvvcuCjpEVYVYRtsDtcmKWZRNe2YndmYkGdjy8Xs590PS+b022Rb8upBmjNXr3SMKGI1nd5+8h5AuzlHAEANc9jfeJVi+YkEwEhkDddaJwaK7DuZO2aKzskCe/fPYatTFLP5DwpnCiRuhPY= Received: from SN2PR03MB2224.namprd03.prod.outlook.com (10.166.209.155) by SN2PR03MB2222.namprd03.prod.outlook.com (10.166.209.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.528.16; Tue, 28 Jun 2016 07:33:18 +0000 Received: from SN2PR03MB2224.namprd03.prod.outlook.com ([10.166.209.155]) by SN2PR03MB2224.namprd03.prod.outlook.com ([10.166.209.155]) with mapi id 15.01.0528.017; Tue, 28 Jun 2016 07:33:18 +0000 From: Hongjiang Zhang To: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= CC: "freebsd-fs@freebsd.org" Subject: RE: ufs freeze does not work Thread-Topic: ufs freeze does not work Thread-Index: AdHQS4KXvmVyW6W2TjqUQxMwTLuTxQAvmm4AAADTrfA= Date: Tue, 28 Jun 2016 07:33:18 +0000 Message-ID: References: <20160628065432.GA20716@brick> In-Reply-To: <20160628065432.GA20716@brick> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=honzhan@microsoft.com; x-originating-ip: [167.220.255.25] x-ms-office365-filtering-correlation-id: fa22d889-80ca-45c1-fb67-08d39f267988 x-microsoft-exchange-diagnostics: 1; SN2PR03MB2222; 6:ANxuG5L5wRut8IhyQwW/ABPNCbZtjna6Wq6R1590/PAcxeX2NcZ29orlGPvOEx3fIp1ROduryqm7V9c6VTUIHOGIWY2YhtxwsyOit1e1qJ3LF1pDqsZ8f19Cg5Ic0ao+yXTYwwqSZsEwqT8Qk/+8w/YPNChPScCqm4LZPqQflQy0iTqwM2kM9Cc2wPIG2OycMWLx7/QwJiezQ4CHaMMlKj0JMDO1REdyERjhA68gZp9NLt6X80ndEoVYjBFn1QEE+fmZOYDZR8jcgl8ItyeNCBto44e7BqKX+wXiINjv2qbfc5VxQWzHLdpcdEag0Jhjv4/R3OBvkx8Pzbl85wdvwg==; 5:ViwU/kHpDmT9QVvJtzLQeI73b6FpkVVHRNIT5WrgZLBN7hQYSKHHORIx+y81Z2mlebaX1IRVzD5VI+vd+9P0OuxeLA++LlegetlZa0W72tg2R3qb5voRLNyk/GJPVzp49Vj5BdP6K1MLilBTOQyKzw==; 24:FP7TFVGV/ZAzOmNJhUyK6qdFPnwExjLGlehBTDzg9ufAZy7ZGGg12UDlklAqedhtlRLZO1P0alxiYNHA7BiEDJql80hCCCSwU3ySQvGdk6w=; 7:nDc4WBR6+l+Bdwgdxji6Vy3FFdU4C9aqSyonMFMsL3hn6HqTqeNlqy8ZtyXkie1B17MY1KHoZCbqKZADDVuxQ8+vsPJcI1XeyOKoiotneKOnC9D/IDxyqz888cWpqcVE1WKgFYbzz+r1iTge04wMaef6IfIdTd0Y3GBJa8xHIdNw13yMwbrWVhVBnR6gMab/9+cW83cn62H8nWuzBQsKsuSNcc1uXkeUOVCLmokXm2FDuf8kfaOfQhegLaYM9xef x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN2PR03MB2222; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425038)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6055026)(61426038)(61427038); SRVR:SN2PR03MB2222; BCL:0; PCL:0; RULEID:; SRVR:SN2PR03MB2222; x-forefront-prvs: 0987ACA2E2 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(199003)(53754006)(189002)(377454003)(13464003)(24454002)(2900100001)(2950100001)(77096005)(33656002)(81156014)(586003)(81166006)(8676002)(74316001)(4326007)(86362001)(450100001)(102836003)(105586002)(66066001)(8936002)(87936001)(7696003)(7736002)(305945005)(106356001)(5003600100003)(11100500001)(7846002)(99286002)(92566002)(9686002)(3846002)(10090500001)(8990500004)(6116002)(97736004)(3280700002)(101416001)(189998001)(110136002)(19580395003)(19580405001)(68736007)(76576001)(86612001)(3660700001)(122556002)(2906002)(10400500002)(10290500002)(76176999)(5005710100001)(54356999)(50986999)(5002640100001); DIR:OUT; SFP:1102; SCL:1; SRVR:SN2PR03MB2222; H:SN2PR03MB2224.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jun 2016 07:33:18.4814 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR03MB2222 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 08:06:54 -0000 I run "./freeze -f", the program should freeze the "/" file partition, but = I can still write something to "/tmp" folder. -----Original Message----- From: Edward Tomasz Napiera=B3a [mailto:etnapierala@gmail.com] On Behalf Of= Edward Tomasz Napierala Sent: Tuesday, June 28, 2016 2:55 PM To: Hongjiang Zhang Cc: freebsd-fs@freebsd.org Subject: Re: ufs freeze does not work On 0627T0815, Hongjiang Zhang via freebsd-fs wrote: > Hi all, >=20 > I wrote a test to freeze ufs, but it does not work even if the ioctl retu= rns successful. What is the problem? What do you mean by 'does not work'? What happens, and what did you expect= to happen? Regarding your example - remember that the filesystem gets automatically un= suspended as soon as you close the /dev/ufssuspend file descriptor. From owner-freebsd-fs@freebsd.org Tue Jun 28 13:05:08 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B869FB85020; Tue, 28 Jun 2016 13:05:08 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A5362D4E; Tue, 28 Jun 2016 13:05:08 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-236-103.lns20.per1.internode.on.net [121.45.236.103]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u5SD4pkp024895 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 28 Jun 2016 06:04:54 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: The small installations network filesystem and users. To: =?UTF-8?Q?Gerrit_K=c3=bchn?= , Daniel Eischen References: <9BB7E8B3-EC0E-457E-B2B2-FB80B1CF02B0@gmail.com> <20160621075631.38c2eeaa7c224aa22ea9be4d@aei.mpg.de> Cc: freebsd-fs , FreeBSD Hackers From: Julian Elischer Message-ID: <761f82d3-ebe9-2cba-9499-049dafbc98df@freebsd.org> Date: Tue, 28 Jun 2016 21:04:45 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160621075631.38c2eeaa7c224aa22ea9be4d@aei.mpg.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 13:05:08 -0000 On 21/06/2016 1:56 PM, Gerrit Kühn wrote: > On Mon, 20 Jun 2016 22:00:31 -0400 (EDT) Daniel Eischen > wrote about Re: The small installations network > filesystem and users.: > > DE> We should support LDAP client out of the box, in base. What > DE> sucks now is that we need 3 packages (plus their dependencies) > DE> and multiple config files for ldap: > DE> > DE> pam_ldap > DE> nss_ldap > DE> openldap-client > > I only have to install/config ldap-clients every now and then, but I would > also strongly favour a more "integrated" setup (if that requires having it > in base is a different question, though). A few weeks ago I used > nss-pam-ldapd instead of pam_ldap and nss_ldap for the first time, and it > appeared to work with a bit less of a hassle for me (otoh, I don't do any > funky things here, I just need a replacement for what we did with NIS > something like 20 years ago). +1 I just had to reinstall certs for my server. which means copying certs to several places (in a default config) sendmail and syrus ad openssl (base) all look in different places. you COULD make them all look in the same place but that requires undersanding what is going on and not just cribbing the config file off the net somewhere. I think ports and pkg are fine, but we need to have some more thought put into how they all go together. > > > cu > Gerrit > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Jun 28 15:23:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC989B85AED; Tue, 28 Jun 2016 15:23:33 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B02B2EE0; Tue, 28 Jun 2016 15:23:32 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.14.9) with ESMTPS id u5SFNTB1024866 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 28 Jun 2016 17:23:29 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id u5SFNRX6008891; Tue, 28 Jun 2016 17:23:27 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id u5SFNM4q008888; Tue, 28 Jun 2016 17:23:22 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Tue, 28 Jun 2016 17:23:22 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Chris Watson cc: Zaphod Beeblebrox , freebsd-fs , FreeBSD Hackers Subject: Re: The small installations network filesystem and users. In-Reply-To: <9BB7E8B3-EC0E-457E-B2B2-FB80B1CF02B0@gmail.com> Message-ID: References: <9BB7E8B3-EC0E-457E-B2B2-FB80B1CF02B0@gmail.com> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Tue, 28 Jun 2016 17:23:30 +0200 (CEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 15:23:33 -0000 On Mon, 20 Jun 2016, Chris Watson wrote: > I'm glad you brought this up. I wanted to but I've heard it before on the lists and realize that there is this disconnect between the developers doing the actual work to implement these things and the end users. > > I have always been very grateful to all the developers who over the years, and I've been a FreeBSD consumer since the late 90s? And attended my first usenix/freenix conf in Monterrey in 2001?, have done some really hard work on many many things in FreeBSD. For zero pay. But the thing that has always bothered me about a lot of it is, it's just to complex to use for most end users. Not all. But people want to get work done. Sifting through .conf files, googling howtos, spending more time configuring it than installing it has always been an issue. Developers in general do not think like an end user. And this leads to non developers just going "screw it I'll just get it running on Linux with my GUI installer." Which is why FreeNas is so popular. It's taken a lot, not all, but a lot of the pain and time consuming nature of learning all the ins and outs of a NAS appliance from the equation. FreeBSD have very well done man pages. I simply read them. From owner-freebsd-fs@freebsd.org Tue Jun 28 15:24:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D0D8B85BA4; Tue, 28 Jun 2016 15:24:23 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 116652070; Tue, 28 Jun 2016 15:24:22 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.14.9) with ESMTPS id u5SFMe0R024841 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 28 Jun 2016 17:22:41 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id u5SFMcRj008885; Tue, 28 Jun 2016 17:22:38 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id u5SFMXSO008882; Tue, 28 Jun 2016 17:22:33 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Tue, 28 Jun 2016 17:22:33 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Zaphod Beeblebrox cc: freebsd-fs , FreeBSD Hackers Subject: Re: The small installations network filesystem and users. In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Tue, 28 Jun 2016 17:22:41 +0200 (CEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 15:24:23 -0000 > Correct me if I'm wrong, but amidst discussions of pNFS (among other > things) I thought I should bring up something someone said to me: and that > is (to quote him) "using NFS is too hard, I always fail." Never had problems with NFS. And i'm using in regularly. From owner-freebsd-fs@freebsd.org Tue Jun 28 19:18:20 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E816DB854BB for ; Tue, 28 Jun 2016 19:18:20 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78B3C2716 for ; Tue, 28 Jun 2016 19:18:20 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm0-x234.google.com with SMTP id v199so152427045wmv.0 for ; Tue, 28 Jun 2016 12:18:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=g7tIn69C2FIcOg6Wsz0XcX+Kfle3PW4i1DaGRo/JeNE=; b=0bKPftvIy4dKx2JNXSHtoZUcqyjCfr6w+53u5DfDUqpGurBlt6v+d0vwyKiWd2JvlM 0fNb8XbyyuBoWzUNt+4ungsVgZi2X8jA3P/6gZk1WBlYwl/YPE2/CD0mleffA3E5Sahj 9rmJqXg5k00OOiSS8dFnAuH0Vp3dR8/PccYrDPI6V6MM1o30Py4lNW4R7iPsQbqkllp7 enScPiQd4MmpuAm1coIGgcdY3k/YYXal/7FufLaH9T501d1eB7aZL4rmgdaE5ssLVDYC zD2vOxeJ+7eZshXq59hKLffYmqtdCAHdGbGYtwjsGO7oQlyFyI9w8/RD01eZ0e/SW+yX IemQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=g7tIn69C2FIcOg6Wsz0XcX+Kfle3PW4i1DaGRo/JeNE=; b=XzY0Wq9x9oSiNynx6Hx5oYZnCPJU0kIpsIh+6NaLXfFUQzZgSZzcwd0FmdoOtoTOpw YCC9EMtyRVn6jKuhBD7n26AMbTbfi4OWmoCYgaaohx+BzeUAKW/7Bnpf4pdyhfrbSDK3 I2AtZzqykCtrW9m+DunetPSTztnQZIGcmODvkRuHb1hCfSOJGel0b06EwM6FoPBWcgHi 49e/8I4HNEdcafU62jl9r/w4hQLL6Dz+IE70qiSpu81gEeFBxOgSid3fNR5ARDMwCL6b 1L9hu/Z8UWddYffiRnB0QDFx9hKYa0LPbVD/UIbJcvSOwbv6zYWHEry7hb7NTHZHEVxj DSDg== X-Gm-Message-State: ALyK8tLemBY2STkgnCIBpJ3zhp8rNskmP1+i0u29krbv6/T8AJV0cmzDqjZLU8hlCuzlRA== X-Received: by 10.28.184.5 with SMTP id i5mr4853734wmf.85.1467141498883; Tue, 28 Jun 2016 12:18:18 -0700 (PDT) Received: from brick (adjg143.neoplus.adsl.tpnet.pl. [79.184.214.143]) by smtp.gmail.com with ESMTPSA id e16sm4747659wma.12.2016.06.28.12.18.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Jun 2016 12:18:17 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Tue, 28 Jun 2016 20:55:23 +0200 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Hongjiang Zhang Cc: "freebsd-fs@freebsd.org" Subject: Re: ufs freeze does not work Message-ID: <20160628185523.GA82035@brick> Mail-Followup-To: Hongjiang Zhang , "freebsd-fs@freebsd.org" References: <20160628065432.GA20716@brick> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 19:18:21 -0000 As I said, the suspension is released when the ufssuspend file descriptor gets closed - which is what happens when the calling process exits. It's a protection mechanism, to avoid the situation where the process malfunction (eg a crash) would leave the system in unrecoverable (suspended) state. You probably want your process to just execute another one, and wait until it exits. On 0628T0733, Hongjiang Zhang wrote: > I run "./freeze -f", the program should freeze the "/" file partition, but I can still write something to "/tmp" folder. > > -----Original Message----- > From: Edward Tomasz NapieraÅ‚a [mailto:etnapierala@gmail.com] On Behalf Of Edward Tomasz Napierala > Sent: Tuesday, June 28, 2016 2:55 PM > To: Hongjiang Zhang > Cc: freebsd-fs@freebsd.org > Subject: Re: ufs freeze does not work > > On 0627T0815, Hongjiang Zhang via freebsd-fs wrote: > > Hi all, > > > > I wrote a test to freeze ufs, but it does not work even if the ioctl returns successful. What is the problem? > > What do you mean by 'does not work'? What happens, and what did you expect to happen? > > Regarding your example - remember that the filesystem gets automatically unsuspended as soon as you close the /dev/ufssuspend file descriptor. > From owner-freebsd-fs@freebsd.org Tue Jun 28 19:57:39 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F728B862FD for ; Tue, 28 Jun 2016 19:57:39 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02C272049 for ; Tue, 28 Jun 2016 19:57:39 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id f126so153901337wma.1 for ; Tue, 28 Jun 2016 12:57:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=rATWQx3fnSMCH+HgNCyfh4FQ82ibPiuaMJxvv2yL0ZA=; b=IP0LtCydT+Jhu5R7fzp10QN2DEOr3yE+LppTTc2lJwzTXi6cT5B9PumrqAU4x7jkap fBpyPimx2uCfYrmbljjBYMaVnEo+HHLqh9xWs9KTjejrd0mEzk1YbObbSVCFqaQvnIwT GEk9SjIRHhATO5oM9TZBq7wgKd7e+V+5kALIMYP8kl7cYOzXj9wQph7H//WKURt10HDY mdB3WmqHSXK10+qSB3YfnmcCP/gC7RNz5hi8+/bn/NqMj6q5hwRaGI95o7+/99jOFXVe XjLyMQQKdFXu7rzdai1LUuGl3MxQdtzRGTxuPRZ3aJ4cWJAuIRHxZhhO8TsPlYaavukI xe/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=rATWQx3fnSMCH+HgNCyfh4FQ82ibPiuaMJxvv2yL0ZA=; b=ZCPzrlU5Sc6XK8GZqdgOeUDd82+8t1yJQsR0SFk30G1A3WCGOXyDCGPd+lDnR1miLw /HWXRh6MyhjjeNeaegeB1n1aHTY9QurTv8iIAnUHzxEAXsCdnCw7fGdisS4vpbtLMUre gz+gbWEaSIH/IUZ4qgtsqiPLhdp5F9007ULKXn+4mA8XbMG6N8DYa6otJFmtPrHmKLye ULw8o5mjlfR5ZtiIPusS24raetoAi3R/a8n/ITMN3AMsFZy+v4fLRDvODl78vmnU5BJF vUZCSUlSW4HQV+QVnNUWNfE4XYwVy3Yd5H079ScktQjD0Es039DlDe6sQzmHtSMgoJH9 dzKQ== X-Gm-Message-State: ALyK8tJ6KWW65WUeRG6asuLUpRj71/hYoOnkw+C4JcIrAkCsPcaC4XPholMMUoAnHI9hiQ== X-Received: by 10.194.67.101 with SMTP id m5mr5174642wjt.129.1467143857448; Tue, 28 Jun 2016 12:57:37 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by smtp.gmail.com with ESMTPSA id qg5sm12761wjc.13.2016.06.28.12.57.33 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Tue, 28 Jun 2016 12:57:33 -0700 (PDT) Date: Tue, 28 Jun 2016 21:57:31 +0200 From: Mateusz Guzik To: Hongjiang Zhang , "freebsd-fs@freebsd.org" Subject: Re: ufs freeze does not work Message-ID: <20160628195731.GA21323@dft-labs.eu> References: <20160628065432.GA20716@brick> <20160628185523.GA82035@brick> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20160628185523.GA82035@brick> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2016 19:57:39 -0000 On Tue, Jun 28, 2016 at 08:55:23PM +0200, Edward Tomasz NapieraÅ‚a wrote: > As I said, the suspension is released when the ufssuspend file descriptor > gets closed - which is what happens when the calling process exits. It's > a protection mechanism, to avoid the situation where the process malfunction > (eg a crash) would leave the system in unrecoverable (suspended) state. > > You probably want your process to just execute another one, and wait until > it exits. > The example with freeze -f strongly hints this is supposed to work as a drop in replacement for linux scripts. As such, maybe ufs should grow another operation which does not automagically unfreeze. > On 0628T0733, Hongjiang Zhang wrote: > > I run "./freeze -f", the program should freeze the "/" file partition, but I can still write something to "/tmp" folder. > > > > -----Original Message----- > > From: Edward Tomasz NapieraÅ‚a [mailto:etnapierala@gmail.com] On Behalf Of Edward Tomasz Napierala > > Sent: Tuesday, June 28, 2016 2:55 PM > > To: Hongjiang Zhang > > Cc: freebsd-fs@freebsd.org > > Subject: Re: ufs freeze does not work > > > > On 0627T0815, Hongjiang Zhang via freebsd-fs wrote: > > > Hi all, > > > > > > I wrote a test to freeze ufs, but it does not work even if the ioctl returns successful. What is the problem? > > > > What do you mean by 'does not work'? What happens, and what did you expect to happen? > > > > Regarding your example - remember that the filesystem gets automatically unsuspended as soon as you close the /dev/ufssuspend file descriptor. > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Mateusz Guzik From owner-freebsd-fs@freebsd.org Wed Jun 29 01:21:11 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6BA9EB81AE8 for ; Wed, 29 Jun 2016 01:21:11 +0000 (UTC) (envelope-from magoil@magoil.it) Received: from vps186535.ovh.net (unknown [IPv6:2001:41d0:52:bff::391]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 32FCC204A for ; Wed, 29 Jun 2016 01:21:11 +0000 (UTC) (envelope-from magoil@magoil.it) Received: by vps186535.ovh.net (Postfix, from userid 10000) id 06D5C1FE502; Tue, 28 Jun 2016 11:48:28 +0200 (CEST) To: "freebsd-fs" Subject: Re: [Ticket C-7396-814991873] your account X-PHP-Originating-Script: 10000:config.php Message-ID: From: "Money System" Date: Tue, 28 Jun 2016 02:48:29 -0700 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="windows-1251"; reply-type=original Content-Transfer-Encoding: quoted-printable X-Priority: 1 X-MSMail-Priority: High X-Mailer: Microsoft Windows Live Mail 14.0.8117.416 X-MimeOLE: Produced By Microsoft MimeOLE V14.0.8117.416 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2016 01:21:11 -0000 Attention! Check your money balance on http://room.opticalrooms.ie/wp-cron/user/1.ht= m Money System Support From owner-freebsd-fs@freebsd.org Wed Jun 29 04:55:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03C68B841F0 for ; Wed, 29 Jun 2016 04:55:01 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA1B12B67 for ; Wed, 29 Jun 2016 04:55:00 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm0-x22e.google.com with SMTP id 187so30981320wmz.1 for ; Tue, 28 Jun 2016 21:55:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=9+kfQ63hrG44GFP/Qbn9CM/tyBcJ7vgtGmUSMJNOy6I=; b=wrFJ50uTZSL4+7/0+oGREHcOtxYPmccmanZiWbn2JSn9o8uamAIEwRvrV/Ti3UyRwB hs2N3bJeKViE8kD9Xm1hPE3luJD5UA9cedjwFXGKFA4gf9lEFiULqJ1ld+tIbmnmFi2g 5ttuap5suLz7Jg7IkGgauPkMlEVFeEZGsnTpKhwQ0l6pxHlQVsduExqxJH74Tcikbrd4 DuBivnqMF0YpJCTGL3i/HfoyUGunB/JN/YLTswbaetonUtl9rP+kaU+goDnQS/IhMbA7 45HIFEBcpwNWIt9JICYloTUzAshUKHYSKt+UWCv+KOqhUFCcMVO/jDI/pis6OtYsmPGF QdiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=9+kfQ63hrG44GFP/Qbn9CM/tyBcJ7vgtGmUSMJNOy6I=; b=QEp6UvcV88y+04VMhqURfFSCXEcQQw5hbglnjXQ6Yln7ibWxgXzMK8Y8p6XqViLtOK 6iNhauwPWhJm7wD/YkMqrVHwB+DRmWzQTQrMlGrxxERu+ZEIDno5UdQKC8ItJFsunYMM 0DK+wgA3Eh5Yc/hY9yu7nXsbqP3GNQ30BufV6ujOo9DMGHsCvZI+M5b/r3DciLyZiKgL HtYik/5oj1MkecgXUbSrF/OSoGCkBqUWt9reJ9Nu0TxnvcUjFFFWjPrhWet8HRfSVQ1f zJZJeROUy3TDCylVm1yxuA2DIpOYV7vLQ2c4Yeaz69k5fSCfFqDbeA82ORIXd57XFqub k99w== X-Gm-Message-State: ALyK8tKBBbXqwC8bQkKMsTEOi+EYQV/eJIDUcoLrhjkEtcXcMVbclT2u33fkugoCHN0p5w== X-Received: by 10.194.191.135 with SMTP id gy7mr6313912wjc.125.1467176098151; Tue, 28 Jun 2016 21:54:58 -0700 (PDT) Received: from brick (adjg143.neoplus.adsl.tpnet.pl. [79.184.214.143]) by smtp.gmail.com with ESMTPSA id ib10sm1596066wjb.31.2016.06.28.21.54.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Jun 2016 21:54:57 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Wed, 29 Jun 2016 06:32:03 +0200 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Mateusz Guzik Cc: Hongjiang Zhang , "freebsd-fs@freebsd.org" Subject: Re: ufs freeze does not work Message-ID: <20160629043203.GA82400@brick> Mail-Followup-To: Mateusz Guzik , Hongjiang Zhang , "freebsd-fs@freebsd.org" References: <20160628065432.GA20716@brick> <20160628185523.GA82035@brick> <20160628195731.GA21323@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20160628195731.GA21323@dft-labs.eu> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2016 04:55:01 -0000 On 0628T2157, Mateusz Guzik wrote: > On Tue, Jun 28, 2016 at 08:55:23PM +0200, Edward Tomasz NapieraÅ‚a wrote: > > As I said, the suspension is released when the ufssuspend file descriptor > > gets closed - which is what happens when the calling process exits. It's > > a protection mechanism, to avoid the situation where the process malfunction > > (eg a crash) would leave the system in unrecoverable (suspended) state. > > > > You probably want your process to just execute another one, and wait until > > it exits. > > > > The example with freeze -f strongly hints this is supposed to work as a > drop in replacement for linux scripts. > > As such, maybe ufs should grow another operation which does not > automagically unfreeze. I'm not sure it's a good idea to provide an inferior mechanism just for backward compatibility with Linux. Especially given how easy it is to do it properly, modeling the utility after eg lockf(1). From owner-freebsd-fs@freebsd.org Thu Jun 30 15:04:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FE0CB87917 for ; Thu, 30 Jun 2016 15:04:01 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b03.edpnet.be (relay-b03.edpnet.be [212.71.1.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5A9B2A1D for ; Thu, 30 Jun 2016 15:04:00 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467297946-0a88181ce55a0040001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b03.edpnet.be with ESMTP id esbEPwCyqcL6Li5F (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 30 Jun 2016 16:45:48 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Thu, 30 Jun 2016 16:45:46 +0200 From: Julien Cigar To: freebsd-fs@freebsd.org Subject: HAST + ZFS + NFS + CARP Message-ID: <20160630144546.GB99997@mordor.lan> X-ASG-Orig-Subj: HAST + ZFS + NFS + CARP MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="1UWUbFP1cBYEclgG" Content-Disposition: inline User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467297947 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.220:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2957 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4783 1.0000 0.0000 X-Barracuda-Spam-Score: 3.13 X-Barracuda-Spam-Status: No, SCORE=3.13 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=BSF_SC0_MV0713, BSF_SC0_MV0713_2, SUBJ_ALL_CAPS, SUBJ_ALL_CAPS_2 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30897 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 SUBJ_ALL_CAPS Subject is all capitals 1.62 SUBJ_ALL_CAPS_2 SUBJ_ALL_CAPS_2 0.50 BSF_SC0_MV0713 Custom rule MV0713 1.00 BSF_SC0_MV0713_2 BSF_SC0_MV0713_2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:04:01 -0000 --1UWUbFP1cBYEclgG Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, I'm always in the process of setting a redundant low-cost storage for=20 our (small, ~30 people) team here. I read quite a lot of articles/documentations/etc and I plan to use HAST with ZFS for the storage, CARP for the failover and the "good old NFS" to mount the shares on the clients. The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for the shared storage. Assuming the following configuration: - MASTER is the active node and BACKUP is the standby node. - two disks in each machine: ada0 and ada1. - two interfaces in each machine: em0 and em1 - em0 is the primary interface (with CARP setup) - em1 is dedicated to the HAST traffic (crossover cable) - FreeBSD is properly installed in each machine. - a HAST resource "disk0" for ada0p2. - a HAST resource "disk1" for ada1p2. - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is created on MASTER A couple of questions I am still wondering: - If a disk dies on the MASTER I guess that zpool will not see it and will transparently use the one on BACKUP through the HAST ressource.. is it a problem? could this lead to some corruption? At this stage the common sense would be to replace the disk quickly, but imagine the worst case scenario where ada1 on MASTER dies, zpool will not see it=20 and will transparently use the one from the BACKUP node (through the=20 "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not=20 see it and will transparently use the one from the BACKUP node=20 (through the "disk0" HAST ressource). At this point on MASTER the two=20 disks are broken but the pool is still considered healthy ... What if=20 after that we unplug the em0 network cable on BACKUP? Storage is down.. - Under heavy I/O the MASTER box suddently dies (for some reasons),=20 thanks to CARP the BACKUP node will switch from standy -> active and=20 execute the failover script which does some "hastctl role primary" for the ressources and a zpool import. I wondered if there are any situations where the pool couldn't be imported (=3D data corruption)? For example what if the pool hasn't been exported on the MASTER before it dies? - Is it a problem if the NFS daemons are started at boot on the standby node, or should they only be started in the failover script? What about stale files and active connections on the clients? - A catastrophic power failure occur and MASTER and BACKUP are suddently powered down. Later the power returns, is it possible that some problem occur (split-brain scenario ?) regarding the order in which the two machines boot up? - Other things I have not thought? Thanks! Julien --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --1UWUbFP1cBYEclgG Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdTCWAAoJELK7NxCiBCPA6igQANRq1XO0qCQ40ZQv5xctdLNI 3NMR10J2Ghf2X59WmvLg0a7qqwxjn1e7Km9XYUfr05UgLFAr8OCXY6gDHiR8tO1l 9V3gKI3iqEjfOKdmzhrE22oow1gSzvjvA5pzTRGQDwmzNQbirtqH+xieRyP++mXN frXyR2SAQxChnJX4vPI7ysbRtR1V2l+QMyVT/aFoJFNTjZ/XxD9M4M7TQsW3zU46 bY18us9CN+inLs/RcPgccDQVqMpkGaDRqsrDhhrCWFklss0dNMYbU9N9uwFR+yaZ 23Li73SmJdlzCsr54z5ZN8iieFW2wCH3wALtaogc/1fdhTYClNHv6eBTEiuxdLK3 UkTFGdLHaeKe4ViIgq8gZfdGUryBfmThWiwsj+Aka3e4fMR78hqAjLCvmKOEwl75 Mak+vxP5p1L6xpmtwfk/oLVyua+eKMOnA+pEQwhttla1ACxOfketPnDamMf/BOEH nuSPuKD2AJrEQzXiONQkKEl25EHFiaF0RPWeWxT5g8jWdC/wsDhfoQZFVzMoqPzT skIfnLarO+JQuynPs2JxBCBPSgGpbv3xEhHfdEebUZlMqc2UI+CstqMt4v6+/kQv davNxLQg30/aMTq0GP31aryEmyq9TBVdTbol5lTFql9bSWmv/4RBe6fy3twip/CD 4RRWJkhwr5phiBWrp9Jf =6ezE -----END PGP SIGNATURE----- --1UWUbFP1cBYEclgG-- From owner-freebsd-fs@freebsd.org Thu Jun 30 15:24:11 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B199BB87D94 for ; Thu, 30 Jun 2016 15:24:11 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id 45753227A for ; Thu, 30 Jun 2016 15:24:10 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 1C7184C4C84C; Thu, 30 Jun 2016 17:14:13 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hzs+ZxqUWwOi; Thu, 30 Jun 2016 17:14:11 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id EB0C445FC0E4; Thu, 30 Jun 2016 17:14:10 +0200 (CEST) Subject: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> To: Julien Cigar , freebsd-fs@freebsd.org Reply-To: jg@internetx.com From: InterNetX - Juergen Gotteswinter Message-ID: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> Date: Thu, 30 Jun 2016 17:14:08 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160630144546.GB99997@mordor.lan> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:24:11 -0000 Am 30.06.2016 um 16:45 schrieb Julien Cigar: > Hello, > > I'm always in the process of setting a redundant low-cost storage for > our (small, ~30 people) team here. > > I read quite a lot of articles/documentations/etc and I plan to use HAST > with ZFS for the storage, CARP for the failover and the "good old NFS" > to mount the shares on the clients. > > The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for the > shared storage. > > Assuming the following configuration: > - MASTER is the active node and BACKUP is the standby node. > - two disks in each machine: ada0 and ada1. > - two interfaces in each machine: em0 and em1 > - em0 is the primary interface (with CARP setup) > - em1 is dedicated to the HAST traffic (crossover cable) > - FreeBSD is properly installed in each machine. > - a HAST resource "disk0" for ada0p2. > - a HAST resource "disk1" for ada1p2. > - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is created > on MASTER > > A couple of questions I am still wondering: > - If a disk dies on the MASTER I guess that zpool will not see it and > will transparently use the one on BACKUP through the HAST ressource.. thats right, as long as writes on $anything have been successful hast is happy and wont start whining > is it a problem? imho yes, at least from management view > could this lead to some corruption? probably, i never heard about anyone who uses that for long time in production At this stage the > common sense would be to replace the disk quickly, but imagine the > worst case scenario where ada1 on MASTER dies, zpool will not see it > and will transparently use the one from the BACKUP node (through the > "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not > see it and will transparently use the one from the BACKUP node > (through the "disk0" HAST ressource). At this point on MASTER the two > disks are broken but the pool is still considered healthy ... What if > after that we unplug the em0 network cable on BACKUP? Storage is > down.. > - Under heavy I/O the MASTER box suddently dies (for some reasons), > thanks to CARP the BACKUP node will switch from standy -> active and > execute the failover script which does some "hastctl role primary" for > the ressources and a zpool import. I wondered if there are any > situations where the pool couldn't be imported (= data corruption)? > For example what if the pool hasn't been exported on the MASTER before > it dies? > - Is it a problem if the NFS daemons are started at boot on the standby > node, or should they only be started in the failover script? What > about stale files and active connections on the clients? sometimes stale mounts recover, sometimes not, sometimes clients need even reboots > - A catastrophic power failure occur and MASTER and BACKUP are suddently > powered down. Later the power returns, is it possible that some > problem occur (split-brain scenario ?) regarding the order in which the sure, you need an exact procedure to recover > two machines boot up? best practice should be to keep everything down after boot > - Other things I have not thought? > > Thanks! > Julien > imho: leave hast where it is, go for zfs replication. will save your butt, sooner or later if you avoid this fragile combination From owner-freebsd-fs@freebsd.org Thu Jun 30 15:28:49 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43300B87EEB for ; Thu, 30 Jun 2016 15:28:49 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D55862617 for ; Thu, 30 Jun 2016 15:28:48 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id v199so226259479wmv.0 for ; Thu, 30 Jun 2016 08:28:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=7JV/OP20p6JPSBt+ShievEbEM6+MLKnrZ/9aJ85vFZE=; b=puQVhO9nlCO0BJFSlm/bcmWr+bxN4e2ai8JgLTO9K+x6TvwDS0QCUNa9LAMrsBwY88 cdctBY52Rg8aOP30hLs+30bGkgZI2lJQXHFzwHvu3F8mmo9PsAMR5tcoBr1M1d44287f AAhI0fzHO4mTx5T26FenlfYXk5+TWk67gpMVjMF/VGwJ+SOZskKejcovkimsXXAY2UVg eAGBOZtSjpXKlStcg5qZ+k3prQw1MZzU3qX/QDMMgUU5edlUCYhMqqxwK3dYSsITDS3q bcOzXdynWSxMWhsWl0KK0qGklM+lNIJSJJmNdkodC5cFyaPpjxoewcfHfnLi3pjmwQZF 2G7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=7JV/OP20p6JPSBt+ShievEbEM6+MLKnrZ/9aJ85vFZE=; b=MP6Dbuyz4vg6+MkxzxeW862rpXz67QrJ4dagldkL8kq+jjG62/KCDwdoF8z4SbvlvN LZH8XR6V40CLfMl19ofd46n492Z0AnbcytrXFFouX+g0yV45VaxEqEuizuBQOus/llr3 +OBJEPtBmEtwMEQk6zciIXjJNwtjqvZYpyh4C299RGxcELaIzMqt/nTr2ZGf1TQxBwBN Acv0gVn1LlCu/NewdyHbZeLR82gF7ecPHh+B/OLI22Y6Wj+l4v8KN39Q8w0G94Xx5CEk 8q93OwvyOOFyOw7y62Z2yU6JzlfXoctUBU/AiZZvoS894Hr/70NdejXf+kOfUcoPMycl V8MA== X-Gm-Message-State: ALyK8tLtbRp02dL/P+Ei2SiOJRQmQv9PDSv96ialMk44VECulTQ2R6DG83ne4oMGc/Z5qw== X-Received: by 10.28.169.66 with SMTP id s63mr28689054wme.87.1467300525399; Thu, 30 Jun 2016 08:28:45 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id m125sm9950726wmm.8.2016.06.30.08.28.44 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 30 Jun 2016 08:28:44 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> Date: Thu, 30 Jun 2016 17:28:41 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:28:49 -0000 > On 30 Jun 2016, at 17:14, InterNetX - Juergen Gotteswinter = wrote: >=20 >=20 >=20 > Am 30.06.2016 um 16:45 schrieb Julien Cigar: >> Hello, >>=20 >> I'm always in the process of setting a redundant low-cost storage for=20= >> our (small, ~30 people) team here. >>=20 >> I read quite a lot of articles/documentations/etc and I plan to use = HAST >> with ZFS for the storage, CARP for the failover and the "good old = NFS" >> to mount the shares on the clients. >>=20 >> The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for = the >> shared storage. >>=20 >> Assuming the following configuration: >> - MASTER is the active node and BACKUP is the standby node. >> - two disks in each machine: ada0 and ada1. >> - two interfaces in each machine: em0 and em1 >> - em0 is the primary interface (with CARP setup) >> - em1 is dedicated to the HAST traffic (crossover cable) >> - FreeBSD is properly installed in each machine. >> - a HAST resource "disk0" for ada0p2. >> - a HAST resource "disk1" for ada1p2. >> - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is = created >> on MASTER >>=20 >> A couple of questions I am still wondering: >> - If a disk dies on the MASTER I guess that zpool will not see it and >> will transparently use the one on BACKUP through the HAST = ressource.. >=20 > thats right, as long as writes on $anything have been successful hast = is > happy and wont start whining >=20 >> is it a problem?=20 >=20 > imho yes, at least from management view >=20 >> could this lead to some corruption? >=20 > probably, i never heard about anyone who uses that for long time in > production >=20 > At this stage the >> common sense would be to replace the disk quickly, but imagine the >> worst case scenario where ada1 on MASTER dies, zpool will not see it=20= >> and will transparently use the one from the BACKUP node (through the=20= >> "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not=20= >> see it and will transparently use the one from the BACKUP node=20 >> (through the "disk0" HAST ressource). At this point on MASTER the = two=20 >> disks are broken but the pool is still considered healthy ... What = if=20 >> after that we unplug the em0 network cable on BACKUP? Storage is >> down.. >> - Under heavy I/O the MASTER box suddently dies (for some reasons),=20= >> thanks to CARP the BACKUP node will switch from standy -> active and=20= >> execute the failover script which does some "hastctl role primary" = for >> the ressources and a zpool import. I wondered if there are any >> situations where the pool couldn't be imported (=3D data = corruption)? >> For example what if the pool hasn't been exported on the MASTER = before >> it dies? >> - Is it a problem if the NFS daemons are started at boot on the = standby >> node, or should they only be started in the failover script? What >> about stale files and active connections on the clients? >=20 > sometimes stale mounts recover, sometimes not, sometimes clients need > even reboots >=20 >> - A catastrophic power failure occur and MASTER and BACKUP are = suddently >> powered down. Later the power returns, is it possible that some >> problem occur (split-brain scenario ?) regarding the order in which = the >=20 > sure, you need an exact procedure to recover >=20 >> two machines boot up? >=20 > best practice should be to keep everything down after boot >=20 >> - Other things I have not thought? >>=20 >=20 >=20 >=20 >> Thanks! >> Julien >>=20 >=20 >=20 > imho: >=20 > leave hast where it is, go for zfs replication. will save your butt, > sooner or later if you avoid this fragile combination I was also replying, and finishing by this : Why don't you set your slave as an iSCSI target and simply do ZFS = mirroring ? ZFS would then know as soon as a disk is failing. And if the master fails, you only have to import (-f certainly, in case = of a master power failure) on the slave. Ben From owner-freebsd-fs@freebsd.org Thu Jun 30 15:30:32 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D99EB87F3F for ; Thu, 30 Jun 2016 15:30:32 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b03.edpnet.be (relay-b03.edpnet.be [212.71.1.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 08F6D26AF for ; Thu, 30 Jun 2016 15:30:31 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467300626-0a88181ce45a77a0001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b03.edpnet.be with ESMTP id BGNjjoetWIMzCRud (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 30 Jun 2016 17:30:28 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Thu, 30 Jun 2016 17:30:26 +0200 From: Julien Cigar To: InterNetX - Juergen Gotteswinter Cc: freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160630153026.GA5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467300627 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.220:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 4065 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30898 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:30:32 -0000 --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 30, 2016 at 05:14:08PM +0200, InterNetX - Juergen Gotteswinter = wrote: >=20 >=20 > Am 30.06.2016 um 16:45 schrieb Julien Cigar: > > Hello, > >=20 > > I'm always in the process of setting a redundant low-cost storage for= =20 > > our (small, ~30 people) team here. > >=20 > > I read quite a lot of articles/documentations/etc and I plan to use HAST > > with ZFS for the storage, CARP for the failover and the "good old NFS" > > to mount the shares on the clients. > >=20 > > The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for the > > shared storage. > >=20 > > Assuming the following configuration: > > - MASTER is the active node and BACKUP is the standby node. > > - two disks in each machine: ada0 and ada1. > > - two interfaces in each machine: em0 and em1 > > - em0 is the primary interface (with CARP setup) > > - em1 is dedicated to the HAST traffic (crossover cable) > > - FreeBSD is properly installed in each machine. > > - a HAST resource "disk0" for ada0p2. > > - a HAST resource "disk1" for ada1p2. > > - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is created > > on MASTER > >=20 > > A couple of questions I am still wondering: > > - If a disk dies on the MASTER I guess that zpool will not see it and > > will transparently use the one on BACKUP through the HAST ressource.. >=20 > thats right, as long as writes on $anything have been successful hast is > happy and wont start whining >=20 > > is it a problem?=20 >=20 > imho yes, at least from management view >=20 > > could this lead to some corruption? >=20 > probably, i never heard about anyone who uses that for long time in > production >=20 > At this stage the > > common sense would be to replace the disk quickly, but imagine the > > worst case scenario where ada1 on MASTER dies, zpool will not see it= =20 > > and will transparently use the one from the BACKUP node (through the= =20 > > "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not=20 > > see it and will transparently use the one from the BACKUP node=20 > > (through the "disk0" HAST ressource). At this point on MASTER the two= =20 > > disks are broken but the pool is still considered healthy ... What if= =20 > > after that we unplug the em0 network cable on BACKUP? Storage is > > down.. > > - Under heavy I/O the MASTER box suddently dies (for some reasons),=20 > > thanks to CARP the BACKUP node will switch from standy -> active and= =20 > > execute the failover script which does some "hastctl role primary" for > > the ressources and a zpool import. I wondered if there are any > > situations where the pool couldn't be imported (=3D data corruption)? > > For example what if the pool hasn't been exported on the MASTER before > > it dies? > > - Is it a problem if the NFS daemons are started at boot on the standby > > node, or should they only be started in the failover script? What > > about stale files and active connections on the clients? >=20 > sometimes stale mounts recover, sometimes not, sometimes clients need > even reboots >=20 > > - A catastrophic power failure occur and MASTER and BACKUP are suddently > > powered down. Later the power returns, is it possible that some > > problem occur (split-brain scenario ?) regarding the order in which t= he >=20 > sure, you need an exact procedure to recover >=20 > > two machines boot up? >=20 > best practice should be to keep everything down after boot >=20 > > - Other things I have not thought? > >=20 >=20 >=20 >=20 > > Thanks! > > Julien > >=20 >=20 >=20 > imho: >=20 > leave hast where it is, go for zfs replication. will save your butt, > sooner or later if you avoid this fragile combination Do you mean a $> zfs snapshot followed by a $> zfs send ... | ssh zfs receive ... ? --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --wRRV7LY7NUeQGEoC Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdTsPAAoJELK7NxCiBCPAKugQAMBExrLenCJ2tMFbkeT8ii/r DApZUEnkbeUeBFxvlbt0BPWIyTYWI7aQoAOeiSV4sdjJmxUqANKSFKnoU909dc29 RrH0j36ijjbeogoBq+QmScM2+odvw13gdJxmkxRqBT/FSKRaKiSUZe51VdibsE43 Dm4YknXLb0Y8V6b0vZ6DdQ1iaWZwa/rakalDK1Y4bSoGhQGZPJocPRxlDIuMBway AZQIIb6HaUueRGDVKAOsJTvVrgV36vNEeHyfeSKakxOm/Qm55qRFwbqfWastFZTd pzLY6ExLDiZ3TM32bphPtuvcj6EFKD1CyjRJr6+wlR0j19SfCoAVaAwBp7wh95B5 u3Kub34z0HzfWGe+qcoKXKe0eYxUIjn6pE4BziRIO3ggiXuD2FZuHiv5n86sB1/G qOIb90Mc/wGvgiSCnTNuXg0xUb9RI3x/BBnwM3cONuBXiu26Thuz3NbHx0S/lI5n G1CfyOhBcZPBHPnfl/BpWLw9+DdCVQ8SU/Rz0rGD0rmHjpeMRtbgXV+hCglpdxC9 bS33+FTqTaLm+L2emMTa/iaM7ZTJwwR6IOPVaHoKKvZ3eJFyfmeWWI+ShZhEx24g x2G08K4m7cdjXtMlWIXesGc7OzCY/7T1je6hUNR4zWyGYN096i8+r7jsX3GcWQ8D b9+pMsCa3+vVZN2YpwLy =2Yy9 -----END PGP SIGNATURE----- --wRRV7LY7NUeQGEoC-- From owner-freebsd-fs@freebsd.org Thu Jun 30 15:34:06 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5C6BB861A2 for ; Thu, 30 Jun 2016 15:34:06 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6CBEC2B65 for ; Thu, 30 Jun 2016 15:34:06 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 5F43445FC0E4; Thu, 30 Jun 2016 17:34:04 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gg5MVAkTjLyo; Thu, 30 Jun 2016 17:33:59 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 3A6BB45FC0E8; Thu, 30 Jun 2016 17:33:59 +0200 (CEST) Reply-To: jg@internetx.com Subject: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153026.GA5695@mordor.lan> To: Julien Cigar Cc: freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Message-ID: <5f003083-e627-82c4-81fb-daa9c3cd71bf@internetx.com> Date: Thu, 30 Jun 2016 17:33:56 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160630153026.GA5695@mordor.lan> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:34:06 -0000 >> imho: >> >> leave hast where it is, go for zfs replication. will save your butt, >> sooner or later if you avoid this fragile combination > > Do you mean a $> zfs snapshot followed by a $> zfs send ... | ssh zfs > receive ... ? > exactly, super simple, super robust, just works without hassle From owner-freebsd-fs@freebsd.org Thu Jun 30 15:35:07 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F94EB86216 for ; Thu, 30 Jun 2016 15:35:07 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id C9E142C78 for ; Thu, 30 Jun 2016 15:35:06 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 071D345FC0EC; Thu, 30 Jun 2016 17:35:05 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pyF4QavgTRpN; Thu, 30 Jun 2016 17:35:03 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id F0F6345FC0E8; Thu, 30 Jun 2016 17:35:02 +0200 (CEST) Reply-To: jg@internetx.com Subject: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153026.GA5695@mordor.lan> <5f003083-e627-82c4-81fb-daa9c3cd71bf@internetx.com> To: Julien Cigar Cc: freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Organization: InterNetX GmbH Message-ID: <7743eb1b-b07e-d5f2-bf28-8004fb06f3c6@internetx.com> Date: Thu, 30 Jun 2016 17:35:00 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <5f003083-e627-82c4-81fb-daa9c3cd71bf@internetx.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:35:07 -0000 Am 30.06.2016 um 17:33 schrieb InterNetX - Juergen Gotteswinter: > >>> imho: >>> >>> leave hast where it is, go for zfs replication. will save your butt, >>> sooner or later if you avoid this fragile combination >> >> Do you mean a $> zfs snapshot followed by a $> zfs send ... | ssh zfs >> receive ... ? >> > > exactly, super simple, super robust, just works without hassle > check zrep, its a solid script for repliocation and failover (keeps the replication partner read only, and manages the switch to rw From owner-freebsd-fs@freebsd.org Thu Jun 30 15:35:51 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BA9CB862B5 for ; Thu, 30 Jun 2016 15:35:51 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 301072D1F for ; Thu, 30 Jun 2016 15:35:50 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id u5UFVZnD080873 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL); Thu, 30 Jun 2016 16:31:36 +0100 (BST) (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1467300698; bh=HE4PMWLW0UcCe60Zm2RRC/739kYLCGVHzO7zK6C0Hf0=; h=From:To:Subject:Date:References:In-Reply-To; b=D7RHHlZQ9fEpNMtbPrlTqVapr9agsSxg7pQVgPr5jduV4p2y/I9/x0BKpcvBeaGDr Dph7AKNz/bCDZ/F1oIskTN7zHi+lHAU9tauSZGFcKZkiEGYsV5vR5hae4G0nMUA4pL MA3WEaktMGh5XMCDuSIIwMmH/bn8JGHHCjHPaNfQ= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Thu, 30 Jun 2016 16:31:30 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Thu, 30 Jun 2016 16:31:30 +0100 From: Matt Churchyard To: "jg@internetx.com" , Julien Cigar , "freebsd-fs@freebsd.org" Subject: RE: HAST + ZFS + NFS + CARP Thread-Topic: HAST + ZFS + NFS + CARP Thread-Index: AQHR0uC4V3fSVogBf0aaC9ypoSFXlaACDZMAgAAUKPA= Date: Thu, 30 Jun 2016 15:31:29 +0000 Message-ID: <9b946ad7e3484099a0067b74ab75faa4@SERVER.ad.usd-group.com> References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> In-Reply-To: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:35:51 -0000 -----Original Message----- From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] On= Behalf Of InterNetX - Juergen Gotteswinter Sent: 30 June 2016 16:14 To: Julien Cigar; freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Am 30.06.2016 um 16:45 schrieb Julien Cigar: > Hello, >=20 > I'm always in the process of setting a redundant low-cost storage for=20 > our (small, ~30 people) team here. >=20 > I read quite a lot of articles/documentations/etc and I plan to use=20 > HAST with ZFS for the storage, CARP for the failover and the "good old NF= S" > to mount the shares on the clients. >=20 > The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for=20 > the shared storage. >=20 > Assuming the following configuration: > - MASTER is the active node and BACKUP is the standby node. > - two disks in each machine: ada0 and ada1. > - two interfaces in each machine: em0 and em1 > - em0 is the primary interface (with CARP setup) > - em1 is dedicated to the HAST traffic (crossover cable) > - FreeBSD is properly installed in each machine. > - a HAST resource "disk0" for ada0p2. > - a HAST resource "disk1" for ada1p2. > - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is created > on MASTER >=20 > A couple of questions I am still wondering: > - If a disk dies on the MASTER I guess that zpool will not see it and > will transparently use the one on BACKUP through the HAST ressource.. thats right, as long as writes on $anything have been successful hast is ha= ppy and wont start whining > is it a problem?=20 imho yes, at least from management view > could this lead to some corruption? probably, i never heard about anyone who uses that for long time in product= ion At this stage the > common sense would be to replace the disk quickly, but imagine the > worst case scenario where ada1 on MASTER dies, zpool will not see it=20 > and will transparently use the one from the BACKUP node (through the=20 > "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not=20 > see it and will transparently use the one from the BACKUP node=20 > (through the "disk0" HAST ressource). At this point on MASTER the two=20 > disks are broken but the pool is still considered healthy ... What if=20 > after that we unplug the em0 network cable on BACKUP? Storage is > down.. > - Under heavy I/O the MASTER box suddently dies (for some reasons),=20 > thanks to CARP the BACKUP node will switch from standy -> active and=20 > execute the failover script which does some "hastctl role primary" for > the ressources and a zpool import. I wondered if there are any > situations where the pool couldn't be imported (=3D data corruption)? > For example what if the pool hasn't been exported on the MASTER before > it dies? > - Is it a problem if the NFS daemons are started at boot on the standby > node, or should they only be started in the failover script? What > about stale files and active connections on the clients? >sometimes stale mounts recover, sometimes not, sometimes clients need even= reboots > - A catastrophic power failure occur and MASTER and BACKUP are suddently > powered down. Later the power returns, is it possible that some > problem occur (split-brain scenario ?) regarding the order in which=20 > the >sure, you need an exact procedure to recover Happy to be correctly, but last time I looked at this, the NFS filesystem I= D was likely to be different on both systems (and cannot be set like on Lin= ux), and so the mounts would be useless on the clients after failover. You'= d need to remount the NFS filesystem on the clients. > two machines boot up? >best practice should be to keep everything down after boot > - Other things I have not thought? >=20 > Thanks! > Julien >=20 >imho: >leave hast where it is, go for zfs replication. will save your butt, soone= r or later if you avoid this fragile combination=20 Personally I agree. This sort of functionality is incredibly difficult to g= et right and I wouldn't want to run anything critical relying on a few HAST= scripts I'd put together manually. Matt From owner-freebsd-fs@freebsd.org Thu Jun 30 15:37:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24512B8636B for ; Thu, 30 Jun 2016 15:37:53 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b02.edpnet.be (relay-b02.edpnet.be [212.71.1.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CB7D12E46 for ; Thu, 30 Jun 2016 15:37:52 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467301067-0a7b8d120f3ca1c00001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b02.edpnet.be with ESMTP id GhfGgF4IWc5iYQSu (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 30 Jun 2016 17:37:49 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Thu, 30 Jun 2016 17:37:47 +0200 From: Julien Cigar To: Ben RUBSON Cc: freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160630153747.GB5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="mxv5cy4qt+RJ9ypb" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467301068 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.222:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 4782 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30898 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:37:53 -0000 --mxv5cy4qt+RJ9ypb Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 30, 2016 at 05:28:41PM +0200, Ben RUBSON wrote: >=20 > > On 30 Jun 2016, at 17:14, InterNetX - Juergen Gotteswinter wrote: > >=20 > >=20 > >=20 > > Am 30.06.2016 um 16:45 schrieb Julien Cigar: > >> Hello, > >>=20 > >> I'm always in the process of setting a redundant low-cost storage for= =20 > >> our (small, ~30 people) team here. > >>=20 > >> I read quite a lot of articles/documentations/etc and I plan to use HA= ST > >> with ZFS for the storage, CARP for the failover and the "good old NFS" > >> to mount the shares on the clients. > >>=20 > >> The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for the > >> shared storage. > >>=20 > >> Assuming the following configuration: > >> - MASTER is the active node and BACKUP is the standby node. > >> - two disks in each machine: ada0 and ada1. > >> - two interfaces in each machine: em0 and em1 > >> - em0 is the primary interface (with CARP setup) > >> - em1 is dedicated to the HAST traffic (crossover cable) > >> - FreeBSD is properly installed in each machine. > >> - a HAST resource "disk0" for ada0p2. > >> - a HAST resource "disk1" for ada1p2. > >> - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is creat= ed > >> on MASTER > >>=20 > >> A couple of questions I am still wondering: > >> - If a disk dies on the MASTER I guess that zpool will not see it and > >> will transparently use the one on BACKUP through the HAST ressource.. > >=20 > > thats right, as long as writes on $anything have been successful hast is > > happy and wont start whining > >=20 > >> is it a problem?=20 > >=20 > > imho yes, at least from management view > >=20 > >> could this lead to some corruption? > >=20 > > probably, i never heard about anyone who uses that for long time in > > production > >=20 > > At this stage the > >> common sense would be to replace the disk quickly, but imagine the > >> worst case scenario where ada1 on MASTER dies, zpool will not see it= =20 > >> and will transparently use the one from the BACKUP node (through the= =20 > >> "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not=20 > >> see it and will transparently use the one from the BACKUP node=20 > >> (through the "disk0" HAST ressource). At this point on MASTER the two= =20 > >> disks are broken but the pool is still considered healthy ... What if= =20 > >> after that we unplug the em0 network cable on BACKUP? Storage is > >> down.. > >> - Under heavy I/O the MASTER box suddently dies (for some reasons),=20 > >> thanks to CARP the BACKUP node will switch from standy -> active and= =20 > >> execute the failover script which does some "hastctl role primary" for > >> the ressources and a zpool import. I wondered if there are any > >> situations where the pool couldn't be imported (=3D data corruption)? > >> For example what if the pool hasn't been exported on the MASTER before > >> it dies? > >> - Is it a problem if the NFS daemons are started at boot on the standby > >> node, or should they only be started in the failover script? What > >> about stale files and active connections on the clients? > >=20 > > sometimes stale mounts recover, sometimes not, sometimes clients need > > even reboots > >=20 > >> - A catastrophic power failure occur and MASTER and BACKUP are suddent= ly > >> powered down. Later the power returns, is it possible that some > >> problem occur (split-brain scenario ?) regarding the order in which t= he > >=20 > > sure, you need an exact procedure to recover > >=20 > >> two machines boot up? > >=20 > > best practice should be to keep everything down after boot > >=20 > >> - Other things I have not thought? > >>=20 > >=20 > >=20 > >=20 > >> Thanks! > >> Julien > >>=20 > >=20 > >=20 > > imho: > >=20 > > leave hast where it is, go for zfs replication. will save your butt, > > sooner or later if you avoid this fragile combination >=20 > I was also replying, and finishing by this : > Why don't you set your slave as an iSCSI target and simply do ZFS mirrori= ng ? Yes that's another option, so a zpool with two mirrors (local +=20 exported iSCSI) ? > ZFS would then know as soon as a disk is failing. > And if the master fails, you only have to import (-f certainly, in case o= f a master power failure) on the slave. >=20 > Ben >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --mxv5cy4qt+RJ9ypb Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdTzLAAoJELK7NxCiBCPAU+EP/2sN1DkF36296xw1+mRA0dIz rqyGM7fBZOhHCSqZ7YzzCKvpb28Pg5qe+oMD/tJ2N0sp5O07nG84sYMGFqEAirE1 LZaDoiMItRYobsIFpNjrMagmX5l9kiMy1aFo//eTFfY/J6Bzka2m7kErOtuhOkiP gp7r02Rsr3dztjAa60Mpo0XYv2A9AOa2KkNJ0wQm2vdzXjzFIYomGvK7Eg0cDdMH cfZWVSEDl5LundgtX7b1NY8gG04eP7c2lgOUTBO++D5p4SjzwZ07px56tSX9pqP1 TMG38CtUmA0w0MDDdXE6KiFg0ZYmFPQSmmV1SD/O34iuP1wo4BkIiOPf7lIYoxYQ O/TZbD8VKbYAEKNChpTjFlKgGYFKPLK1IL3H6x0G4UtLC2SnUFlel4eH5LvZmlvz hhTpjX9Ybjdn3Y4YXbnCr02rhwacapVghnwJQ8kjPHp/azLtAI7eyHtMWdZv2m2I fxGw3npMM1CgF0xCF56CdM8t4XyqD5XMyU2RCYFNeTx/5t7MbjktCdK53QDlxHT9 3kuGYJG6MhLdjXuedl1QFyb91x3di7BjNDxt0qmrIMCsjO58J+0GOGmP5Te/sBlA vWYMkfwKcX8lNuqT3xKESrclBofyboGsFRQBK5tbnCkfQEr4w4AlpVM3SPH7LspV wYacJIrWBrAlEAunATQ8 =OY/+ -----END PGP SIGNATURE----- --mxv5cy4qt+RJ9ypb-- From owner-freebsd-fs@freebsd.org Thu Jun 30 15:42:13 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D328CB86615 for ; Thu, 30 Jun 2016 15:42:13 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 68666214B for ; Thu, 30 Jun 2016 15:42:13 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x231.google.com with SMTP id a66so124581142wme.0 for ; Thu, 30 Jun 2016 08:42:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-transfer-encoding:mime-version:subject:message-id:date :references:in-reply-to:to; bh=EI3XykD+h8ujZGRsw/tzjnPL+7Orw+s8umEpNec4vgE=; b=CBBnLykSiIUEWDUIxHTGVFcks1MDg3e6WUnwjaMfLiM8AFVVZy5jOmITNL/XsM9bhl cG+GZhPc2YYLkaiJHZ1K+1uzl9egPRhjf9SD90cbMTUtTc6fjy3a3leifToZ2oHOOXJe SzXiwEmCwTBr+kNd7aYaYrlgaq386XPkyCV4p/uX9UtFoRQJocmsN3ixms/RlXPxKlko S5FLpEIPV33FL4bs90ldZ8bwgYXrYFVxjvJnD3eMFnAJOSyQMZP07GW5TKnenpnC1bnN g6ijcm/chfGaFzowx/tTnW53QvAgbK2xbgqeqSwJNppVlkoQa02egGWeUiuSbfGIntUQ I1Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:references:in-reply-to:to; bh=EI3XykD+h8ujZGRsw/tzjnPL+7Orw+s8umEpNec4vgE=; b=LUg7cOhNuUlwhl21kiFXFu7Uq1KkdrYxNlmHKuSi5iC8zpZesXTVNKjJJD56qVY7n5 xo3C43wkS8c1kdF2Ghna4sX96snxgmYTUFwfz6qnWyOkQMSrfrKgKaID5702DPidrhU7 TCn5D4PYgZkWdvYLJGwdXPr0kQnNKV0yuvx26CBy/JzSlQ9JUC9FhxuGHUogWGoUTmgf C2dJYyGisVlAsLVN46oddA4bALEvYV5poXDGpNkKjL3GCwCIn+rthXkZotZT5lpMvfl8 kN/C4kkYdSubEXcbszg+/jJBG6NLuEVvQ70sMXtgj/PCcG0wrDG9mcDCGMp89FR+QEfW 3eIw== X-Gm-Message-State: ALyK8tK7p1dovE/s9lbkhuC+pXJA1s7VrGkUZ1e/fuv7UOobxffZu0AFoCbcTLzyeHfUJg== X-Received: by 10.194.110.234 with SMTP id id10mr14402863wjb.17.1467301331537; Thu, 30 Jun 2016 08:42:11 -0700 (PDT) Received: from [10.149.45.183] ([80.12.43.183]) by smtp.gmail.com with ESMTPSA id bh4sm3887451wjc.43.2016.06.30.08.42.10 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 30 Jun 2016 08:42:10 -0700 (PDT) From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Subject: Re: HAST + ZFS + NFS + CARP Message-Id: <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> Date: Thu, 30 Jun 2016 17:42:04 +0200 References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> In-Reply-To: <20160630153747.GB5695@mordor.lan> To: freebsd-fs@freebsd.org X-Mailer: iPhone Mail (13F69) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:42:13 -0000 > On 30 Jun 2016, at 17:37, Julien Cigar wrote: >=20 >> On Thu, Jun 30, 2016 at 05:28:41PM +0200, Ben RUBSON wrote: >>=20 >>> On 30 Jun 2016, at 17:14, InterNetX - Juergen Gotteswinter wrote: >>>=20 >>>=20 >>>=20 >>>> Am 30.06.2016 um 16:45 schrieb Julien Cigar: >>>> Hello, >>>>=20 >>>> I'm always in the process of setting a redundant low-cost storage for=20= >>>> our (small, ~30 people) team here. >>>>=20 >>>> I read quite a lot of articles/documentations/etc and I plan to use HAS= T >>>> with ZFS for the storage, CARP for the failover and the "good old NFS" >>>> to mount the shares on the clients. >>>>=20 >>>> The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for the= >>>> shared storage. >>>>=20 >>>> Assuming the following configuration: >>>> - MASTER is the active node and BACKUP is the standby node. >>>> - two disks in each machine: ada0 and ada1. >>>> - two interfaces in each machine: em0 and em1 >>>> - em0 is the primary interface (with CARP setup) >>>> - em1 is dedicated to the HAST traffic (crossover cable) >>>> - FreeBSD is properly installed in each machine. >>>> - a HAST resource "disk0" for ada0p2. >>>> - a HAST resource "disk1" for ada1p2. >>>> - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is create= d >>>> on MASTER >>>>=20 >>>> A couple of questions I am still wondering: >>>> - If a disk dies on the MASTER I guess that zpool will not see it and >>>> will transparently use the one on BACKUP through the HAST ressource.. >>>=20 >>> thats right, as long as writes on $anything have been successful hast is= >>> happy and wont start whining >>>=20 >>>> is it a problem?=20 >>>=20 >>> imho yes, at least from management view >>>=20 >>>> could this lead to some corruption? >>>=20 >>> probably, i never heard about anyone who uses that for long time in >>> production >>>=20 >>> At this stage the >>>> common sense would be to replace the disk quickly, but imagine the >>>> worst case scenario where ada1 on MASTER dies, zpool will not see it=20= >>>> and will transparently use the one from the BACKUP node (through the=20= >>>> "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not=20 >>>> see it and will transparently use the one from the BACKUP node=20 >>>> (through the "disk0" HAST ressource). At this point on MASTER the two=20= >>>> disks are broken but the pool is still considered healthy ... What if=20= >>>> after that we unplug the em0 network cable on BACKUP? Storage is >>>> down.. >>>> - Under heavy I/O the MASTER box suddently dies (for some reasons),=20 >>>> thanks to CARP the BACKUP node will switch from standy -> active and=20= >>>> execute the failover script which does some "hastctl role primary" for >>>> the ressources and a zpool import. I wondered if there are any >>>> situations where the pool couldn't be imported (=3D data corruption)? >>>> For example what if the pool hasn't been exported on the MASTER before >>>> it dies? >>>> - Is it a problem if the NFS daemons are started at boot on the standby= >>>> node, or should they only be started in the failover script? What >>>> about stale files and active connections on the clients? >>>=20 >>> sometimes stale mounts recover, sometimes not, sometimes clients need >>> even reboots >>>=20 >>>> - A catastrophic power failure occur and MASTER and BACKUP are suddentl= y >>>> powered down. Later the power returns, is it possible that some >>>> problem occur (split-brain scenario ?) regarding the order in which the= >>>=20 >>> sure, you need an exact procedure to recover >>>=20 >>>> two machines boot up? >>>=20 >>> best practice should be to keep everything down after boot >>>=20 >>>> - Other things I have not thought? >>>>=20 >>>=20 >>>=20 >>>=20 >>>> Thanks! >>>> Julien >>>>=20 >>>=20 >>>=20 >>> imho: >>>=20 >>> leave hast where it is, go for zfs replication. will save your butt, >>> sooner or later if you avoid this fragile combination >>=20 >> I was also replying, and finishing by this : >> Why don't you set your slave as an iSCSI target and simply do ZFS mirrori= ng ? >=20 > Yes that's another option, so a zpool with two mirrors (local +=20 > exported iSCSI) ? Yes, you would then have a real time replication solution (as HAST), compare= d to ZFS send/receive which is not. Depends on what you need :) >=20 >> ZFS would then know as soon as a disk is failing. >> And if the master fails, you only have to import (-f certainly, in case o= f a master power failure) on the slave. >>=20 >> Ben From owner-freebsd-fs@freebsd.org Thu Jun 30 16:32:20 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 668B5B87352 for ; Thu, 30 Jun 2016 16:32:20 +0000 (UTC) (envelope-from bsdunix44@gmail.com) Received: from mail-oi0-x22b.google.com (mail-oi0-x22b.google.com [IPv6:2607:f8b0:4003:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 27FB22865 for ; Thu, 30 Jun 2016 16:32:20 +0000 (UTC) (envelope-from bsdunix44@gmail.com) Received: by mail-oi0-x22b.google.com with SMTP id f189so72768670oig.3 for ; Thu, 30 Jun 2016 09:32:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-transfer-encoding :message-id:cc:from:subject:date:to; bh=gSSTDdS6MdvFNtOWpW1Q6rfyl8LWCEUiV2FPqo/MbAg=; b=Q8E634AYsJ6yMwS18IIPml1YVXwE8ybeaCzrMH2vChwGB8zVaDQQCy/n/I+pkoXA4O R7el0EfVREhc+wsuH/BGFkG13NC9g41z0kNZ8ou1HPNQihWzLHXrrkkEwxbbdflK8xlS htVyZBrZpeiBxv9PkXKYn5BsAL/1cOXlgM1+2WqR4dd2OTN6yatafSYgl3hmeRCmve2m sZZyrJKEM53PN1LHvd2QCzirs4xY7qNg+h8eA6YMle49Zs2GPNTtAl31rvDD9x1Qo2X/ 0MzCrmg3NKu9DtsZslJEXUhFxKiOvHmybkJ3igsOnMmKlL50x03dSQ5w5OBNxaSAXYXm NSjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=gSSTDdS6MdvFNtOWpW1Q6rfyl8LWCEUiV2FPqo/MbAg=; b=KhRXmaR2YpRXihegoqZUT+0pVNAU2kvrQDWW36mT8mnamiwjvYCZHjotjvuJVr2c8w e9kUyL0pPvNq8hL0x1Y9WLvkFAi+kaCEFNyJOqfmH0TnpjsHHuNpUhlntuK4qZ8fLTPK J3KGNmUoqoozzuWWWekZ76+STZ8eKCNhHFJiY9nAG9VV3UXuqM6mOFPktv8kYZGwOmUD EFX5OvGxaGTiseD5DSNwRSEzqqMKYM7+posNC6da3+q+/nowDqAhP2aHxC9VuDTZEINJ tbILf8BRULgZvDhk6aCZO4Hl56gKMixPye8A4us2yyZ1tpUdQd46ahVL7AdSxL51kfgF tITA== X-Gm-Message-State: ALyK8tL7TFeZVeD9J4h5mRbSLa/UhvZ+rzqptxagf9WNKZjD+ip8qSBwebSUnCkRzHXQdA== X-Received: by 10.202.195.200 with SMTP id t191mr9468911oif.28.1467304339445; Thu, 30 Jun 2016 09:32:19 -0700 (PDT) Received: from [192.168.0.10] (cpe-70-118-225-173.kc.res.rr.com. [70.118.225.173]) by smtp.gmail.com with ESMTPSA id y142sm3562361oie.4.2016.06.30.09.32.18 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 30 Jun 2016 09:32:18 -0700 (PDT) References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> Mime-Version: 1.0 (1.0) In-Reply-To: <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <678321AB-A9F7-4890-A8C7-E20DFDC69137@gmail.com> Cc: freebsd-fs@freebsd.org X-Mailer: iPhone Mail (13F69) From: Chris Watson Subject: Re: HAST + ZFS + NFS + CARP Date: Thu, 30 Jun 2016 11:32:17 -0500 To: Ben RUBSON X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 16:32:20 -0000 Sent from my iPhone 5 >=20 >>=20 >> Yes that's another option, so a zpool with two mirrors (local +=20 >> exported iSCSI) ? >=20 > Yes, you would then have a real time replication solution (as HAST), compa= red to ZFS send/receive which is not. > Depends on what you need :) >=20 >>=20 >>> ZFS would then know as soon as a disk is failing. So as an aside, but related, for those watching this from the peanut gallery= and for the benefit of the OP perhaps those that run with this setup might g= ive some best practices and tips here in this thread on making this a good r= eliable setup. I can see someone reading this thread and tossing two crappy E= thernet cards in a box and then complaining it doesn't work well.=20 Perhaps those doing this could listed recommended NICs so write performance o= f the slave over the network isn't slow as sin, or their relevant part of th= eir config scripts to get this running well, any sysctl's set to boost perfo= rmance, gotcha's you e seen using this setup, and if a mirror failed were yo= u happy with this. India during recovery? Anything that will help current an= d future folks stumbling across this thread I think would be a big help in g= etting this good idea used a little more widely.=20 Chris= From owner-freebsd-fs@freebsd.org Thu Jun 30 16:35:47 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CA14B873EA for ; Thu, 30 Jun 2016 16:35:47 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b03.edpnet.be (relay-b03.edpnet.be [212.71.1.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B25A32959 for ; Thu, 30 Jun 2016 16:35:46 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467304541-0a88181ce65b1c30001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b03.edpnet.be with ESMTP id BHJ5pFotM736xmfs (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 30 Jun 2016 18:35:43 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Thu, 30 Jun 2016 18:35:41 +0200 From: Julien Cigar To: Ben RUBSON Cc: freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160630163541.GC5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="+xNpyl7Qekk2NvDX" Content-Disposition: inline In-Reply-To: <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467304542 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.220:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 5518 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30900 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 16:35:47 -0000 --+xNpyl7Qekk2NvDX Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 30, 2016 at 05:42:04PM +0200, Ben RUBSON wrote: >=20 >=20 > > On 30 Jun 2016, at 17:37, Julien Cigar wrote: > >=20 > >> On Thu, Jun 30, 2016 at 05:28:41PM +0200, Ben RUBSON wrote: > >>=20 > >>> On 30 Jun 2016, at 17:14, InterNetX - Juergen Gotteswinter wrote: > >>>=20 > >>>=20 > >>>=20 > >>>> Am 30.06.2016 um 16:45 schrieb Julien Cigar: > >>>> Hello, > >>>>=20 > >>>> I'm always in the process of setting a redundant low-cost storage fo= r=20 > >>>> our (small, ~30 people) team here. > >>>>=20 > >>>> I read quite a lot of articles/documentations/etc and I plan to use = HAST > >>>> with ZFS for the storage, CARP for the failover and the "good old NF= S" > >>>> to mount the shares on the clients. > >>>>=20 > >>>> The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for = the > >>>> shared storage. > >>>>=20 > >>>> Assuming the following configuration: > >>>> - MASTER is the active node and BACKUP is the standby node. > >>>> - two disks in each machine: ada0 and ada1. > >>>> - two interfaces in each machine: em0 and em1 > >>>> - em0 is the primary interface (with CARP setup) > >>>> - em1 is dedicated to the HAST traffic (crossover cable) > >>>> - FreeBSD is properly installed in each machine. > >>>> - a HAST resource "disk0" for ada0p2. > >>>> - a HAST resource "disk1" for ada1p2. > >>>> - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is cre= ated > >>>> on MASTER > >>>>=20 > >>>> A couple of questions I am still wondering: > >>>> - If a disk dies on the MASTER I guess that zpool will not see it and > >>>> will transparently use the one on BACKUP through the HAST ressource.. > >>>=20 > >>> thats right, as long as writes on $anything have been successful hast= is > >>> happy and wont start whining > >>>=20 > >>>> is it a problem?=20 > >>>=20 > >>> imho yes, at least from management view > >>>=20 > >>>> could this lead to some corruption? > >>>=20 > >>> probably, i never heard about anyone who uses that for long time in > >>> production > >>>=20 > >>> At this stage the > >>>> common sense would be to replace the disk quickly, but imagine the > >>>> worst case scenario where ada1 on MASTER dies, zpool will not see it= =20 > >>>> and will transparently use the one from the BACKUP node (through the= =20 > >>>> "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not= =20 > >>>> see it and will transparently use the one from the BACKUP node=20 > >>>> (through the "disk0" HAST ressource). At this point on MASTER the tw= o=20 > >>>> disks are broken but the pool is still considered healthy ... What i= f=20 > >>>> after that we unplug the em0 network cable on BACKUP? Storage is > >>>> down.. > >>>> - Under heavy I/O the MASTER box suddently dies (for some reasons),= =20 > >>>> thanks to CARP the BACKUP node will switch from standy -> active and= =20 > >>>> execute the failover script which does some "hastctl role primary" f= or > >>>> the ressources and a zpool import. I wondered if there are any > >>>> situations where the pool couldn't be imported (=3D data corruption)? > >>>> For example what if the pool hasn't been exported on the MASTER befo= re > >>>> it dies? > >>>> - Is it a problem if the NFS daemons are started at boot on the stan= dby > >>>> node, or should they only be started in the failover script? What > >>>> about stale files and active connections on the clients? > >>>=20 > >>> sometimes stale mounts recover, sometimes not, sometimes clients need > >>> even reboots > >>>=20 > >>>> - A catastrophic power failure occur and MASTER and BACKUP are sudde= ntly > >>>> powered down. Later the power returns, is it possible that some > >>>> problem occur (split-brain scenario ?) regarding the order in which = the > >>>=20 > >>> sure, you need an exact procedure to recover > >>>=20 > >>>> two machines boot up? > >>>=20 > >>> best practice should be to keep everything down after boot > >>>=20 > >>>> - Other things I have not thought? > >>>>=20 > >>>=20 > >>>=20 > >>>=20 > >>>> Thanks! > >>>> Julien > >>>>=20 > >>>=20 > >>>=20 > >>> imho: > >>>=20 > >>> leave hast where it is, go for zfs replication. will save your butt, > >>> sooner or later if you avoid this fragile combination > >>=20 > >> I was also replying, and finishing by this : > >> Why don't you set your slave as an iSCSI target and simply do ZFS mirr= oring ? > >=20 > > Yes that's another option, so a zpool with two mirrors (local +=20 > > exported iSCSI) ? >=20 > Yes, you would then have a real time replication solution (as HAST), comp= ared to ZFS send/receive which is not. > Depends on what you need :) More a real time replication solution in fact ... :) Do you have any resource which resume all the pro(s) and con(s) of HAST vs iSCSI ? I have found a lot of article on ZFS + HAST but not that much with ZFS + iSCSI ..=20 >=20 > >=20 > >> ZFS would then know as soon as a disk is failing. > >> And if the master fails, you only have to import (-f certainly, in cas= e of a master power failure) on the slave. > >>=20 > >> Ben > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --+xNpyl7Qekk2NvDX Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdUpaAAoJELK7NxCiBCPABNYQAJ1w/LOIlZMx+d2z52OhKPpU 74wxuBvbwMa+/+P5BSplydYHj3zMjATTolcfS6hR/gjG0Y5C4XVObB6CxsduM3o9 Rmfzsx+Z+e6RhjK7+/DRj+aJvuSIpXkVoUHiCb/u2FO9RpYDdigjLNmW6J+a+kOW 26Wkkzx+bKFi2V7p6KqWQ1/JlXLMbU2xqw1NhF8XaeSTu1Ywcju6VJXKalwdzQug 5+r+CObnogruy3PPVao08/Hxpv7VMp/qQhUb1IlZKuWczEg2GlCPSLjFradDqy32 EvTzqyV0XkxE+DtCUYUAHdul8MUiJZVsCCpNrSuQsyY42637FyZGhiP2iePc6qk3 1GMy3c2JRiZF0f43EZmy4HRNujESwCbfPkXq6oIC4UozgLc7AMuRYA+b9rBlxe3W fT+fYS/lPun4i7gOZBGcY4FmSO01/7LzqMld51xjdwFq2Gn1PGJU8/LJbaHwF/T/ 4gAQJtPjaGZGVMWd6e0yeW5j0bQGCfLWYy7eiTrRe2XegDoFdz3NoH0GyRqNIIo9 Kkad7938fkwsPtKdk2rlNIYDVjHrj1U6V4KyWn8iPY8qeRl+75S0ITk/xc+8jC5K y5+438BdfYBOjjxvklR9a566JKZBTrXrtgo8FB1hns8qNnHwtz1MLSdwxA67Sigr fN+2SY2FtAWxfBZWU9qB =lUJE -----END PGP SIGNATURE----- --+xNpyl7Qekk2NvDX-- From owner-freebsd-fs@freebsd.org Thu Jun 30 18:57:06 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD278B87F21 for ; Thu, 30 Jun 2016 18:57:06 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b03.edpnet.be (relay-b03.edpnet.be [212.71.1.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 555D92626 for ; Thu, 30 Jun 2016 18:57:05 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467313022-0a88181ce65c6900001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b03.edpnet.be with ESMTP id etcvwDXBSNFndvoE (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 30 Jun 2016 20:57:03 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Thu, 30 Jun 2016 20:57:01 +0200 From: Julien Cigar To: Chris Watson Cc: Ben RUBSON , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160630185701.GD5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <678321AB-A9F7-4890-A8C7-E20DFDC69137@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="9UV9rz0O2dU/yYYn" Content-Disposition: inline In-Reply-To: <678321AB-A9F7-4890-A8C7-E20DFDC69137@gmail.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467313022 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.220:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2082 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30904 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 18:57:06 -0000 --9UV9rz0O2dU/yYYn Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 30, 2016 at 11:32:17AM -0500, Chris Watson wrote: >=20 >=20 > Sent from my iPhone 5 >=20 > >=20 > >>=20 > >> Yes that's another option, so a zpool with two mirrors (local +=20 > >> exported iSCSI) ? > >=20 > > Yes, you would then have a real time replication solution (as HAST), co= mpared to ZFS send/receive which is not. > > Depends on what you need :) > >=20 > >>=20 > >>> ZFS would then know as soon as a disk is failing. >=20 > So as an aside, but related, for those watching this from the peanut gall= ery and for the benefit of the OP perhaps those that run with this setup mi= ght give some best practices and tips here in this thread on making this a = good reliable setup. I can see someone reading this thread and tossing two = crappy Ethernet cards in a box and then complaining it doesn't work well.= =20 It would be more than welcome indeed..! I have the feeling that HAST isn't that much used (but maybe I am wrong) and it's difficult to find=20 informations on it's reliability and concrete long-term use cases... Also the pros vs cons of HAST vs iSCSI >=20 > Perhaps those doing this could listed recommended NICs so write performan= ce of the slave over the network isn't slow as sin, or their relevant part = of their config scripts to get this running well, any sysctl's set to boost= performance, gotcha's you e seen using this setup, and if a mirror failed = were you happy with this. India during recovery? Anything that will help cu= rrent and future folks stumbling across this thread I think would be a big = help in getting this good idea used a little more widely.=20 >=20 > Chris > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --9UV9rz0O2dU/yYYn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdWt6AAoJELK7NxCiBCPA4mgQAKWuRRHmT1irM05jCF8sf/0J OlVfiF1cRnMHQSHXyehD56sSAYx0cszsFAWNANuVXoDX5YHdz5VRRWAZl5mZkB2N hxMbW5SiHUON9u7oIJq8/dfQ5SI2JF6Hk8nZ72K/NdFqKB02RHyyyMo+0306WtEe PPDKiU+T+VRpbDb7H5v/swclxbp4J8tw6qDxyofM2FUxg4hsW2yKIB9NfHqcVM+j iwruGBetMTdzFrxyTXZX2NmnYfW1Q7GhxFmq49/jKQoG810P2pN35F1jaPsO7Nlm Pnr8arLpIscRoDHumkMSXkxDMkueuKBjvlIvAhEy2mRkC9S9Dpr3Fqdbt5Ea9aiG FIIUcECKeW0iZsMOIIfFHUEbp+JJDHehycuezvyz1HuOe01qK1n6mxB0QxLq4kKB ro23h6uVAE1hrz29bpud9u7C1k5WRhghjAQHQHo+xXEU/vsnP4UAYtUHkmKstsXH 0uspiPHI/EqiJXZfu2a16YNxxdhZr1a8nyTFbczR0PqHSrlimhVmMn3JuhPPlhLG osQ54E/2nDXxiPiB6AdG97A9SLUr8eA/BZgSkbgx0vSnVKqeT1NIafP90DXkBEFT ZdrwgljXahB+XXiuhlW9pX76NNk3vnohptr2sjdp88Ce3A1tsw0YdeP4OyO6CKS6 fSEUSh3cuQBLzP14rjkn =P4A4 -----END PGP SIGNATURE----- --9UV9rz0O2dU/yYYn-- From owner-freebsd-fs@freebsd.org Thu Jun 30 21:35:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF966B87241 for ; Thu, 30 Jun 2016 21:35:54 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 563DD20E5 for ; Thu, 30 Jun 2016 21:35:54 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id a66so3845105wme.0 for ; Thu, 30 Jun 2016 14:35:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=psNyJwT1DbTKzCpgAUJmb/Q/KogSgg99tG9MgWIEGzg=; b=XhTan6asp7sfSX69u5/8vqVejMUXsl2geOsVaQwFcBkSGoV8pT0wslDCBHkVbW8R/i lwr8yzpSxeIOUjqycE6FUqj5iPKVx1VYbeZ2EJW+oBkkhtuCMxnQJDjvpqboux+gnG4N 06inja6o+epkTAAx+MK0g7V8qa23VRjXeRMiTKPqJnR2BznJKhg0wdaNSMRByATfUG9A S2hW/0WuCfDoXwiRoMyOPLTIwsQGvtERB/luTyJmzDpa5u0S1JVQA8AehRsP89A0uB1D 6LSxxDklWI84AQOLoTF4+xY5/Hefd1PQwZajB1NmrfZlE8PB84lOmIwOOklSoOBXErEL XvPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=psNyJwT1DbTKzCpgAUJmb/Q/KogSgg99tG9MgWIEGzg=; b=JE1/TU7NaREA4A0EPaUyE0Dw2LzkRHtQKE1hw/4ZZwFSAeDn8pL7UtnsWDHGE1ITFk +FaEIMO8DNeVDsFlP4wfLZaOMT3pqVUo0pmy7s8tZTevHH7IeN+8ZOSFLflZI3FDElRo x/uG65CWFALANt4XRMb4LX4F85QIlbM09wDs8l/YnDCw5TrQYVpGn9iq5cIaZE0WX2R9 1It3U6tyY2ZaEcsUNFdc6AiYeiK5XSROoRT1nb/1uezR3XjSVwE0rl4x/I2AXc8nkxl1 1XgQBOLXwembNT2o6d6PjsX4N9DWtwEDl59MhivrjUCvcMWXbmelfCz+wW8ds2bCI8XO dP2A== X-Gm-Message-State: ALyK8tJjU0sIAo2iWqC9Xxof5bDyzRdB+zkSu/pyE2O2vpXpznKhDjrKp+18x6mm6SasxQ== X-Received: by 10.28.229.147 with SMTP id c141mr30450520wmh.5.1467322551701; Thu, 30 Jun 2016 14:35:51 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id k6sm5262286wjz.28.2016.06.30.14.35.50 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 30 Jun 2016 14:35:50 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: <20160630163541.GC5695@mordor.lan> Date: Thu, 30 Jun 2016 23:35:49 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 21:35:54 -0000 > On 30 Jun 2016, at 18:35, Julien Cigar wrote: >=20 > On Thu, Jun 30, 2016 at 05:42:04PM +0200, Ben RUBSON wrote: >>=20 >>=20 >>> On 30 Jun 2016, at 17:37, Julien Cigar = wrote: >>>=20 >>>> On Thu, Jun 30, 2016 at 05:28:41PM +0200, Ben RUBSON wrote: >>>>=20 >>>>> On 30 Jun 2016, at 17:14, InterNetX - Juergen Gotteswinter = wrote: >>>>>=20 >>>>>=20 >>>>>=20 >>>>>> Am 30.06.2016 um 16:45 schrieb Julien Cigar: >>>>>> Hello, >>>>>>=20 >>>>>> I'm always in the process of setting a redundant low-cost storage = for=20 >>>>>> our (small, ~30 people) team here. >>>>>>=20 >>>>>> I read quite a lot of articles/documentations/etc and I plan to = use HAST >>>>>> with ZFS for the storage, CARP for the failover and the "good old = NFS" >>>>>> to mount the shares on the clients. >>>>>>=20 >>>>>> The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks = for the >>>>>> shared storage. >>>>>>=20 >>>>>> Assuming the following configuration: >>>>>> - MASTER is the active node and BACKUP is the standby node. >>>>>> - two disks in each machine: ada0 and ada1. >>>>>> - two interfaces in each machine: em0 and em1 >>>>>> - em0 is the primary interface (with CARP setup) >>>>>> - em1 is dedicated to the HAST traffic (crossover cable) >>>>>> - FreeBSD is properly installed in each machine. >>>>>> - a HAST resource "disk0" for ada0p2. >>>>>> - a HAST resource "disk1" for ada1p2. >>>>>> - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is = created >>>>>> on MASTER >>>>>>=20 >>>>>> A couple of questions I am still wondering: >>>>>> - If a disk dies on the MASTER I guess that zpool will not see it = and >>>>>> will transparently use the one on BACKUP through the HAST = ressource.. >>>>>=20 >>>>> thats right, as long as writes on $anything have been successful = hast is >>>>> happy and wont start whining >>>>>=20 >>>>>> is it a problem?=20 >>>>>=20 >>>>> imho yes, at least from management view >>>>>=20 >>>>>> could this lead to some corruption? >>>>>=20 >>>>> probably, i never heard about anyone who uses that for long time = in >>>>> production >>>>>=20 >>>>> At this stage the >>>>>> common sense would be to replace the disk quickly, but imagine = the >>>>>> worst case scenario where ada1 on MASTER dies, zpool will not see = it=20 >>>>>> and will transparently use the one from the BACKUP node (through = the=20 >>>>>> "disk1" HAST ressource), later ada0 on MASTER dies, zpool will = not=20 >>>>>> see it and will transparently use the one from the BACKUP node=20 >>>>>> (through the "disk0" HAST ressource). At this point on MASTER the = two=20 >>>>>> disks are broken but the pool is still considered healthy ... = What if=20 >>>>>> after that we unplug the em0 network cable on BACKUP? Storage is >>>>>> down.. >>>>>> - Under heavy I/O the MASTER box suddently dies (for some = reasons),=20 >>>>>> thanks to CARP the BACKUP node will switch from standy -> active = and=20 >>>>>> execute the failover script which does some "hastctl role = primary" for >>>>>> the ressources and a zpool import. I wondered if there are any >>>>>> situations where the pool couldn't be imported (=3D data = corruption)? >>>>>> For example what if the pool hasn't been exported on the MASTER = before >>>>>> it dies? >>>>>> - Is it a problem if the NFS daemons are started at boot on the = standby >>>>>> node, or should they only be started in the failover script? What >>>>>> about stale files and active connections on the clients? >>>>>=20 >>>>> sometimes stale mounts recover, sometimes not, sometimes clients = need >>>>> even reboots >>>>>=20 >>>>>> - A catastrophic power failure occur and MASTER and BACKUP are = suddently >>>>>> powered down. Later the power returns, is it possible that some >>>>>> problem occur (split-brain scenario ?) regarding the order in = which the >>>>>=20 >>>>> sure, you need an exact procedure to recover >>>>>=20 >>>>>> two machines boot up? >>>>>=20 >>>>> best practice should be to keep everything down after boot >>>>>=20 >>>>>> - Other things I have not thought? >>>>>>=20 >>>>>=20 >>>>>=20 >>>>>=20 >>>>>> Thanks! >>>>>> Julien >>>>>>=20 >>>>>=20 >>>>>=20 >>>>> imho: >>>>>=20 >>>>> leave hast where it is, go for zfs replication. will save your = butt, >>>>> sooner or later if you avoid this fragile combination >>>>=20 >>>> I was also replying, and finishing by this : >>>> Why don't you set your slave as an iSCSI target and simply do ZFS = mirroring ? >>>=20 >>> Yes that's another option, so a zpool with two mirrors (local +=20 >>> exported iSCSI) ? >>=20 >> Yes, you would then have a real time replication solution (as HAST), = compared to ZFS send/receive which is not. >> Depends on what you need :) >=20 > More a real time replication solution in fact ... :) > Do you have any resource which resume all the pro(s) and con(s) of = HAST > vs iSCSI ? I have found a lot of article on ZFS + HAST but not that = much > with ZFS + iSCSI ..=20 # No resources, but some ideas : - ZFS likes to see all the details of its underlying disks, which is = possible with local disks (of course) and iSCSI disks, not with HAST. - iSCSI solution is simpler, you only have ZFS to manage, your = replication is made by ZFS itself, not by an additional stack. - HAST does not seem to be really maintained (I may be wrong), at least = compared to DRBD HAST seems to be inspired from. - You do not have to cross your fingers when you promote your slave to = master ("will ZFS be happy with my HAST replicated disks ?"), ZFS = mirrored data by itself, you only have to import [-f]. - (auto)reconnection of iSCSI could not be as simple as with HAST, iSCSI = could require more administration after a disconnection. But this could = easily be done by a script. # Some "advices" based on my findings (I'm finishing my tests of such a = solution) : Write performance will suffer from network latency, but while your 2 = nodes are in the same room, that should be OK. If you are over a long distance link, you may add several ms to each = write IO, which, depending on the use case, may be wrong, ZFS may also = be unresponsive. Max throughput is also more difficult to achieve over a high latency = link. You will have to choose network cards depending on the number of disks = and their throughput. For example, if you need to resilver a SATA disk (180MB/s), then a = simple 1GB interface (120MB/s) will be a serious bottleneck. Think about scrub too. You should have to perform some network tuning (TCP window size, jumbo = frame...) to reach your max bandwidth. Trying to saturate network link with (for example) iPerf before dealing = with iSCSI seems to be a good thing. Here are some interesting sysctl so that ZFS will not hang (too long) in = case of an unreachable iSCSI disk : kern.iscsi.ping_timeout=3D5 kern.iscsi.iscsid_timeout=3D5 kern.iscsi.login_timeout=3D5 kern.iscsi.fail_on_disconnection=3D1 (adjust the 5 seconds depending on your needs / on your network = quality). Take care when you (auto)replace disks, you may replace an iSCSI disk = with a local disk, which of course would work but would be wrong in = terms of master/slave redundancy. Use nice labels on your disks so that if you have a lot of disks in your = pool, you quickly know which one is local, which one is remote. # send/receive pro(s) : In terms of data safety, one of the interests of ZFS send/receive is = that you have a totally different target pool, which can be interesting = if ever you have a disaster with your primary pool. As a 3rd node solution ? On another site ? (as send/receive does not = suffer as iSCSI would from latency) >>>> ZFS would then know as soon as a disk is failing. >>>> And if the master fails, you only have to import (-f certainly, in = case of a master power failure) on the slave. >>>>=20 >>>> Ben From owner-freebsd-fs@freebsd.org Thu Jun 30 23:56:43 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4139B87247 for ; Thu, 30 Jun 2016 23:56:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C3C622D14 for ; Thu, 30 Jun 2016 23:56:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u5UNuhOw086865 for ; Thu, 30 Jun 2016 23:56:43 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203864] ZFS deadlock between zfs send, zfs rename and ctrl-C Date: Thu, 30 Jun 2016 23:56:43 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: dogfood, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 23:56:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203864 Cassiano Peixoto changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |peixoto.cassiano@gmail.com --- Comment #8 from Cassiano Peixoto --- (In reply to roel from comment #6) Hi guys, Any news about this issue? I've same issue when running zrep script as roel said. Roel, did you find any workaround for this? Thanks. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jul 1 08:59:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F680B87EEC for ; Fri, 1 Jul 2016 08:59:30 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b01.edpnet.be (relay-b01.edpnet.be [212.71.1.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA0D62D06 for ; Fri, 1 Jul 2016 08:59:29 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467362837-0a7ff569f8fff360001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b01.edpnet.be with ESMTP id KhufTBeGlSOj6BdQ (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 10:47:18 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 10:47:17 +0200 From: Julien Cigar To: Ben RUBSON Cc: freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701084717.GE5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="M/SuVGWktc5uNpra" Content-Disposition: inline In-Reply-To: <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467362837 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.221:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 9943 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30919 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 08:59:30 -0000 --M/SuVGWktc5uNpra Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 30, 2016 at 11:35:49PM +0200, Ben RUBSON wrote: >=20 > > On 30 Jun 2016, at 18:35, Julien Cigar wrote: > >=20 > > On Thu, Jun 30, 2016 at 05:42:04PM +0200, Ben RUBSON wrote: > >>=20 > >>=20 > >>> On 30 Jun 2016, at 17:37, Julien Cigar wrote: > >>>=20 > >>>> On Thu, Jun 30, 2016 at 05:28:41PM +0200, Ben RUBSON wrote: > >>>>=20 > >>>>> On 30 Jun 2016, at 17:14, InterNetX - Juergen Gotteswinter wrote: > >>>>>=20 > >>>>>=20 > >>>>>=20 > >>>>>> Am 30.06.2016 um 16:45 schrieb Julien Cigar: > >>>>>> Hello, > >>>>>>=20 > >>>>>> I'm always in the process of setting a redundant low-cost storage = for=20 > >>>>>> our (small, ~30 people) team here. > >>>>>>=20 > >>>>>> I read quite a lot of articles/documentations/etc and I plan to us= e HAST > >>>>>> with ZFS for the storage, CARP for the failover and the "good old = NFS" > >>>>>> to mount the shares on the clients. > >>>>>>=20 > >>>>>> The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks fo= r the > >>>>>> shared storage. > >>>>>>=20 > >>>>>> Assuming the following configuration: > >>>>>> - MASTER is the active node and BACKUP is the standby node. > >>>>>> - two disks in each machine: ada0 and ada1. > >>>>>> - two interfaces in each machine: em0 and em1 > >>>>>> - em0 is the primary interface (with CARP setup) > >>>>>> - em1 is dedicated to the HAST traffic (crossover cable) > >>>>>> - FreeBSD is properly installed in each machine. > >>>>>> - a HAST resource "disk0" for ada0p2. > >>>>>> - a HAST resource "disk1" for ada1p2. > >>>>>> - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is c= reated > >>>>>> on MASTER > >>>>>>=20 > >>>>>> A couple of questions I am still wondering: > >>>>>> - If a disk dies on the MASTER I guess that zpool will not see it = and > >>>>>> will transparently use the one on BACKUP through the HAST ressourc= e.. > >>>>>=20 > >>>>> thats right, as long as writes on $anything have been successful ha= st is > >>>>> happy and wont start whining > >>>>>=20 > >>>>>> is it a problem?=20 > >>>>>=20 > >>>>> imho yes, at least from management view > >>>>>=20 > >>>>>> could this lead to some corruption? > >>>>>=20 > >>>>> probably, i never heard about anyone who uses that for long time in > >>>>> production > >>>>>=20 > >>>>> At this stage the > >>>>>> common sense would be to replace the disk quickly, but imagine the > >>>>>> worst case scenario where ada1 on MASTER dies, zpool will not see = it=20 > >>>>>> and will transparently use the one from the BACKUP node (through t= he=20 > >>>>>> "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not= =20 > >>>>>> see it and will transparently use the one from the BACKUP node=20 > >>>>>> (through the "disk0" HAST ressource). At this point on MASTER the = two=20 > >>>>>> disks are broken but the pool is still considered healthy ... What= if=20 > >>>>>> after that we unplug the em0 network cable on BACKUP? Storage is > >>>>>> down.. > >>>>>> - Under heavy I/O the MASTER box suddently dies (for some reasons)= ,=20 > >>>>>> thanks to CARP the BACKUP node will switch from standy -> active a= nd=20 > >>>>>> execute the failover script which does some "hastctl role primary"= for > >>>>>> the ressources and a zpool import. I wondered if there are any > >>>>>> situations where the pool couldn't be imported (=3D data corruptio= n)? > >>>>>> For example what if the pool hasn't been exported on the MASTER be= fore > >>>>>> it dies? > >>>>>> - Is it a problem if the NFS daemons are started at boot on the st= andby > >>>>>> node, or should they only be started in the failover script? What > >>>>>> about stale files and active connections on the clients? > >>>>>=20 > >>>>> sometimes stale mounts recover, sometimes not, sometimes clients ne= ed > >>>>> even reboots > >>>>>=20 > >>>>>> - A catastrophic power failure occur and MASTER and BACKUP are sud= dently > >>>>>> powered down. Later the power returns, is it possible that some > >>>>>> problem occur (split-brain scenario ?) regarding the order in whic= h the > >>>>>=20 > >>>>> sure, you need an exact procedure to recover > >>>>>=20 > >>>>>> two machines boot up? > >>>>>=20 > >>>>> best practice should be to keep everything down after boot > >>>>>=20 > >>>>>> - Other things I have not thought? > >>>>>>=20 > >>>>>=20 > >>>>>=20 > >>>>>=20 > >>>>>> Thanks! > >>>>>> Julien > >>>>>>=20 > >>>>>=20 > >>>>>=20 > >>>>> imho: > >>>>>=20 > >>>>> leave hast where it is, go for zfs replication. will save your butt, > >>>>> sooner or later if you avoid this fragile combination > >>>>=20 > >>>> I was also replying, and finishing by this : > >>>> Why don't you set your slave as an iSCSI target and simply do ZFS mi= rroring ? > >>>=20 > >>> Yes that's another option, so a zpool with two mirrors (local +=20 > >>> exported iSCSI) ? > >>=20 > >> Yes, you would then have a real time replication solution (as HAST), c= ompared to ZFS send/receive which is not. > >> Depends on what you need :) > >=20 > > More a real time replication solution in fact ... :) > > Do you have any resource which resume all the pro(s) and con(s) of HAST > > vs iSCSI ? I have found a lot of article on ZFS + HAST but not that much > > with ZFS + iSCSI ..=20 >=20 > # No resources, but some ideas : >=20 > - ZFS likes to see all the details of its underlying disks, which is poss= ible with local disks (of course) and iSCSI disks, not with HAST. > - iSCSI solution is simpler, you only have ZFS to manage, your replicatio= n is made by ZFS itself, not by an additional stack. > - HAST does not seem to be really maintained (I may be wrong), at least c= ompared to DRBD HAST seems to be inspired from. > - You do not have to cross your fingers when you promote your slave to ma= ster ("will ZFS be happy with my HAST replicated disks ?"), ZFS mirrored da= ta by itself, you only have to import [-f]. >=20 > - (auto)reconnection of iSCSI could not be as simple as with HAST, iSCSI = could require more administration after a disconnection. But this could eas= ily be done by a script. >=20 > # Some "advices" based on my findings (I'm finishing my tests of such a s= olution) : >=20 > Write performance will suffer from network latency, but while your 2 node= s are in the same room, that should be OK. > If you are over a long distance link, you may add several ms to each writ= e IO, which, depending on the use case, may be wrong, ZFS may also be unres= ponsive. > Max throughput is also more difficult to achieve over a high latency link. >=20 > You will have to choose network cards depending on the number of disks an= d their throughput. > For example, if you need to resilver a SATA disk (180MB/s), then a simple= 1GB interface (120MB/s) will be a serious bottleneck. > Think about scrub too. >=20 > You should have to perform some network tuning (TCP window size, jumbo fr= ame...) to reach your max bandwidth. > Trying to saturate network link with (for example) iPerf before dealing w= ith iSCSI seems to be a good thing. >=20 > Here are some interesting sysctl so that ZFS will not hang (too long) in = case of an unreachable iSCSI disk : > kern.iscsi.ping_timeout=3D5 > kern.iscsi.iscsid_timeout=3D5 > kern.iscsi.login_timeout=3D5 > kern.iscsi.fail_on_disconnection=3D1 > (adjust the 5 seconds depending on your needs / on your network quality). >=20 > Take care when you (auto)replace disks, you may replace an iSCSI disk wit= h a local disk, which of course would work but would be wrong in terms of m= aster/slave redundancy. > Use nice labels on your disks so that if you have a lot of disks in your = pool, you quickly know which one is local, which one is remote. >=20 > # send/receive pro(s) : >=20 > In terms of data safety, one of the interests of ZFS send/receive is that= you have a totally different target pool, which can be interesting if ever= you have a disaster with your primary pool. > As a 3rd node solution ? On another site ? (as send/receive does not suff= er as iSCSI would from latency) Thank you very much for those "advices", it is much appreciated!=20 I'll definitively go with iSCSI (for which I haven't that much=20 experience) over HAST. Maybe a stupid question but, assuming on the MASTER with ada{0,1} the=20 local disks and da{0,1} the exported iSCSI disks from the SLAVE, would=20 you go with: $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 /dev/da1 or rather: $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 /dev/da1 I guess the former is better, but it's just to be sure .. (or maybe it's better to iSCSI export a ZVOL from the SLAVE?) Correct me if I'm wrong but, from a safety point of view this setup is=20 also the safest as you'll get the "fullsync" equivalent mode of HAST (but but it's also the slowest), so I can be 99,99% confident that the pool on the SLAVE will never be corrupted, even in the case where the MASTER suddently die (power outage, etc), and that a zpool import -f storage will always work? One last thing: this "storage" pool will be exported through NFS on the=20 clients, and when a failover occur they should, in theory, not notice it. I know that it's pretty hypothetical but I wondered if pfsync could play a role in this area (active connections)..? Thanks! Julien >=20 > >>>> ZFS would then know as soon as a disk is failing. > >>>> And if the master fails, you only have to import (-f certainly, in c= ase of a master power failure) on the slave. > >>>>=20 > >>>> Ben > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --M/SuVGWktc5uNpra Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdi4SAAoJELK7NxCiBCPAx+gQAJms3tvFVEensADV8Iys+ScD k2uU6Gtj7V7ZtoHf5wGdcu467UffTKozk5VFVmkSv1kP0rDkyi/IqMPia30jq4E4 43fhyLmZysMY9E8Lehm5ECX04uIouhaLaJpPn2flV4BMzsdKKoid0UTs4+gKv1e5 yjdCfLIOA5tBNl5wdB3+5+sdepPJprxyGb8grpQ+RlifvxUkIgf0NW3W0Th52nY1 6m2lgd900ZzAD+ySab15xbJ6dg2/bJnLkc3RHfryc3fPRiL8nydsr1yv/FDTNxGp xfXeDQ+CriHcRaxVYOXtAeaXlETXMTRByd5LYqwEXAQl4sutKuBbqnv9zLYpYJhj BGOMUG4z/5UEDc2EtKFCBH6AC/qpM4YUbI0POZoEnorO0zp96bBUTLOwPZn3ovZ5 g11O9qdGJls2o5Lvtue1ZxxpvJR1Bd3zZxOH3RBB591RPlvubSZPXRIeccc0Q9S9 /Z5yrV4Zw+t/xt5qJlWx6fnCy2iXaGn44XWgmVgZ0fhgfPoRn4Ae13i2CG7ky4f6 ieq9OpSp2oq6Huu/u/m04WFjWLMfZyq2dFS4eHua+7al7D++n/sIiYMwSgwIgjjh YLJzExZqE7xDxe+gHUHE48vqdsfJveJYG3xWfTh1hC1z5sy8v3fTbDFFMkRUXCBK hyytLtlAU/LjmO5/mApt =fvNz -----END PGP SIGNATURE----- --M/SuVGWktc5uNpra-- From owner-freebsd-fs@freebsd.org Fri Jul 1 09:19:12 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF69BB887A8 for ; Fri, 1 Jul 2016 09:19:12 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C5F5287F for ; Fri, 1 Jul 2016 09:19:12 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id v199so18419706wmv.0 for ; Fri, 01 Jul 2016 02:19:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=9I7AKZWQtgPQxzx2ov5AbuqfLZ1hWl+5s0VJ2llZo5A=; b=0pKwrJXrfXmEaNoZwb3smFEza1scdzIEkFG0OrBLskPVQ9s0U09E8KO+rnBgTiXvYg 09Rvpd8IkahoO4Xa4EZX6E8PaLBj7pSwGfLV5/TA47wRlavDisc4bLKQ7MtIhZBU1jz7 o6PrcaCCJIPcoUP09EKLgAwISWS3G8WUlE6TUkVmQ1ORBC8dY7+gqbGnNIgF+clLKZHN LL0SGBszz1uBCudhMAoATsydXSfhpGEQ+ZHfY3CFW8y1M4hH+4exxwLfVgc5TDWm5rFx RLLk7IPNGDNPgoGoflr/8nae7DtwyixvX6tY0lUxBwwnIw0X+n4LsZKKcEL+oLav4Hyi 67xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=9I7AKZWQtgPQxzx2ov5AbuqfLZ1hWl+5s0VJ2llZo5A=; b=dAlTp61CB8/p/hjn1dZERsa4AG/oGOUjwaCggsajZ7mA10J3l4N85S3zxF0QkogW8L atMY41IB1wwJl7zQi0vNAyHL3Iv/clN+UgrjRdfhcjE2tJf65rxQdWGR4sI442jEnm70 nPOClwvMvtqMoBrtPUJq+IjbOOFAL7gMb1Z8eKdLtxYr84z/YeYxDagWL6hO7CbUnaZP cypNHLV0RAQsCds4Ubk9IprD+yBN1h3YQLKxtv6WIPxaFRHjYUpbX1kHQaQt+UMg+jTB dtCLjWnPV4YkFTCLfc37koeFShIKUM9vXmOeUl4h74W+FzAjsc9ueBchn6Mv6fsWg5N+ 8rIA== X-Gm-Message-State: ALyK8tK+kiIGnDpg3ysDQC0/x/E4eY49/66dj1u227NbEnFt4TrvTMD3T77l5pPiOmKeJw== X-Received: by 10.28.57.212 with SMTP id g203mr19208544wma.7.1467364750372; Fri, 01 Jul 2016 02:19:10 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id b4sm1333615wjd.16.2016.07.01.02.19.09 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Jul 2016 02:19:09 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: <20160701084717.GE5695@mordor.lan> Date: Fri, 1 Jul 2016 11:19:08 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <26A31227-B71D-4854-B046-61CD3449E442@gmail.com> References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 09:19:13 -0000 > On 01 Jul 2016, at 10:47, Julien Cigar wrote: >=20 > On Thu, Jun 30, 2016 at 11:35:49PM +0200, Ben RUBSON wrote: >>=20 >>> On 30 Jun 2016, at 18:35, Julien Cigar = wrote: >>>=20 >>> On Thu, Jun 30, 2016 at 05:42:04PM +0200, Ben RUBSON wrote: >>>>=20 >>>>=20 >>>>> On 30 Jun 2016, at 17:37, Julien Cigar = wrote: >>>>>=20 >>>>>> On Thu, Jun 30, 2016 at 05:28:41PM +0200, Ben RUBSON wrote: >>>>>>=20 >>>>>>> On 30 Jun 2016, at 17:14, InterNetX - Juergen Gotteswinter = wrote: >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>> Am 30.06.2016 um 16:45 schrieb Julien Cigar: >>>>>>>> Hello, >>>>>>>>=20 >>>>>>>> I'm always in the process of setting a redundant low-cost = storage for=20 >>>>>>>> our (small, ~30 people) team here. >>>>>>>>=20 >>>>>>>> I read quite a lot of articles/documentations/etc and I plan to = use HAST >>>>>>>> with ZFS for the storage, CARP for the failover and the "good = old NFS" >>>>>>>> to mount the shares on the clients. >>>>>>>>=20 >>>>>>>> The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks = for the >>>>>>>> shared storage. >>>>>>>>=20 >>>>>>>> Assuming the following configuration: >>>>>>>> - MASTER is the active node and BACKUP is the standby node. >>>>>>>> - two disks in each machine: ada0 and ada1. >>>>>>>> - two interfaces in each machine: em0 and em1 >>>>>>>> - em0 is the primary interface (with CARP setup) >>>>>>>> - em1 is dedicated to the HAST traffic (crossover cable) >>>>>>>> - FreeBSD is properly installed in each machine. >>>>>>>> - a HAST resource "disk0" for ada0p2. >>>>>>>> - a HAST resource "disk1" for ada1p2. >>>>>>>> - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 = is created >>>>>>>> on MASTER >>>>>>>>=20 >>>>>>>> A couple of questions I am still wondering: >>>>>>>> - If a disk dies on the MASTER I guess that zpool will not see = it and >>>>>>>> will transparently use the one on BACKUP through the HAST = ressource.. >>>>>>>=20 >>>>>>> thats right, as long as writes on $anything have been successful = hast is >>>>>>> happy and wont start whining >>>>>>>=20 >>>>>>>> is it a problem?=20 >>>>>>>=20 >>>>>>> imho yes, at least from management view >>>>>>>=20 >>>>>>>> could this lead to some corruption? >>>>>>>=20 >>>>>>> probably, i never heard about anyone who uses that for long time = in >>>>>>> production >>>>>>>=20 >>>>>>> At this stage the >>>>>>>> common sense would be to replace the disk quickly, but imagine = the >>>>>>>> worst case scenario where ada1 on MASTER dies, zpool will not = see it=20 >>>>>>>> and will transparently use the one from the BACKUP node = (through the=20 >>>>>>>> "disk1" HAST ressource), later ada0 on MASTER dies, zpool will = not=20 >>>>>>>> see it and will transparently use the one from the BACKUP node=20= >>>>>>>> (through the "disk0" HAST ressource). At this point on MASTER = the two=20 >>>>>>>> disks are broken but the pool is still considered healthy ... = What if=20 >>>>>>>> after that we unplug the em0 network cable on BACKUP? Storage = is >>>>>>>> down.. >>>>>>>> - Under heavy I/O the MASTER box suddently dies (for some = reasons),=20 >>>>>>>> thanks to CARP the BACKUP node will switch from standy -> = active and=20 >>>>>>>> execute the failover script which does some "hastctl role = primary" for >>>>>>>> the ressources and a zpool import. I wondered if there are any >>>>>>>> situations where the pool couldn't be imported (=3D data = corruption)? >>>>>>>> For example what if the pool hasn't been exported on the MASTER = before >>>>>>>> it dies? >>>>>>>> - Is it a problem if the NFS daemons are started at boot on the = standby >>>>>>>> node, or should they only be started in the failover script? = What >>>>>>>> about stale files and active connections on the clients? >>>>>>>=20 >>>>>>> sometimes stale mounts recover, sometimes not, sometimes clients = need >>>>>>> even reboots >>>>>>>=20 >>>>>>>> - A catastrophic power failure occur and MASTER and BACKUP are = suddently >>>>>>>> powered down. Later the power returns, is it possible that some >>>>>>>> problem occur (split-brain scenario ?) regarding the order in = which the >>>>>>>=20 >>>>>>> sure, you need an exact procedure to recover >>>>>>>=20 >>>>>>>> two machines boot up? >>>>>>>=20 >>>>>>> best practice should be to keep everything down after boot >>>>>>>=20 >>>>>>>> - Other things I have not thought? >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>> Thanks! >>>>>>>> Julien >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> imho: >>>>>>>=20 >>>>>>> leave hast where it is, go for zfs replication. will save your = butt, >>>>>>> sooner or later if you avoid this fragile combination >>>>>>=20 >>>>>> I was also replying, and finishing by this : >>>>>> Why don't you set your slave as an iSCSI target and simply do ZFS = mirroring ? >>>>>=20 >>>>> Yes that's another option, so a zpool with two mirrors (local +=20 >>>>> exported iSCSI) ? >>>>=20 >>>> Yes, you would then have a real time replication solution (as = HAST), compared to ZFS send/receive which is not. >>>> Depends on what you need :) >>>=20 >>> More a real time replication solution in fact ... :) >>> Do you have any resource which resume all the pro(s) and con(s) of = HAST >>> vs iSCSI ? I have found a lot of article on ZFS + HAST but not that = much >>> with ZFS + iSCSI ..=20 >>=20 >> # No resources, but some ideas : >>=20 >> - ZFS likes to see all the details of its underlying disks, which is = possible with local disks (of course) and iSCSI disks, not with HAST. >> - iSCSI solution is simpler, you only have ZFS to manage, your = replication is made by ZFS itself, not by an additional stack. >> - HAST does not seem to be really maintained (I may be wrong), at = least compared to DRBD HAST seems to be inspired from. >> - You do not have to cross your fingers when you promote your slave = to master ("will ZFS be happy with my HAST replicated disks ?"), ZFS = mirrored data by itself, you only have to import [-f]. >>=20 >> - (auto)reconnection of iSCSI could not be as simple as with HAST, = iSCSI could require more administration after a disconnection. But this = could easily be done by a script. >>=20 >> # Some "advices" based on my findings (I'm finishing my tests of such = a solution) : >>=20 >> Write performance will suffer from network latency, but while your 2 = nodes are in the same room, that should be OK. >> If you are over a long distance link, you may add several ms to each = write IO, which, depending on the use case, may be wrong, ZFS may also = be unresponsive. >> Max throughput is also more difficult to achieve over a high latency = link. >>=20 >> You will have to choose network cards depending on the number of = disks and their throughput. >> For example, if you need to resilver a SATA disk (180MB/s), then a = simple 1GB interface (120MB/s) will be a serious bottleneck. >> Think about scrub too. >>=20 >> You should have to perform some network tuning (TCP window size, = jumbo frame...) to reach your max bandwidth. >> Trying to saturate network link with (for example) iPerf before = dealing with iSCSI seems to be a good thing. >>=20 >> Here are some interesting sysctl so that ZFS will not hang (too long) = in case of an unreachable iSCSI disk : >> kern.iscsi.ping_timeout=3D5 >> kern.iscsi.iscsid_timeout=3D5 >> kern.iscsi.login_timeout=3D5 >> kern.iscsi.fail_on_disconnection=3D1 >> (adjust the 5 seconds depending on your needs / on your network = quality). >>=20 >> Take care when you (auto)replace disks, you may replace an iSCSI disk = with a local disk, which of course would work but would be wrong in = terms of master/slave redundancy. >> Use nice labels on your disks so that if you have a lot of disks in = your pool, you quickly know which one is local, which one is remote. >>=20 >> # send/receive pro(s) : >>=20 >> In terms of data safety, one of the interests of ZFS send/receive is = that you have a totally different target pool, which can be interesting = if ever you have a disaster with your primary pool. >> As a 3rd node solution ? On another site ? (as send/receive does not = suffer as iSCSI would from latency) >=20 > Thank you very much for those "advices", it is much appreciated!=20 >=20 > I'll definitively go with iSCSI (for which I haven't that much=20 > experience) over HAST. >=20 > Maybe a stupid question but, assuming on the MASTER with ada{0,1} the=20= > local disks and da{0,1} the exported iSCSI disks from the SLAVE, would=20= > you go with: >=20 > $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 > /dev/da1 No, if you loose connection with slave node, your pool will go offline ! > or rather: >=20 > $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > /dev/da1 Yes, each master disk is mirrored with a slave disk. > I guess the former is better, but it's just to be sure .. (or maybe = it's > better to iSCSI export a ZVOL from the SLAVE?) >=20 > Correct me if I'm wrong but, from a safety point of view this setup is=20= > also the safest as you'll get the "fullsync" equivalent mode of HAST > (but but it's also the slowest), so I can be 99,99% confident that the > pool on the SLAVE will never be corrupted, even in the case where the > MASTER suddently die (power outage, etc), and that a zpool import -f > storage will always work? Pool on slave is the same as pool on master, as it uses the same disks = :) Only the physical host will change. So yes you can be confident. There is still the case where any ZFS pool could be totally damaged (due = to a bug for example). It "should" not arrive, but we never know :) This is why I was talking about a third node / second pool made from a = delayed send/receive. > One last thing: this "storage" pool will be exported through NFS on = the=20 > clients, and when a failover occur they should, in theory, not notice > it. I know that it's pretty hypothetical but I wondered if pfsync = could > play a role in this area (active connections)..? There will certainly be some small timeouts due to the failover delay. You should make some tests to analyze NFS behaviour depending on the = failover delay. Good question regarding pfsync, I'm not so familiar with it :) Of course, make a good POC before going with this into production. Don't forget to test scrub, resilver, power failure, network failure... And perhaps one may have additional comments / ideas / reserve on this = topic ? > Thanks! > Julien >=20 >>=20 >>>>>> ZFS would then know as soon as a disk is failing. >>>>>> And if the master fails, you only have to import (-f certainly, = in case of a master power failure) on the slave. >>>>>>=20 >>>>>> Ben >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 > --=20 > Julien Cigar > Belgian Biodiversity Platform (http://www.biodiversity.be) > PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 > No trees were killed in the creation of this message. > However, many electrons were terribly inconvenienced. From owner-freebsd-fs@freebsd.org Fri Jul 1 09:42:19 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 12B8AB88F7A for ; Fri, 1 Jul 2016 09:42:19 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB1B6224E for ; Fri, 1 Jul 2016 09:42:18 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id C79C945FC0E4; Fri, 1 Jul 2016 11:42:15 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CzIXECtz2mKi; Fri, 1 Jul 2016 11:42:13 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 706F745FC0E5; Fri, 1 Jul 2016 11:42:13 +0200 (CEST) Subject: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> To: Julien Cigar , Ben RUBSON Cc: freebsd-fs@freebsd.org Reply-To: jg@internetx.com From: InterNetX - Juergen Gotteswinter Message-ID: <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> Date: Fri, 1 Jul 2016 11:42:13 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160701084717.GE5695@mordor.lan> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 09:42:19 -0000 > > Thank you very much for those "advices", it is much appreciated! > > I'll definitively go with iSCSI (for which I haven't that much > experience) over HAST. good luck, i rather cut one of my fingers than using something like this in production. but its probably a quick way if one targets to find a new opportunity ;) > > Maybe a stupid question but, assuming on the MASTER with ada{0,1} the > local disks and da{0,1} the exported iSCSI disks from the SLAVE, would > you go with: > > $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 > /dev/da1 > > or rather: > > $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > /dev/da1 > > I guess the former is better, but it's just to be sure .. (or maybe it's > better to iSCSI export a ZVOL from the SLAVE?) > are you really sure you understand what you trying to do? even if its currently so, i bet in a desaster case you will be lost. > Correct me if I'm wrong but, from a safety point of view this setup is > also the safest as you'll get the "fullsync" equivalent mode of HAST > (but but it's also the slowest), so I can be 99,99% confident that the > pool on the SLAVE will never be corrupted, even in the case where the > MASTER suddently die (power outage, etc), and that a zpool import -f > storage will always work? 99,99% ? optimistic, very optimistic. we are playing with recovery of a test pool which has been imported on two nodes at the same time. looks pretty messy > > One last thing: this "storage" pool will be exported through NFS on the > clients, and when a failover occur they should, in theory, not notice > it. I know that it's pretty hypothetical but I wondered if pfsync could > play a role in this area (active connections)..? > they will notice, and they will stuck or worse (reboot) > Thanks! > Julien > >> >>>>>> ZFS would then know as soon as a disk is failing. >>>>>> And if the master fails, you only have to import (-f certainly, in case of a master power failure) on the slave. >>>>>> >>>>>> Ben >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Jul 1 10:15:29 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89586B8991B for ; Fri, 1 Jul 2016 10:15:29 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b01.edpnet.be (relay-b01.edpnet.be [212.71.1.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 353E42E15 for ; Fri, 1 Jul 2016 10:15:28 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467368124-0a7ff569f61010f70001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b01.edpnet.be with ESMTP id ZjsvAlY8xm21USlM (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 12:15:25 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 12:15:24 +0200 From: Julien Cigar To: InterNetX - Juergen Gotteswinter Cc: Ben RUBSON , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701101524.GF5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="w3uUfsyyY1Pqa/ej" Content-Disposition: inline In-Reply-To: <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467368124 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.221:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3817 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30920 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 10:15:29 -0000 --w3uUfsyyY1Pqa/ej Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 11:42:13AM +0200, InterNetX - Juergen Gotteswinter = wrote: > >=20 > > Thank you very much for those "advices", it is much appreciated!=20 > >=20 > > I'll definitively go with iSCSI (for which I haven't that much=20 > > experience) over HAST. >=20 > good luck, i rather cut one of my fingers than using something like this > in production. but its probably a quick way if one targets to find a new > opportunity ;) why...? I guess iSCSI is slower but should be safer than HAST, no? >=20 > >=20 > > Maybe a stupid question but, assuming on the MASTER with ada{0,1} the= =20 > > local disks and da{0,1} the exported iSCSI disks from the SLAVE, would= =20 > > you go with: > >=20 > > $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 > > /dev/da1 > >=20 > > or rather: > >=20 > > $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > > /dev/da1 > >=20 > > I guess the former is better, but it's just to be sure .. (or maybe it's > > better to iSCSI export a ZVOL from the SLAVE?) > >=20 >=20 > are you really sure you understand what you trying to do? even if its > currently so, i bet in a desaster case you will be lost. >=20 >=20 well this is pretty new to me, but I don't see what could be wrong with: $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 /dev/da1 Let's take some use-cases: - MASTER and SLAVE are alive, the data is "replicated" on both nodes. As iSCSI is used, ZFS will see all the details of the underlying disks and we can be sure that no corruption will occur (contrary to HAST) - SLAVE die, correct me if I'm wrong the but pool is still available, fix the SLAVE, resilver and that's it ..? - MASTER die, CARP will notice it and SLAVE will take the VIP, the failover script will be executed with a $> zpool import -f > > Correct me if I'm wrong but, from a safety point of view this setup is= =20 > > also the safest as you'll get the "fullsync" equivalent mode of HAST > > (but but it's also the slowest), so I can be 99,99% confident that the > > pool on the SLAVE will never be corrupted, even in the case where the > > MASTER suddently die (power outage, etc), and that a zpool import -f > > storage will always work? >=20 > 99,99% ? optimistic, very optimistic. the only situation where corruption could occur is some sort of network corruption (bug in the driver, broken network card, etc), or a bug in ZFS ... but you'll have the same with a zfs send|ssh zfs receive >=20 > we are playing with recovery of a test pool which has been imported on > two nodes at the same time. looks pretty messy >=20 > >=20 > > One last thing: this "storage" pool will be exported through NFS on the= =20 > > clients, and when a failover occur they should, in theory, not notice > > it. I know that it's pretty hypothetical but I wondered if pfsync could > > play a role in this area (active connections)..? > >=20 >=20 > they will notice, and they will stuck or worse (reboot) this is something that should be properly tested I agree.. >=20 > > Thanks! > > Julien > >=20 > >> > >>>>>> ZFS would then know as soon as a disk is failing. > >>>>>> And if the master fails, you only have to import (-f certainly, in= case of a master power failure) on the slave. > >>>>>> > >>>>>> Ben > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >=20 --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --w3uUfsyyY1Pqa/ej Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdkK4AAoJELK7NxCiBCPAH3YQAOQsTySZm0whwykZIiG0ivVk nvik34mCVTWMpmtfhTW3m9+u+0YeoQlnyT1sI4Mtw+hLrU0U+EZoDafAk99jnaRA WHyLi+0232fdGD8K0VO+tq3JOqCAJBN5AzXTVzJtDzyLnMl1x+AzX4c6FMIQMyiV tSDBYJ8vmJmKWrlvvsNaxheVadnkitpsHuHYMykw+NKKv6MRVNtMlWtGIUCmAhbf o0fA+LsWgpkWNi5q6IpUqdbRc4Btz8XR9v5zkQiY4gDWN146AIFzWi8irCzqnoe8 Hgk9ZrPoPWlfYW12kWkO++zyd5zFEO+0SY+o9I996snXA2soaMyJkAGbISnoqX3F 5zQ9nVKvf/A9xtLZyLBIxm89ltxomAtxVCL/iEQ/NNKXxaz02DXDpbgA3aO1mloj WpVnldKWeHh3cVXc4+6t2LdYUsCY5ZrRetN0RPCn1HMs0UJvHPVukl0EldgbBTqa Q2E4A9Uz+sgEpqPJqIIOnrf1XOy29ax4jpUljswH1LMkrt81KhumEsRLlgBLEkwx 1/YmSCjYorV1QW1fGmyCi0+lVUm7WZB4DPkj+JEQqe12JE7acjy7fOgBGCt5YIv2 DEarGdAFqCQp+KzlFK+Thnk5n+PCtXv4GJOLWDrqRVLYmsJb3A4aml+uj+ZRs2H3 seWCUQE2MCAJKihq7h45 =SNJt -----END PGP SIGNATURE----- --w3uUfsyyY1Pqa/ej-- From owner-freebsd-fs@freebsd.org Fri Jul 1 10:18:45 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33C3CB899C8 for ; Fri, 1 Jul 2016 10:18:45 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id BD8962F7B for ; Fri, 1 Jul 2016 10:18:44 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id BA9744C4C87D; Fri, 1 Jul 2016 12:18:42 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DGADzSD4Htby; Fri, 1 Jul 2016 12:18:40 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 45B194C4C87B; Fri, 1 Jul 2016 12:18:40 +0200 (CEST) Reply-To: jg@internetx.com Subject: Re: HAST + ZFS + NFS + CARP References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> To: Julien Cigar Cc: Ben RUBSON , freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Message-ID: Date: Fri, 1 Jul 2016 12:18:39 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160701101524.GF5695@mordor.lan> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 10:18:45 -0000 Am 01.07.2016 um 12:15 schrieb Julien Cigar: > On Fri, Jul 01, 2016 at 11:42:13AM +0200, InterNetX - Juergen Gotteswinter wrote: >>> >>> Thank you very much for those "advices", it is much appreciated! >>> >>> I'll definitively go with iSCSI (for which I haven't that much >>> experience) over HAST. >> >> good luck, i rather cut one of my fingers than using something like this >> in production. but its probably a quick way if one targets to find a new >> opportunity ;) > > why...? I guess iSCSI is slower but should be safer than HAST, no? do your testing, please. even with simulated short network cuts. 10-20 secs are way enaugh to give you a picture of what is going to happen >> >>> >>> Maybe a stupid question but, assuming on the MASTER with ada{0,1} the >>> local disks and da{0,1} the exported iSCSI disks from the SLAVE, would >>> you go with: >>> >>> $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 >>> /dev/da1 >>> >>> or rather: >>> >>> $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 >>> /dev/da1 >>> >>> I guess the former is better, but it's just to be sure .. (or maybe it's >>> better to iSCSI export a ZVOL from the SLAVE?) >>> >> >> are you really sure you understand what you trying to do? even if its >> currently so, i bet in a desaster case you will be lost. >> >> > > well this is pretty new to me, but I don't see what could be wrong with: > > $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > /dev/da1 > > Let's take some use-cases: > - MASTER and SLAVE are alive, the data is "replicated" on both > nodes. As iSCSI is used, ZFS will see all the details of the > underlying disks and we can be sure that no corruption will occur > (contrary to HAST) > - SLAVE die, correct me if I'm wrong the but pool is still available, > fix the SLAVE, resilver and that's it ..? > - MASTER die, CARP will notice it and SLAVE will take the VIP, the > failover script will be executed with a $> zpool import -f > >>> Correct me if I'm wrong but, from a safety point of view this setup is >>> also the safest as you'll get the "fullsync" equivalent mode of HAST >>> (but but it's also the slowest), so I can be 99,99% confident that the >>> pool on the SLAVE will never be corrupted, even in the case where the >>> MASTER suddently die (power outage, etc), and that a zpool import -f >>> storage will always work? >> >> 99,99% ? optimistic, very optimistic. > > the only situation where corruption could occur is some sort of network > corruption (bug in the driver, broken network card, etc), or a bug in > ZFS ... but you'll have the same with a zfs send|ssh zfs receive > >> optimistic >> we are playing with recovery of a test pool which has been imported on >> two nodes at the same time. looks pretty messy >> >>> >>> One last thing: this "storage" pool will be exported through NFS on the >>> clients, and when a failover occur they should, in theory, not notice >>> it. I know that it's pretty hypothetical but I wondered if pfsync could >>> play a role in this area (active connections)..? >>> >> >> they will notice, and they will stuck or worse (reboot) > > this is something that should be properly tested I agree.. > do your testing, and keep your clients under load while testing. do writes onto the nfs mounts and then cut. you will be surprised about the impact. >> >>> Thanks! >>> Julien >>> >>>> >>>>>>>> ZFS would then know as soon as a disk is failing. >>>>>>>> And if the master fails, you only have to import (-f certainly, in case of a master power failure) on the slave. >>>>>>>> >>>>>>>> Ben >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> > From owner-freebsd-fs@freebsd.org Fri Jul 1 10:57:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7404FB8E430 for ; Fri, 1 Jul 2016 10:57:41 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b02.edpnet.be (relay-b02.edpnet.be [212.71.1.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26F3B209F for ; Fri, 1 Jul 2016 10:57:40 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467370655-0a7b8d1905c0a50001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b02.edpnet.be with ESMTP id ZHfirQ03CvbH3WzE (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 12:57:36 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 12:57:35 +0200 From: Julien Cigar To: InterNetX - Juergen Gotteswinter Cc: Ben RUBSON , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701105735.GG5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Qf1oXS95uex85X0R" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467370655 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.222:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 4964 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30921 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 10:57:41 -0000 --Qf1oXS95uex85X0R Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotteswinter = wrote: > Am 01.07.2016 um 12:15 schrieb Julien Cigar: > > On Fri, Jul 01, 2016 at 11:42:13AM +0200, InterNetX - Juergen Gotteswin= ter wrote: > >>> > >>> Thank you very much for those "advices", it is much appreciated!=20 > >>> > >>> I'll definitively go with iSCSI (for which I haven't that much=20 > >>> experience) over HAST. > >> > >> good luck, i rather cut one of my fingers than using something like th= is > >> in production. but its probably a quick way if one targets to find a n= ew > >> opportunity ;) > >=20 > > why...? I guess iSCSI is slower but should be safer than HAST, no? >=20 > do your testing, please. even with simulated short network cuts. 10-20 > secs are way enaugh to give you a picture of what is going to happen of course I'll test everything properly :) I don't have the hardware yet so ATM I'm just looking for all the possible "candidates", and I'm=20 aware that a redundant storage is not that easy to implement ... but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI),=20 either zfs send|ssh zfs receive as you suggest (but it's not realtime), either a distributed FS (which I avoid like the plague..) >=20 > >> > >>> > >>> Maybe a stupid question but, assuming on the MASTER with ada{0,1} the= =20 > >>> local disks and da{0,1} the exported iSCSI disks from the SLAVE, woul= d=20 > >>> you go with: > >>> > >>> $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 > >>> /dev/da1 > >>> > >>> or rather: > >>> > >>> $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > >>> /dev/da1 > >>> > >>> I guess the former is better, but it's just to be sure .. (or maybe i= t's > >>> better to iSCSI export a ZVOL from the SLAVE?) > >>> > >> > >> are you really sure you understand what you trying to do? even if its > >> currently so, i bet in a desaster case you will be lost. > >> > >> > >=20 > > well this is pretty new to me, but I don't see what could be wrong with: > >=20 > > $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > > /dev/da1 > >=20 > > Let's take some use-cases: > > - MASTER and SLAVE are alive, the data is "replicated" on both > > nodes. As iSCSI is used, ZFS will see all the details of the > > underlying disks and we can be sure that no corruption will occur > > (contrary to HAST) > > - SLAVE die, correct me if I'm wrong the but pool is still available, > > fix the SLAVE, resilver and that's it ..? > > - MASTER die, CARP will notice it and SLAVE will take the VIP, the > > failover script will be executed with a $> zpool import -f > >=20 > >>> Correct me if I'm wrong but, from a safety point of view this setup i= s=20 > >>> also the safest as you'll get the "fullsync" equivalent mode of HAST > >>> (but but it's also the slowest), so I can be 99,99% confident that the > >>> pool on the SLAVE will never be corrupted, even in the case where the > >>> MASTER suddently die (power outage, etc), and that a zpool import -f > >>> storage will always work? > >> > >> 99,99% ? optimistic, very optimistic. > >=20 > > the only situation where corruption could occur is some sort of network > > corruption (bug in the driver, broken network card, etc), or a bug in > > ZFS ... but you'll have the same with a zfs send|ssh zfs receive > >=20 > >> >=20 > optimistic >=20 > >> we are playing with recovery of a test pool which has been imported on > >> two nodes at the same time. looks pretty messy > >> > >>> > >>> One last thing: this "storage" pool will be exported through NFS on t= he=20 > >>> clients, and when a failover occur they should, in theory, not notice > >>> it. I know that it's pretty hypothetical but I wondered if pfsync cou= ld > >>> play a role in this area (active connections)..? > >>> > >> > >> they will notice, and they will stuck or worse (reboot) > >=20 > > this is something that should be properly tested I agree.. > >=20 >=20 > do your testing, and keep your clients under load while testing. do > writes onto the nfs mounts and then cut. you will be surprised about the > impact. >=20 > >> >=20 >=20 >=20 > >>> Thanks! > >>> Julien > >>> > >>>> > >>>>>>>> ZFS would then know as soon as a disk is failing. > >>>>>>>> And if the master fails, you only have to import (-f certainly, = in case of a master power failure) on the slave. > >>>>>>>> > >>>>>>>> Ben > >>>> _______________________________________________ > >>>> freebsd-fs@freebsd.org mailing list > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >>> > >=20 >=20 --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --Qf1oXS95uex85X0R Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdkybAAoJELK7NxCiBCPAXy4QAKb/OswvhYZVN/0FcWRLdrPk X7oBaNKYTXnVa2+CWpkKWIXWzLuYjtU7mO8YpDeTRR97HwWaz39hQlC86Rtwvl+K bU91c5h/ykCD9pBe7IOA8bu86ejGJyW619C3VjbPFhRIpslNRULu5lGFaGSfka2S 7JiGXwicr8AhQd8TAj9DJCQcK/pr8u9k3KYgqU4MxxFrr14+++HzRKk9m/fFpQBy LhIK4DIzc32Wbta0bIfOeIhNS7ATJhXcyfoWEKHEy5JUq5j78p779An76UuyJLil 63H9s3pKT1QdEFiiVBjIEz02Y8XHDXEpqDgA9TRWLw+jGngXhGs79Syjdzo0RPlT DyV5TIgjcXoDyONvxFxcBsiEgI25LzKFwNQeTOWYEQzST1hAolzAgMEK0cKiQfde 3OBp3jvOocT0/7XfTE/Il8UWMdad2fZS6VAJGX0Ei9TXHCHCMhaTkCoqZ20py//7 4k4gBV3yeQpyMkHKHDlokRV63SnDON+V5/XyAyvbPS0df3zmhMVe2v0d0ODDxAis MQBkfnBtzQ6MT3OZWwL+xQgXA1u+iXkZsbWklGwcxTCy52wCLlBNtYjEyEoxX6Yg xEV1KG8m2u2e/U/uoBW9/aKKJC8wLmsDjMkss/ReO/aNIX+cZsTUQcq0dV73kW3O 6Z8kXmiz6O2xK5VmEksm =64Uc -----END PGP SIGNATURE----- --Qf1oXS95uex85X0R-- From owner-freebsd-fs@freebsd.org Fri Jul 1 11:09:58 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD004B8E6E6 for ; Fri, 1 Jul 2016 11:09:58 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 395DA2524 for ; Fri, 1 Jul 2016 11:09:57 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 5032445FC0E4; Fri, 1 Jul 2016 13:09:55 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X+0YQe7FwG9s; Fri, 1 Jul 2016 13:09:52 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id A320D4C4C54B; Fri, 1 Jul 2016 13:09:52 +0200 (CEST) Reply-To: jg@internetx.com Subject: Re: HAST + ZFS + NFS + CARP References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> To: Julien Cigar Cc: Ben RUBSON , freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Message-ID: <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> Date: Fri, 1 Jul 2016 13:09:52 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160701105735.GG5695@mordor.lan> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 11:09:58 -0000 Am 01.07.2016 um 12:57 schrieb Julien Cigar: > On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotteswinter wrote: >> Am 01.07.2016 um 12:15 schrieb Julien Cigar: >>> On Fri, Jul 01, 2016 at 11:42:13AM +0200, InterNetX - Juergen Gotteswinter wrote: >>>>> >>>>> Thank you very much for those "advices", it is much appreciated! >>>>> >>>>> I'll definitively go with iSCSI (for which I haven't that much >>>>> experience) over HAST. >>>> >>>> good luck, i rather cut one of my fingers than using something like this >>>> in production. but its probably a quick way if one targets to find a new >>>> opportunity ;) >>> >>> why...? I guess iSCSI is slower but should be safer than HAST, no? >> >> do your testing, please. even with simulated short network cuts. 10-20 >> secs are way enaugh to give you a picture of what is going to happen > > of course I'll test everything properly :) I don't have the hardware yet > so ATM I'm just looking for all the possible "candidates", and I'm > aware that a redundant storage is not that easy to implement ... > > but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), > either zfs send|ssh zfs receive as you suggest (but it's > not realtime), either a distributed FS (which I avoid like the plague..) zfs send/receive can be nearly realtime. external jbods with cross cabled sas + commercial cluster solution like rsf-1. anything else is a fragile construction which begs for desaster. > >> >>>> >>>>> >>>>> Maybe a stupid question but, assuming on the MASTER with ada{0,1} the >>>>> local disks and da{0,1} the exported iSCSI disks from the SLAVE, would >>>>> you go with: >>>>> >>>>> $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 >>>>> /dev/da1 >>>>> >>>>> or rather: >>>>> >>>>> $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 >>>>> /dev/da1 >>>>> >>>>> I guess the former is better, but it's just to be sure .. (or maybe it's >>>>> better to iSCSI export a ZVOL from the SLAVE?) >>>>> >>>> >>>> are you really sure you understand what you trying to do? even if its >>>> currently so, i bet in a desaster case you will be lost. >>>> >>>> >>> >>> well this is pretty new to me, but I don't see what could be wrong with: >>> >>> $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 >>> /dev/da1 >>> >>> Let's take some use-cases: >>> - MASTER and SLAVE are alive, the data is "replicated" on both >>> nodes. As iSCSI is used, ZFS will see all the details of the >>> underlying disks and we can be sure that no corruption will occur >>> (contrary to HAST) >>> - SLAVE die, correct me if I'm wrong the but pool is still available, >>> fix the SLAVE, resilver and that's it ..? >>> - MASTER die, CARP will notice it and SLAVE will take the VIP, the >>> failover script will be executed with a $> zpool import -f >>> >>>>> Correct me if I'm wrong but, from a safety point of view this setup is >>>>> also the safest as you'll get the "fullsync" equivalent mode of HAST >>>>> (but but it's also the slowest), so I can be 99,99% confident that the >>>>> pool on the SLAVE will never be corrupted, even in the case where the >>>>> MASTER suddently die (power outage, etc), and that a zpool import -f >>>>> storage will always work? >>>> >>>> 99,99% ? optimistic, very optimistic. >>> >>> the only situation where corruption could occur is some sort of network >>> corruption (bug in the driver, broken network card, etc), or a bug in >>> ZFS ... but you'll have the same with a zfs send|ssh zfs receive >>> >>>> >> >> optimistic >> >>>> we are playing with recovery of a test pool which has been imported on >>>> two nodes at the same time. looks pretty messy >>>> >>>>> >>>>> One last thing: this "storage" pool will be exported through NFS on the >>>>> clients, and when a failover occur they should, in theory, not notice >>>>> it. I know that it's pretty hypothetical but I wondered if pfsync could >>>>> play a role in this area (active connections)..? >>>>> >>>> >>>> they will notice, and they will stuck or worse (reboot) >>> >>> this is something that should be properly tested I agree.. >>> >> >> do your testing, and keep your clients under load while testing. do >> writes onto the nfs mounts and then cut. you will be surprised about the >> impact. >> >>>> >> >> >> >>>>> Thanks! >>>>> Julien >>>>> >>>>>> >>>>>>>>>> ZFS would then know as soon as a disk is failing. >>>>>>>>>> And if the master fails, you only have to import (-f certainly, in case of a master power failure) on the slave. >>>>>>>>>> >>>>>>>>>> Ben >>>>>> _______________________________________________ >>>>>> freebsd-fs@freebsd.org mailing list >>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>>>> >>> >> > From owner-freebsd-fs@freebsd.org Fri Jul 1 11:39:14 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E6F0B8E002 for ; Fri, 1 Jul 2016 11:39:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E0F62FC6 for ; Fri, 1 Jul 2016 11:39:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u61BdEuF021799 for ; Fri, 1 Jul 2016 11:39:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203864] ZFS deadlock between zfs send, zfs rename and ctrl-C Date: Fri, 01 Jul 2016 11:39:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: dogfood, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: roel@qsp.nl X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 11:39:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203864 --- Comment #9 from roel@qsp.nl --- Yes, I just introduced a short sleep in the script as a workaround. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jul 1 11:40:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEC8AB8E089 for ; Fri, 1 Jul 2016 11:40:15 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F980206C for ; Fri, 1 Jul 2016 11:40:15 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B4BB028450; Fri, 1 Jul 2016 13:40:12 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id E331A28458; Fri, 1 Jul 2016 13:40:11 +0200 (CEST) Message-ID: <5776569B.3050504@quip.cz> Date: Fri, 01 Jul 2016 13:40:11 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Julien Cigar CC: InterNetX - Juergen Gotteswinter , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> In-Reply-To: <20160701105735.GG5695@mordor.lan> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 11:40:15 -0000 Julien Cigar wrote on 07/01/2016 12:57: >>> why...? I guess iSCSI is slower but should be safer than HAST, no? >> >> do your testing, please. even with simulated short network cuts. 10-20 >> secs are way enaugh to give you a picture of what is going to happen > > of course I'll test everything properly :) I don't have the hardware yet > so ATM I'm just looking for all the possible "candidates", and I'm > aware that a redundant storage is not that easy to implement ... > > but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), > either zfs send|ssh zfs receive as you suggest (but it's > not realtime), either a distributed FS (which I avoid like the plague..) When disaster comes you will need to restart NFS clients in almost all cases (with CARP + ZFS + HAST|iSCSI) and you will lose some writes too. And if something bad happens with your mgmt scripts or network you can end up with corrupted ZFS pool on master and slave too - you will need to recovery from backups. For example in some split brain scenario when both nodes will try to import pool. With ZFS send & receive you will lose some writes but the chance you will corrupt both pools are much lower than in the first case and the setup is much simpler and runtime error proof. I rather prefer some downtime with safe data than shorter downtime but data in risk. YMMV Miroslav Lachman From owner-freebsd-fs@freebsd.org Fri Jul 1 11:58:46 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 492B9B8E9C0 for ; Fri, 1 Jul 2016 11:58:46 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE744287C for ; Fri, 1 Jul 2016 11:58:45 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22f.google.com with SMTP id v199so23639624wmv.0 for ; Fri, 01 Jul 2016 04:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=hKuGzqdo64rbu4luDd1331+OCPqVltngLBtk74XI72M=; b=TvTR9Q/oI/Qjp55Vngs3QGjOWGeVyV2E0QBNlFixXPmVbZMP9kaxvQZaPlBcCqfAaN o52ZmRAL2BwpACqNVhgiIVQcOuPY2xKXHyJ+sVq2tFGP3lXgWyGqBsj3veRJJyh4E0AX j2ro23Di0IPaA1LFyk+bk4+O0UrKak+xc8pEcj9ck3VqddmrhnHTgknjpDKi2nejnPu2 7BulKOFAlasKyM19zuPr4w95ziIwlJiOUrvxjRMZGpBixxZLEqrwiCIleI32xI3pt+oh qMxpxj+zSvW1+DqsBApojyULVY/hw5Y2OYnon75VxrT82xsx0YF7kIVuJy/AfjmTULsS Dprg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=hKuGzqdo64rbu4luDd1331+OCPqVltngLBtk74XI72M=; b=T+LSmkoXiMdE4C6xtXBlyfMki6DOtvlmBwGm6ewY7/2v2zmGRp4fXx6aa5LlQKGhzF s4vdpCP6d9BZYyEc8SSX236ftHp/juIXdBPARxohrmNwv6XInsbKyh8oCOA5itiSrs/W qWl2E6guhI+608S0P8177dh/q8TvBRN90Y7WlXPsbkA/y9uMeT11d0++OE0Hl8E9X+uo HezAN8pENC8AwlIBprzRD6vW4/AnxvLXg1GRJvXjZ2Kvitb067kU9KOu8IoCPrmVz3O2 bf3yZQXiN384xNjByG9AFVYRiFROuJcfhPijfRArFzv8tD5whxiiYe9AiZMi5UntQlkp 0BGQ== X-Gm-Message-State: ALyK8tJ3R779LJDEJjyI20Ahl2C9e1hXq2Es8u2ELd/aFQ+DAwfvptPM03L5Dr3QsB0vpg== X-Received: by 10.28.17.132 with SMTP id 126mr32796392wmr.90.1467374324029; Fri, 01 Jul 2016 04:58:44 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id u4sm2865318wjz.4.2016.07.01.04.58.43 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Jul 2016 04:58:43 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: <5776569B.3050504@quip.cz> Date: Fri, 1 Jul 2016 13:58:42 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <5776569B.3050504@quip.cz> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 11:58:46 -0000 > On 01 Jul 2016, at 13:40, Miroslav Lachman <000.fbsd@quip.cz> wrote: >=20 > Julien Cigar wrote on 07/01/2016 12:57: >=20 >>>> why...? I guess iSCSI is slower but should be safer than HAST, no? >>>=20 >>> do your testing, please. even with simulated short network cuts. = 10-20 >>> secs are way enaugh to give you a picture of what is going to happen >>=20 >> of course I'll test everything properly :) I don't have the hardware = yet >> so ATM I'm just looking for all the possible "candidates", and I'm >> aware that a redundant storage is not that easy to implement ... >>=20 >> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), >> either zfs send|ssh zfs receive as you suggest (but it's >> not realtime), either a distributed FS (which I avoid like the = plague..) >=20 > When disaster comes you will need to restart NFS clients in almost all = cases (with CARP + ZFS + HAST|iSCSI) and you will lose some writes too. > And if something bad happens with your mgmt scripts or network you can = end up with corrupted ZFS pool on master and slave too - you will need = to recovery from backups. For example in some split brain scenario when = both nodes will try to import pool. Of course you must take care that both nodes do not import the pool at = the same time. For the slave to import the pool, first stop iSCSI targets (ctld), and = also put network replication interface down, to be sure. Then, import the pool. Once old master repaired, export its pool (if still imported), make its = disks iSCSI targets and give them the old slave (promoted master just = above). Of course it implies some meticulous administration. > With ZFS send & receive you will lose some writes but the chance you = will corrupt both pools are much lower than in the first case and the = setup is much simpler and runtime error proof. Only some ? Depending on the write throughput, won't you loose a lot of data on the = target/slave ? How do you make ZFS send/receive quite realtime ? while [ 1 ] do ; snapshot ; send/receive ; delete old snapshots ; done ? Thanks != From owner-freebsd-fs@freebsd.org Fri Jul 1 12:17:10 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70DC7B86A14 for ; Fri, 1 Jul 2016 12:17:10 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 318712679 for ; Fri, 1 Jul 2016 12:17:09 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 40C4B28450; Fri, 1 Jul 2016 14:17:07 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 7CBB028416; Fri, 1 Jul 2016 14:17:06 +0200 (CEST) Message-ID: <57765F42.4090904@quip.cz> Date: Fri, 01 Jul 2016 14:17:06 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Ben RUBSON , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <5776569B.3050504@quip.cz> <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> In-Reply-To: <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 12:17:10 -0000 Ben RUBSON wrote on 07/01/2016 13:58: >> With ZFS send & receive you will lose some writes but the chance you will corrupt both pools are much lower than in the first case and the setup is much simpler and runtime error proof. > > Only some ? > Depending on the write throughput, won't you loose a lot of data on the target/slave ? > How do you make ZFS send/receive quite realtime ? > while [ 1 ] do ; snapshot ; send/receive ; delete old snapshots ; done ? It depends on throughput and how often do you send. But you need to compare it to the HAST / iSCSI scenario. Even with this setup ZFS don't write to disk immediately but in batch delayed according to some sysctl settings and you will lose this amount of data in all cases + data on clients which cannot write and must restart their NFS sessions (again this apply to HAST / iSCSI scenario) If you will ZFS send often, you can lose about 2 or 4 times more. It depends on you if it is too much or not. Miroslav Lachman From owner-freebsd-fs@freebsd.org Fri Jul 1 12:31:32 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9B1DB870A1 for ; Fri, 1 Jul 2016 12:31:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7A5F2D60 for ; Fri, 1 Jul 2016 12:31:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u61CVWd6057887 for ; Fri, 1 Jul 2016 12:31:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 140068] [smbfs] [patch] smbfs does not allow semicolon in filenames, but should Date: Fri, 01 Jul 2016 12:31:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: easy, needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_severity bug_status keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 12:31:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D140068 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|Affects Only Me |Affects Some People Status|In Progress |Open Keywords| |easy, needs-qa, patch --- Comment #2 from Kubilay Kocak --- Can't be In Progress without an Assignee. Open to take --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jul 1 12:33:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C241EB87120 for ; Fri, 1 Jul 2016 12:33:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B1E102F97 for ; Fri, 1 Jul 2016 12:33:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u61CXR9G099122 for ; Fri, 1 Jul 2016 12:33:27 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 140068] [smbfs] [patch] smbfs does not allow semicolon in filenames, but should Date: Fri, 01 Jul 2016 12:33:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: easy, needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 12:33:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D140068 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |mfc-stable10? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jul 1 12:38:11 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93B99B87398 for ; Fri, 1 Jul 2016 12:38:11 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b02.edpnet.be (relay-b02.edpnet.be [212.71.1.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44DA0220A for ; Fri, 1 Jul 2016 12:38:10 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467376680-0a7b8d12103d727b0001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b02.edpnet.be with ESMTP id GIrk1I5C0l7xrKMp (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 14:38:01 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 14:38:00 +0200 From: Julien Cigar To: Ben RUBSON Cc: freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701123800.GA41276@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <5776569B.3050504@quip.cz> <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467376680 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.222:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2961 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30922 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 12:38:11 -0000 --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 01:58:42PM +0200, Ben RUBSON wrote: >=20 > > On 01 Jul 2016, at 13:40, Miroslav Lachman <000.fbsd@quip.cz> wrote: > >=20 > > Julien Cigar wrote on 07/01/2016 12:57: > >=20 > >>>> why...? I guess iSCSI is slower but should be safer than HAST, no? > >>>=20 > >>> do your testing, please. even with simulated short network cuts. 10-20 > >>> secs are way enaugh to give you a picture of what is going to happen > >>=20 > >> of course I'll test everything properly :) I don't have the hardware y= et > >> so ATM I'm just looking for all the possible "candidates", and I'm > >> aware that a redundant storage is not that easy to implement ... > >>=20 > >> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), > >> either zfs send|ssh zfs receive as you suggest (but it's > >> not realtime), either a distributed FS (which I avoid like the plague.= =2E) > >=20 > > When disaster comes you will need to restart NFS clients in almost all = cases (with CARP + ZFS + HAST|iSCSI) and you will lose some writes too. > > And if something bad happens with your mgmt scripts or network you can = end up with corrupted ZFS pool on master and slave too - you will need to r= ecovery from backups. For example in some split brain scenario when both no= des will try to import pool. >=20 > Of course you must take care that both nodes do not import the pool at th= e same time. > For the slave to import the pool, first stop iSCSI targets (ctld), and al= so put network replication interface down, to be sure. > Then, import the pool. > Once old master repaired, export its pool (if still imported), make its d= isks iSCSI targets and give them the old slave (promoted master just above). > Of course it implies some meticulous administration. I was thinking something like this also.. and I definitively think that=20 the switch from old save (promoted master) to "old master repaired" MUST be done manually! >=20 > > With ZFS send & receive you will lose some writes but the chance you wi= ll corrupt both pools are much lower than in the first case and the setup i= s much simpler and runtime error proof. I think losing some writes is somewhat unavoidable, corruption on the other hand is unacceptable >=20 > Only some ? > Depending on the write throughput, won't you loose a lot of data on the t= arget/slave ? > How do you make ZFS send/receive quite realtime ? > while [ 1 ] do ; snapshot ; send/receive ; delete old snapshots ; done ? >=20 > Thanks ! > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --wRRV7LY7NUeQGEoC Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdmQkAAoJELK7NxCiBCPAK2oQAOhKqx3mmjsfU7CPpQqym8zY NYMwPXKPOCYPZOe8jiq10j26JTdWgnjR+mXgEjuf8TioH7Xwk+O18w4i8P3X6XlX gvpxF2smrhf0ujV/roQh/8MUW3zl6EbC/MQl65pZPotiMFCeWzi6CUcFPXXBr8/9 GxRU9TTQMET3z/wAED/iFvehJIIHu97u+imRp3hmEIAJuWJBqdU9NkucQSsLjmWA 1PYUHjm9VVtvazrDG59UGSRp7vmQMf3v0vhMkTz1iFR2U1lHekySp5xfPQ2NvFar PaUGjN7oEsHYv0vLI5v3urfMirCa/WSIbvQM1xNuLFSitNwAzJCV1bK/0ORXwkca VjZ8x+dt0LIFUOFtQ1A32dtebJMSDZC/XWOBgyFv4hysUUhtcs1cUK510UzMDPmA lTz101Fs2mGTHlMXohTLHMV9vqDPBiCBw/b35APIy9be0ggtErHySCsha2cWIPU8 xTSAO2Sojn0JuzLIj04Qwfy9UGHtmj4PKF2XaRlQhdcLiOfQCFevJKo7c8PqqOhc q1fDmpVd/fRU7OQW3O4rvM5e5oCN/+yCUY6mBoBcTJaQvKT/Ayu/NYLptnN+Dh15 8t78F18hYJdf+2B2ZeB4XiLJU94vWjd2W7sc6IZS+jHEX2JmQpo2O7+4l4LzHyKP tUnBLu0eIaWdRkS1KlIU =RNKE -----END PGP SIGNATURE----- --wRRV7LY7NUeQGEoC-- From owner-freebsd-fs@freebsd.org Fri Jul 1 12:40:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CADC9B87451 for ; Fri, 1 Jul 2016 12:40:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 738D62446 for ; Fri, 1 Jul 2016 12:40:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:BBfnxB9DBLsRvP9uRHKM819IXTAuvvDOBiVQ1KB91eIcTK2v8tzYMVDF4r011RmSDN2dsqMP1rKempujcFRI2YyGvnEGfc4EfD4+ouJSoTYdBtWYA1bwNv/gYn9yNs1DUFh44yPzahANS47AblHf6ke/8SQVUk2mc1EkfqKsS8WP14ye7KObw9XreQJGhT6wM/tZDS6dikHvjPQQmpZoMa0ryxHE8TNicuVSwn50dxrIx06vru/5xpNo8jxRtvQ97IYAFPyiJ+VrBYBfWR4rNSgP2efQkj+LGQGC4D0GT28NlRxgDA3M7RW8VZD05HjUrO14jRObNs6+aLk/WjCv6u8/UhrhgyQDOjsR7WbYl8F0lKIdqxv39E83+JLdfIzAbKk2RajaZ95PADMZBss= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DaBACqY3ZX/61jaINdhBQtTwa7RSKFLEoCgXQQAQEBAQEBAQFkJ4IyghoBAQQBAQEhBCcfAQsFCwIBCBgCAg0ZAgInAQkmAgQIBwQBGgIEiAcIDrQKkBsBAQEBAQEBAQIBAQEBAQEBGwWBAYUnhE2ECRoBAQUCgxWCWgWIEIcjiV2GCYU4hG2EVoMuhTyQBwI1H4IIHIFoIDIHhyoOFx9/AQEB X-IronPort-AV: E=Sophos;i="5.26,556,1459828800"; d="scan'208";a="292830461" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 01 Jul 2016 08:39:43 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 58EF915F587; Fri, 1 Jul 2016 08:39:43 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id VXzMSJjboBnU; Fri, 1 Jul 2016 08:39:42 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 4409615F59B; Fri, 1 Jul 2016 08:39:42 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ijMOkja6yn3y; Fri, 1 Jul 2016 08:39:42 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 29C7715F587; Fri, 1 Jul 2016 08:39:42 -0400 (EDT) Date: Fri, 1 Jul 2016 08:39:41 -0400 (EDT) From: Rick Macklem To: Ben RUBSON Cc: freebsd-fs@freebsd.org Message-ID: <1963819198.192910387.1467376781932.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <5776569B.3050504@quip.cz> <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> Subject: Re: HAST + ZFS + NFS + CARP MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF47 (Win)/8.0.9_GA_6191) Thread-Topic: HAST + ZFS + NFS + CARP Thread-Index: egxN84KtccXh4b/epO4z3k4HdZD2hA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 12:40:53 -0000 Choosing the most recent post in the thread (Ben RUBSON): First off, this topic is of interest to me because a pNFS server would need a similar fail-over for a metadata server. Having said that, I know little about this and am learning from what I've read (ie. keep up the good work, folks;-). I will put a few NFS related comments inline. > > > On 01 Jul 2016, at 13:40, Miroslav Lachman <000.fbsd@quip.cz> wrote: > > > > Julien Cigar wrote on 07/01/2016 12:57: > > > >>>> why...? I guess iSCSI is slower but should be safer than HAST, no? > >>> > >>> do your testing, please. even with simulated short network cuts. 10-20 > >>> secs are way enaugh to give you a picture of what is going to happen > >> > >> of course I'll test everything properly :) I don't have the hardware yet > >> so ATM I'm just looking for all the possible "candidates", and I'm > >> aware that a redundant storage is not that easy to implement ... > >> > >> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), > >> either zfs send|ssh zfs receive as you suggest (but it's > >> not realtime), either a distributed FS (which I avoid like the plague..) > > > > When disaster comes you will need to restart NFS clients in almost all > > cases (with CARP + ZFS + HAST|iSCSI) and you will lose some writes too. > > And if something bad happens with your mgmt scripts or network you can end > > up with corrupted ZFS pool on master and slave too - you will need to > > recovery from backups. For example in some split brain scenario when both > > nodes will try to import pool. > What the clients need to have happen is their TCP connection fail, so they must do a new TCP connection. At that point, they will retry any outstanding RPCs, including ones that did UNSTABLE writes but there was no subsequent Commit RPC done successfully. I know nothing about CARP (have never used it), but I have a hunch it will not do the above at the correct point in time. In general, I would consider failing over from the NFS server to the backup one (where the drives have been kept up to date via a ZFS mirror using iSCSI) is a major event. I don't think you want to do this for short outages. (Server overload, server crash/reboot, etc.) I think it is hard to distinguish between a slow server and a failed one and you don't want this switchover happening unless it is really needed, imho. When it happens, I think you want a strict ordering of events like: - old master server shut down (off the network or at least moved to different IP addresses so it appears off the network to the clients). - an orderly switchover of the ZFS file system, so that the backup is now the only system handling the file system (the paragraph just below here covers that, I think?) - new server (was backup) comes up with the same IP address(es) as the old one had before the failure. --> I am not sure if anything has to be done to speed up ARP cache invalidation for the old server's IP->MAC mapping. (I'm not sure what you do w.r.t. mirroring for the backup server. Ideally there would be a way for a third server to have mirrored disks resilvered from the backup server's. I don't know enough about ZFS to know if this could be done with a third server, where its empty disks are set up as iSCSI mirrors or the backup server's after the failover? I think this is also covered by the para. below.) > Of course you must take care that both nodes do not import the pool at the > same time. > For the slave to import the pool, first stop iSCSI targets (ctld), and also > put network replication interface down, to be sure. > Then, import the pool. > Once old master repaired, export its pool (if still imported), make its disks > iSCSI targets and give them the old slave (promoted master just above). > Of course it implies some meticulous administration. > Yes. I don't think CARP can be trusted to do this. The problem is that there is a time ordering w.r.t. activities on the two servers. However, you can't assume that the two servers can communicate with each other. The simpler/reliable way would be done manually be a sysadmin (who might also know why and how long the master will be down for). It might be possible to automate this with daemons on the two servers, with something like: - master sends regular heartbeat messages to backup. - if master gets no NFS client activity for N seconds, it sends "shutdown message to backup" and then does a full shutdown. - if slave gets shutdown message from master OR doesn't see a heartbeat message for something like 2 * N seconds, then it assumes it can take over and does so. --> The trick is, the backup can't start a takeover until the master really is offline. > > With ZFS send & receive you will lose some writes but the chance you will > > corrupt both pools are much lower than in the first case and the setup is > > much simpler and runtime error proof. > > Only some ? > Depending on the write throughput, won't you loose a lot of data on the > target/slave ? > How do you make ZFS send/receive quite realtime ? > while [ 1 ] do ; snapshot ; send/receive ; delete old snapshots ; done ? > Well, if the NFS clients aren't buggy and the server isn't running sync=disabled, then nothing should get lost if the clients recognize that the server has "crashed/rebooted". This happens when the TCP connection to the server breaks. You can't have a case where the client TCP connection to the server keeps functioning but actually goes to the backup server and you can't have the case where some clients are still doing NFS RPCs to the old master while others are doing RPCs to the backup. Good luck with it. It is an interesting challenge and worth exploring. rick > Thanks ! > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Jul 1 13:18:16 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1235B87BC8 for ; Fri, 1 Jul 2016 13:18:16 +0000 (UTC) (envelope-from joe@getsomewhere.net) Received: from prak.gameowls.com (prak.gameowls.com [IPv6:2001:19f0:5c00:950b:5400:ff:fe14:46b7]) by mx1.freebsd.org (Postfix) with ESMTP id 60A812456 for ; Fri, 1 Jul 2016 13:18:16 +0000 (UTC) (envelope-from joe@getsomewhere.net) Received: from [IPv6:2001:470:c412:beef:135:c8df:2d0e:4ea6] (unknown [IPv6:2001:470:c412:beef:135:c8df:2d0e:4ea6]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by prak.gameowls.com (Postfix) with ESMTPSA id 32489183FC for ; Fri, 1 Jul 2016 08:18:15 -0500 (CDT) From: Joe Love Message-Id: <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP Date: Fri, 1 Jul 2016 08:18:14 -0500 References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> To: freebsd-fs@freebsd.org In-Reply-To: <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> X-Mailer: Apple Mail (2.3124) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 13:18:16 -0000 > On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter = wrote: >=20 > Am 01.07.2016 um 12:57 schrieb Julien Cigar: >> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen = Gotteswinter wrote: >>=20 >> of course I'll test everything properly :) I don't have the hardware = yet >> so ATM I'm just looking for all the possible "candidates", and I'm=20 >> aware that a redundant storage is not that easy to implement ... >>=20 >> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI),=20= >> either zfs send|ssh zfs receive as you suggest (but it's >> not realtime), either a distributed FS (which I avoid like the = plague..) >=20 > zfs send/receive can be nearly realtime. >=20 > external jbods with cross cabled sas + commercial cluster solution = like > rsf-1. anything else is a fragile construction which begs for = desaster. This sounds similar to the CTL-HA code that went in last year, for which = I haven=E2=80=99t seen any sort of how-to. The RSF-1 stuff sounds like = it has more scaling options, though. Which it probably should, given = its commercial operation. -Joe From owner-freebsd-fs@freebsd.org Fri Jul 1 13:44:44 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9698B88557 for ; Fri, 1 Jul 2016 13:44:44 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6B0C520F0 for ; Fri, 1 Jul 2016 13:44:44 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 2C67645FC0E5; Fri, 1 Jul 2016 15:44:41 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y63CvrDQ8ijk; Fri, 1 Jul 2016 15:44:37 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 3E4184C4C87E; Fri, 1 Jul 2016 15:44:37 +0200 (CEST) Subject: Re: HAST + ZFS + NFS + CARP References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> To: Joe Love , freebsd-fs@freebsd.org Reply-To: jg@internetx.com From: InterNetX - Juergen Gotteswinter Message-ID: Date: Fri, 1 Jul 2016 15:44:36 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 13:44:44 -0000 Am 01.07.2016 um 15:18 schrieb Joe Love: > >> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: >> >> Am 01.07.2016 um 12:57 schrieb Julien Cigar: >>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotteswinter wrote: >>> >>> of course I'll test everything properly :) I don't have the hardware yet >>> so ATM I'm just looking for all the possible "candidates", and I'm >>> aware that a redundant storage is not that easy to implement ... >>> >>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), >>> either zfs send|ssh zfs receive as you suggest (but it's >>> not realtime), either a distributed FS (which I avoid like the plague..) >> >> zfs send/receive can be nearly realtime. >> >> external jbods with cross cabled sas + commercial cluster solution like >> rsf-1. anything else is a fragile construction which begs for desaster. > > This sounds similar to the CTL-HA code that went in last year, for which I haven’t seen any sort of how-to. The RSF-1 stuff sounds like it has more scaling options, though. Which it probably should, given its commercial operation. rsf is what pacemaker / heartbeat tries to be, judge me for linking whitepapers but in this case its not such evil marketing blah http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-PLUGIN-ZFS-STORAGE-CLUSTER.pdf @ Julien seems like you take availability really serious, so i guess you also got plans how to accomplish network problems like dead switches, flaky cables and so on. like using multiple network cards in the boxes, cross cabling between the hosts (rs232 and ethernet of course, using proved reliable network switches in a stacked configuration for example cisco 3750 stacked). not to forget redundant power feeds to redundant power supplies. if not, i whould start again from scratch. > > -Joe > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Jul 1 14:02:58 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48836B889C2 for ; Fri, 1 Jul 2016 14:02:58 +0000 (UTC) (envelope-from lists.zxinn@otaking.se) Received: from smtprelay-b12.telenor.se (smtprelay-b12.telenor.se [62.127.194.21]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE0FB2900 for ; Fri, 1 Jul 2016 14:02:57 +0000 (UTC) (envelope-from lists.zxinn@otaking.se) Received: from ipb4.telenor.se (ipb4.telenor.se [195.54.127.167]) by smtprelay-b12.telenor.se (Postfix) with ESMTP id AC657C4D2 for ; Fri, 1 Jul 2016 15:44:00 +0200 (CEST) X-SENDER-IP: [213.112.54.213] X-LISTENER: [smtp.bredband.net] X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BvJwBAc3ZXENU2cNVdKYMVVjUoH6RfBQYBBZZbEhCHJj0QAQEBAQEBAQYBAQEBAQEBAT5AhQsCHBgfegwKNYgsAQmkLaAiBYVgiiGFDwEEhgWTC4Ihg2iFbIJIgVsBFYFGgxCIapAHAjWBYoIvOjIBiHMBAQE X-IPAS-Result: A2BvJwBAc3ZXENU2cNVdKYMVVjUoH6RfBQYBBZZbEhCHJj0QAQEBAQEBAQYBAQEBAQEBAT5AhQsCHBgfegwKNYgsAQmkLaAiBYVgiiGFDwEEhgWTC4Ihg2iFbIJIgVsBFYFGgxCIapAHAjWBYoIvOjIBiHMBAQE X-IronPort-AV: E=Sophos;i="5.26,557,1459807200"; d="scan'208";a="415033542" Received: from c-d53670d5.06-5-73746f28.cust.bredbandsbolaget.se (HELO mail.otaking.se) ([213.112.54.213]) by ipb4.telenor.se with ESMTP; 01 Jul 2016 15:44:00 +0200 Received: from mail.otaking.se (localhost [127.0.0.1]) by mail.otaking.se (Postfix) with ESMTP id 6C986EB287 for ; Fri, 1 Jul 2016 15:43:58 +0200 (CEST) X-Virus-Scanned: amavisd-new at otaking.se Received: from mail.otaking.se ([127.0.0.1]) by mail.otaking.se (mail.otaking.se [127.0.0.1]) (amavisd-new, port 10587) with ESMTP id 9t4fZoV8_3Pl for ; Fri, 1 Jul 2016 15:43:55 +0200 (CEST) Received: from mail.otaking.se (localhost [127.0.0.1]) by mail.otaking.se (Postfix) with ESMTP id C2E9FEB285 for ; Fri, 1 Jul 2016 15:43:55 +0200 (CEST) Received: from mail.otaking.se (otaking-web [10.11.11.112]) (Authenticated sender: kai@otaking.se) by mail.otaking.se (Postfix) with ESMTPSA id A0F1DEB284 for ; Fri, 1 Jul 2016 15:43:55 +0200 (CEST) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 01 Jul 2016 15:43:55 +0200 From: Tobias To: freebsd-fs@freebsd.org Subject: smbfs bug 140068 open since 7 years, please revisit Message-ID: <7cee78250c9c5ec531747efd0e243c4a@otaking.se> X-Sender: lists.zxinn@otaking.se User-Agent: Roundcube Webmail/1.1.5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 14:02:58 -0000 Hi, I'd like to bring to your attention bug 140068, which is open since 2009-October and not yet corrected. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=140068 Please revisit this bug report, and include the correction in FBSD 10.3. :) I posted on the FreeBSD forums and they suggested I notify this mailing list. https://forums.freebsd.org/threads/56792/ BR Tobias From owner-freebsd-fs@freebsd.org Fri Jul 1 14:39:23 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6256BB8E467 for ; Fri, 1 Jul 2016 14:39:23 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b03.edpnet.be (relay-b03.edpnet.be [212.71.1.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 12BED2F92 for ; Fri, 1 Jul 2016 14:39:22 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467383957-0a88181ce6677820001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b03.edpnet.be with ESMTP id 7rkGYjFHQ8qeil4q (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 16:39:18 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 16:39:17 +0200 From: Julien Cigar To: InterNetX - Juergen Gotteswinter Cc: Joe Love , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701143917.GB41276@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="mxv5cy4qt+RJ9ypb" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467383957 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.220:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2942 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0000 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=6.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30923 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 14:39:23 -0000 --mxv5cy4qt+RJ9ypb Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen Gotteswinter = wrote: >=20 >=20 > Am 01.07.2016 um 15:18 schrieb Joe Love: > >=20 > >> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: > >> > >> Am 01.07.2016 um 12:57 schrieb Julien Cigar: > >>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gottesw= inter wrote: > >>> > >>> of course I'll test everything properly :) I don't have the hardware = yet > >>> so ATM I'm just looking for all the possible "candidates", and I'm=20 > >>> aware that a redundant storage is not that easy to implement ... > >>> > >>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI),= =20 > >>> either zfs send|ssh zfs receive as you suggest (but it's > >>> not realtime), either a distributed FS (which I avoid like the plague= =2E.) > >> > >> zfs send/receive can be nearly realtime. > >> > >> external jbods with cross cabled sas + commercial cluster solution like > >> rsf-1. anything else is a fragile construction which begs for desaster. > >=20 > > This sounds similar to the CTL-HA code that went in last year, for whic= h I haven=E2=80=99t seen any sort of how-to. The RSF-1 stuff sounds like i= t has more scaling options, though. Which it probably should, given its co= mmercial operation. >=20 > rsf is what pacemaker / heartbeat tries to be, judge me for linking > whitepapers but in this case its not such evil marketing blah >=20 > http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-PLUG= IN-ZFS-STORAGE-CLUSTER.pdf >=20 >=20 > @ Julien >=20 > seems like you take availability really serious, so i guess you also got > plans how to accomplish network problems like dead switches, flaky > cables and so on. >=20 > like using multiple network cards in the boxes, cross cabling between > the hosts (rs232 and ethernet of course, using proved reliable network > switches in a stacked configuration for example cisco 3750 stacked). not > to forget redundant power feeds to redundant power supplies. the only thing that is not redundant (yet?) is our switch, an HP Pro=20 Curve 2530-24G) .. it's the next step :) >=20 > if not, i whould start again from scratch. >=20 > >=20 > > -Joe > >=20 > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --mxv5cy4qt+RJ9ypb Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdoCRAAoJELK7NxCiBCPATMoQAN5O8PPvvyN2S03MbK8e0Ax0 5QHW9gkECXeQ7UqMbu8HVo4IQIh+od4YhrNrLnTn59Ouv2B95D/tbRO6rsrASEGg 4yIoLUEsDS2B/XzAtOlBkTcwn8/mawrjBG9VjVZrTGHVEGUPX1bvfItDsioT0eRb 0t36vDadycjw+igfs77cnxvEW/SqnNboXlrgfQb1KIpxncpQ6xc0JiqdnmsBtscB ZBd593ADpsO+wjRj1DYeEJOibLdTnhrFwRDLTtcAH0PdM1EKk0BCrEdRwxUdNUKF YVR7f0ZUWv0gNRkePqg7Ag8Wbcbn/Uwc9vl9gDVrkxyXZQ5LU0hwrEAU7BlPPbra FCn6yyfW6r00boskOoNvg5hMOzdO5lzE38Yv9TTs0Y41qSL9q1Hyy+jqjfPI2XvJ U/34dbtuTejWQ/Xno3RHel4OWqKAfLgebhXwIrQHtFFgROnRvFi2gBb80A6bbSO7 Nv1hjFm3+HayLB4sGI2r1+Nq6YN3VZc/NHCoW+xQ3Y2p65fH/k0VAi5iNuIV660n fxTzWo7R8FYN79u7WsjSoCzjBhVHiDJRwtmcB/ijClzbLIafKtGfIOpilrpL63PM hDcg+6MUonkUeweJt0RUjjmdvJA8U+K2u9PvUkNdEaWjeGpe+1Gb3GlF9DeCDprC hKiGTl2UQ9qkZDf9iSL0 =z/8C -----END PGP SIGNATURE----- --mxv5cy4qt+RJ9ypb-- From owner-freebsd-fs@freebsd.org Fri Jul 1 14:42:08 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B054B8E5C5 for ; Fri, 1 Jul 2016 14:42:08 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id A38242221 for ; Fri, 1 Jul 2016 14:42:07 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id AB7F145FC0EB; Fri, 1 Jul 2016 16:42:05 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Yzae51YOY0vm; Fri, 1 Jul 2016 16:42:00 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 060AB4C4C881; Fri, 1 Jul 2016 16:42:00 +0200 (CEST) Reply-To: jg@internetx.com Subject: Re: HAST + ZFS + NFS + CARP References: <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> <20160701143917.GB41276@mordor.lan> To: Julien Cigar Cc: Joe Love , freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Message-ID: <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> Date: Fri, 1 Jul 2016 16:41:59 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160701143917.GB41276@mordor.lan> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 14:42:08 -0000 Am 01.07.2016 um 16:39 schrieb Julien Cigar: > On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen Gotteswinter wrote: >> >> >> Am 01.07.2016 um 15:18 schrieb Joe Love: >>> >>>> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: >>>> >>>> Am 01.07.2016 um 12:57 schrieb Julien Cigar: >>>>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotteswinter wrote: >>>>> >>>>> of course I'll test everything properly :) I don't have the hardware yet >>>>> so ATM I'm just looking for all the possible "candidates", and I'm >>>>> aware that a redundant storage is not that easy to implement ... >>>>> >>>>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), >>>>> either zfs send|ssh zfs receive as you suggest (but it's >>>>> not realtime), either a distributed FS (which I avoid like the plague..) >>>> >>>> zfs send/receive can be nearly realtime. >>>> >>>> external jbods with cross cabled sas + commercial cluster solution like >>>> rsf-1. anything else is a fragile construction which begs for desaster. >>> >>> This sounds similar to the CTL-HA code that went in last year, for which I haven’t seen any sort of how-to. The RSF-1 stuff sounds like it has more scaling options, though. Which it probably should, given its commercial operation. >> >> rsf is what pacemaker / heartbeat tries to be, judge me for linking >> whitepapers but in this case its not such evil marketing blah >> >> http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-PLUGIN-ZFS-STORAGE-CLUSTER.pdf >> >> >> @ Julien >> >> seems like you take availability really serious, so i guess you also got >> plans how to accomplish network problems like dead switches, flaky >> cables and so on. >> >> like using multiple network cards in the boxes, cross cabling between >> the hosts (rs232 and ethernet of course, using proved reliable network >> switches in a stacked configuration for example cisco 3750 stacked). not >> to forget redundant power feeds to redundant power supplies. > > the only thing that is not redundant (yet?) is our switch, an HP Pro > Curve 2530-24G) .. it's the next step :) Arubas, okay, a quick view in the spec sheet does not seem to list stacking option. what about power? > >> >> if not, i whould start again from scratch. >> >>> >>> -Joe >>> >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Jul 1 14:44:36 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1083BB8E673 for ; Fri, 1 Jul 2016 14:44:36 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 97BB22466 for ; Fri, 1 Jul 2016 14:44:35 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 8B32A4C4C882; Fri, 1 Jul 2016 16:44:33 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FqOm5iTibXMe; Fri, 1 Jul 2016 16:44:24 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id E327A4C4C881; Fri, 1 Jul 2016 16:44:24 +0200 (CEST) Reply-To: jg@internetx.com, jg@internetx.com Subject: Re: HAST + ZFS + NFS + CARP References: <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> <20160701143917.GB41276@mordor.lan> <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> To: Julien Cigar Cc: freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Message-ID: <4d13f123-de18-693a-f98b-d02c8864f02e@internetx.com> Date: Fri, 1 Jul 2016 16:44:24 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 14:44:36 -0000 dont get me wrong, what i try to say is that imho you are trying to reach something which looks great until something goes wrong. keep it simple, stupid simple, without much moving parts and avoid automagic voodoo wherever possible. Am 01.07.2016 um 16:41 schrieb InterNetX - Juergen Gotteswinter: > Am 01.07.2016 um 16:39 schrieb Julien Cigar: >> On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen Gotteswinter wrote: >>> >>> >>> Am 01.07.2016 um 15:18 schrieb Joe Love: >>>> >>>>> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: >>>>> >>>>> Am 01.07.2016 um 12:57 schrieb Julien Cigar: >>>>>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotteswinter wrote: >>>>>> >>>>>> of course I'll test everything properly :) I don't have the hardware yet >>>>>> so ATM I'm just looking for all the possible "candidates", and I'm >>>>>> aware that a redundant storage is not that easy to implement ... >>>>>> >>>>>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), >>>>>> either zfs send|ssh zfs receive as you suggest (but it's >>>>>> not realtime), either a distributed FS (which I avoid like the plague..) >>>>> >>>>> zfs send/receive can be nearly realtime. >>>>> >>>>> external jbods with cross cabled sas + commercial cluster solution like >>>>> rsf-1. anything else is a fragile construction which begs for desaster. >>>> >>>> This sounds similar to the CTL-HA code that went in last year, for which I haven’t seen any sort of how-to. The RSF-1 stuff sounds like it has more scaling options, though. Which it probably should, given its commercial operation. >>> >>> rsf is what pacemaker / heartbeat tries to be, judge me for linking >>> whitepapers but in this case its not such evil marketing blah >>> >>> http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-PLUGIN-ZFS-STORAGE-CLUSTER.pdf >>> >>> >>> @ Julien >>> >>> seems like you take availability really serious, so i guess you also got >>> plans how to accomplish network problems like dead switches, flaky >>> cables and so on. >>> >>> like using multiple network cards in the boxes, cross cabling between >>> the hosts (rs232 and ethernet of course, using proved reliable network >>> switches in a stacked configuration for example cisco 3750 stacked). not >>> to forget redundant power feeds to redundant power supplies. >> >> the only thing that is not redundant (yet?) is our switch, an HP Pro >> Curve 2530-24G) .. it's the next step :) > > Arubas, okay, a quick view in the spec sheet does not seem to list > stacking option. > > what about power? > >> >>> >>> if not, i whould start again from scratch. >>> >>>> >>>> -Joe >>>> >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>>> >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Jul 1 15:02:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C6A9B8ED61 for ; Fri, 1 Jul 2016 15:02:30 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 902302CFD for ; Fri, 1 Jul 2016 15:02:29 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22c.google.com with SMTP id f126so30137970wma.1 for ; Fri, 01 Jul 2016 08:02:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=Yi0xbIPfxZzwToNcO62ei1Tb1jROk4RR4IRvEMb/Eyk=; b=cdjQMki7oG0lw0P8/0dt78hJl0vhfkAmEGshGvH6FH3L+aHvDFkwS9DxtD+XJzJq2k A5Ozh/P/rih7RexVIpdq6PKpEEkO24e1eqmUfUtzzkmwZQ/t9zHqHku3kDILHnXbsv0X 5tFSTga47L4IgEAYGMB5Y4TVnvVacj97MWN0SPnxUBlmzQ97Ltn8to4JaTpn8K7NFJWt /BBdlZEObwNlPd5BMufO+FedJUtY+cbxGFoE9ELu/xBmwpAJqqzre2uEppMIDQqczfpT VtQLJbfIIVfa1CJOYfPLs57KpTIwLotWNjblnFhQ83BlHtIwJicIp8Rbo8LyBidS0Ty3 5apg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=Yi0xbIPfxZzwToNcO62ei1Tb1jROk4RR4IRvEMb/Eyk=; b=Q8inYi2j2LUW/VCQcVCQSPS6fPH+bb59YBt/rl7t+o1uO5+5Q3PN9vigq4k14hDKAd sR37TKVWQ6vb59Y+vQkohC72KGfIafzNjC5shHxTP3fwZ2w0EDY/9FsrCAt8jxri7hTO VqQTNYQBOcOwhSaR6/qIBFQ+eKhgQuG2dVIOt4JU3hKQLkNdvm1X087Ev/GFq6VfD72l YUQqo7b+3cHMNQ1PJIax8+JraYx8shWwEnW+UK5HIKURsR4KNDJUidMTM1coy4FtLJRK x34Tmo0X+SSNPtyQzaWWeSwQccdBEJMabxMXBxuc+fGA6k/OfcL0exJaQcdfyDM9Y26b Ug8A== X-Gm-Message-State: ALyK8tIJgVhEJ9L4LuanWzefMnYT4GsRK5Bd+4OItA5+VMxYxl4gp+4J4359dBcGwDnuKA== X-Received: by 10.194.165.138 with SMTP id yy10mr4595041wjb.92.1467385347810; Fri, 01 Jul 2016 08:02:27 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id kc8sm2481396wjb.0.2016.07.01.08.02.26 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Jul 2016 08:02:27 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: <4d13f123-de18-693a-f98b-d02c8864f02e@internetx.com> Date: Fri, 1 Jul 2016 17:02:25 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> <20160701143917.GB41276@mordor.lan> <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> <4d13f123-de18-693a-f98b-d02c8864f02e@internetx.com> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 15:02:30 -0000 I think what we miss is some kind of this : http://milek.blogspot.fr/2007/03/zfs-online-replication.html http://www.compnect.net/?p=3D16461 Online replication built in ZFS would be awesome. > On 01 Jul 2016, at 16:44, InterNetX - Juergen Gotteswinter = wrote: >=20 > dont get me wrong, what i try to say is that imho you are trying to > reach something which looks great until something goes wrong. >=20 > keep it simple, stupid simple, without much moving parts and avoid > automagic voodoo wherever possible. >=20 > Am 01.07.2016 um 16:41 schrieb InterNetX - Juergen Gotteswinter: >> Am 01.07.2016 um 16:39 schrieb Julien Cigar: >>> On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen = Gotteswinter wrote: >>>>=20 >>>>=20 >>>> Am 01.07.2016 um 15:18 schrieb Joe Love: >>>>>=20 >>>>>> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter = wrote: >>>>>>=20 >>>>>> Am 01.07.2016 um 12:57 schrieb Julien Cigar: >>>>>>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen = Gotteswinter wrote: >>>>>>>=20 >>>>>>> of course I'll test everything properly :) I don't have the = hardware yet >>>>>>> so ATM I'm just looking for all the possible "candidates", and = I'm=20 >>>>>>> aware that a redundant storage is not that easy to implement ... >>>>>>>=20 >>>>>>> but what solutions do we have? It's either CARP + ZFS + = (HAST|iSCSI),=20 >>>>>>> either zfs send|ssh zfs receive as you suggest (but it's >>>>>>> not realtime), either a distributed FS (which I avoid like the = plague..) >>>>>>=20 >>>>>> zfs send/receive can be nearly realtime. >>>>>>=20 >>>>>> external jbods with cross cabled sas + commercial cluster = solution like >>>>>> rsf-1. anything else is a fragile construction which begs for = desaster. >>>>>=20 >>>>> This sounds similar to the CTL-HA code that went in last year, for = which I haven=E2=80=99t seen any sort of how-to. The RSF-1 stuff sounds = like it has more scaling options, though. Which it probably should, = given its commercial operation. >>>>=20 >>>> rsf is what pacemaker / heartbeat tries to be, judge me for linking >>>> whitepapers but in this case its not such evil marketing blah >>>>=20 >>>> = http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-PLUGI= N-ZFS-STORAGE-CLUSTER.pdf >>>>=20 >>>>=20 >>>> @ Julien >>>>=20 >>>> seems like you take availability really serious, so i guess you = also got >>>> plans how to accomplish network problems like dead switches, flaky >>>> cables and so on. >>>>=20 >>>> like using multiple network cards in the boxes, cross cabling = between >>>> the hosts (rs232 and ethernet of course, using proved reliable = network >>>> switches in a stacked configuration for example cisco 3750 = stacked). not >>>> to forget redundant power feeds to redundant power supplies. >>>=20 >>> the only thing that is not redundant (yet?) is our switch, an HP Pro=20= >>> Curve 2530-24G) .. it's the next step :) >>=20 >> Arubas, okay, a quick view in the spec sheet does not seem to list >> stacking option. >>=20 >> what about power? >>=20 >>>=20 >>>>=20 >>>> if not, i whould start again from scratch. >>>>=20 >>>>>=20 >>>>> -Joe >>>>>=20 >>>>> _______________________________________________ >>>>> freebsd-fs@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >>>>>=20 >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >>>=20 >>=20 >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri Jul 1 15:03:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF6AFB8EDCA for ; Fri, 1 Jul 2016 15:03:41 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b01.edpnet.be (relay-b01.edpnet.be [212.71.1.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 916432DAE for ; Fri, 1 Jul 2016 15:03:41 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467385416-0a7ff569f61048c20001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b01.edpnet.be with ESMTP id m4TEWcbog6lFwjev (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 17:03:37 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 17:03:36 +0200 From: Julien Cigar To: InterNetX - Juergen Gotteswinter Cc: Joe Love , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701150336.GC41276@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> <20160701143917.GB41276@mordor.lan> <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="+xNpyl7Qekk2NvDX" Content-Disposition: inline In-Reply-To: <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467385416 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.221:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 3988 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=6.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30924 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 15:03:42 -0000 --+xNpyl7Qekk2NvDX Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 04:41:59PM +0200, InterNetX - Juergen Gotteswinter = wrote: > Am 01.07.2016 um 16:39 schrieb Julien Cigar: > > On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen Gotteswin= ter wrote: > >> > >> > >> Am 01.07.2016 um 15:18 schrieb Joe Love: > >>> > >>>> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: > >>>> > >>>> Am 01.07.2016 um 12:57 schrieb Julien Cigar: > >>>>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotte= swinter wrote: > >>>>> > >>>>> of course I'll test everything properly :) I don't have the hardwar= e yet > >>>>> so ATM I'm just looking for all the possible "candidates", and I'm= =20 > >>>>> aware that a redundant storage is not that easy to implement ... > >>>>> > >>>>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI= ),=20 > >>>>> either zfs send|ssh zfs receive as you suggest (but it's > >>>>> not realtime), either a distributed FS (which I avoid like the plag= ue..) > >>>> > >>>> zfs send/receive can be nearly realtime. > >>>> > >>>> external jbods with cross cabled sas + commercial cluster solution l= ike > >>>> rsf-1. anything else is a fragile construction which begs for desast= er. > >>> > >>> This sounds similar to the CTL-HA code that went in last year, for wh= ich I haven=E2=80=99t seen any sort of how-to. The RSF-1 stuff sounds like= it has more scaling options, though. Which it probably should, given its = commercial operation. > >> > >> rsf is what pacemaker / heartbeat tries to be, judge me for linking > >> whitepapers but in this case its not such evil marketing blah > >> > >> http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-P= LUGIN-ZFS-STORAGE-CLUSTER.pdf > >> > >> > >> @ Julien > >> > >> seems like you take availability really serious, so i guess you also g= ot > >> plans how to accomplish network problems like dead switches, flaky > >> cables and so on. > >> > >> like using multiple network cards in the boxes, cross cabling between > >> the hosts (rs232 and ethernet of course, using proved reliable network > >> switches in a stacked configuration for example cisco 3750 stacked). n= ot > >> to forget redundant power feeds to redundant power supplies. > >=20 > > the only thing that is not redundant (yet?) is our switch, an HP Pro=20 > > Curve 2530-24G) .. it's the next step :) >=20 > Arubas, okay, a quick view in the spec sheet does not seem to list > stacking option. >=20 > what about power? there is a "diesel generator" for the server room, and redundant power supply on "most critical" servers (our PostgreSQL servers for example).=20 Router and Firewall (Soekris 6501) runs CARP / PFSync, same for=20 the load balancer (HAProxy), local Unbound, etc (Everything is jailed and managed by SaltStack, so in worst case scenario I could always redeploy "something" (service, webapp, etc) in minutes on $somemachine ..) No, the really SPOF that should be fixed quickly ATM is the shared files=20 storage, it's a NFS mount on a single machine (with redundant power=20 supply) and if the hardware die we're in troubles (...) >=20 > >=20 > >> > >> if not, i whould start again from scratch. > >> > >>> > >>> -Joe > >>> > >>> _______________________________________________ > >>> freebsd-fs@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >>> > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >=20 >=20 --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --+xNpyl7Qekk2NvDX Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdoZEAAoJELK7NxCiBCPAOO4QAOpHnQ/QIsIMS7+JZpGqF39A ZIUduicxuT/THwFzthPUObqlBUCBvXcnsEobzrR9iXAP/o8ysR+7guvA3+5KL1T8 enP58wb+ZLzCdOyB7zrdNAoL3pE7lpiYN5YgeqwpK0z7+57pCevFM37sgvSIqoE3 b5Zv+2Uc/C+uzEV2AkOgwM7F8zSLlVsIfZ79pEaEX8wWhihruXrme4QOvD/wGMnw vX1DdS+OKyV+NxeBpOTMUmdCg0ybYh7B8wxX44L2mGtbG0QaxFkbeAYo+ue0sBfn 6X9SZfE9wVB6ezYRUHYlq5pPcbtkwvUQQyJNI3RnA36Ow6SMrfhNoPiF48RB0d72 xSGvsT95rbBNLPkAxKLju+d13k54+YHdNT6KNxkzVxxFsZt5lq4eiVrxlGyRcUnK OpeU0TjCJP06koMB838yXMkfwtWmZMzVuIy0SKxTmww9CmxUqmC1CCBMOOSbRTU0 3OAaUQB+Xhza20d9zjlKm8ZrU0q6BTEQqSUnCMEkbEq5bq42r/9N+HsQqiIoLEcj BYgUGWzVVastPOb/NOggE5W5PNBG7M9cUm2Ylw+J4mcnxz2nKHKf+aB5y4X+jxvh r7Sq5HjTiMPzlunNlEk/CsBbX3NCp5C7AJGiVSGVORpFEtQRi5lcud6WLYjfttp2 AYUq6if4i/qS8JcXpuy/ =LsG7 -----END PGP SIGNATURE----- --+xNpyl7Qekk2NvDX-- From owner-freebsd-fs@freebsd.org Fri Jul 1 15:11:57 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96C75B8E235 for ; Fri, 1 Jul 2016 15:11:57 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b01.edpnet.be (relay-b01.edpnet.be [212.71.1.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 448722406 for ; Fri, 1 Jul 2016 15:11:57 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467385907-0a7ff569f8104b300001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b01.edpnet.be with ESMTP id A8hj2PRFDq2Cifmt (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 17:11:49 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 17:11:47 +0200 From: Julien Cigar To: jg@internetx.com Cc: freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701151146.GD41276@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> <20160701143917.GB41276@mordor.lan> <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> <4d13f123-de18-693a-f98b-d02c8864f02e@internetx.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="9UV9rz0O2dU/yYYn" Content-Disposition: inline In-Reply-To: <4d13f123-de18-693a-f98b-d02c8864f02e@internetx.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467385907 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.221:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 4218 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=6.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30924 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 15:11:57 -0000 --9UV9rz0O2dU/yYYn Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 04:44:24PM +0200, InterNetX - Juergen Gotteswinter = wrote: > dont get me wrong, what i try to say is that imho you are trying to > reach something which looks great until something goes wrong. I agree..! :) >=20 > keep it simple, stupid simple, without much moving parts and avoid > automagic voodoo wherever possible. >=20 to be honnest I've always been relunctant to "automatic failover", as I think the problem is always not "how" to do it but "when".. and as Rick said "The simpler/reliable way would be done manually be a sysadmin".. > Am 01.07.2016 um 16:41 schrieb InterNetX - Juergen Gotteswinter: > > Am 01.07.2016 um 16:39 schrieb Julien Cigar: > >> On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen Gotteswi= nter wrote: > >>> > >>> > >>> Am 01.07.2016 um 15:18 schrieb Joe Love: > >>>> > >>>>> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: > >>>>> > >>>>> Am 01.07.2016 um 12:57 schrieb Julien Cigar: > >>>>>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gott= eswinter wrote: > >>>>>> > >>>>>> of course I'll test everything properly :) I don't have the hardwa= re yet > >>>>>> so ATM I'm just looking for all the possible "candidates", and I'm= =20 > >>>>>> aware that a redundant storage is not that easy to implement ... > >>>>>> > >>>>>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCS= I),=20 > >>>>>> either zfs send|ssh zfs receive as you suggest (but it's > >>>>>> not realtime), either a distributed FS (which I avoid like the pla= gue..) > >>>>> > >>>>> zfs send/receive can be nearly realtime. > >>>>> > >>>>> external jbods with cross cabled sas + commercial cluster solution = like > >>>>> rsf-1. anything else is a fragile construction which begs for desas= ter. > >>>> > >>>> This sounds similar to the CTL-HA code that went in last year, for w= hich I haven=E2=80=99t seen any sort of how-to. The RSF-1 stuff sounds lik= e it has more scaling options, though. Which it probably should, given its= commercial operation. > >>> > >>> rsf is what pacemaker / heartbeat tries to be, judge me for linking > >>> whitepapers but in this case its not such evil marketing blah > >>> > >>> http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-= PLUGIN-ZFS-STORAGE-CLUSTER.pdf > >>> > >>> > >>> @ Julien > >>> > >>> seems like you take availability really serious, so i guess you also = got > >>> plans how to accomplish network problems like dead switches, flaky > >>> cables and so on. > >>> > >>> like using multiple network cards in the boxes, cross cabling between > >>> the hosts (rs232 and ethernet of course, using proved reliable network > >>> switches in a stacked configuration for example cisco 3750 stacked). = not > >>> to forget redundant power feeds to redundant power supplies. > >> > >> the only thing that is not redundant (yet?) is our switch, an HP Pro= =20 > >> Curve 2530-24G) .. it's the next step :) > >=20 > > Arubas, okay, a quick view in the spec sheet does not seem to list > > stacking option. > >=20 > > what about power? > >=20 > >> > >>> > >>> if not, i whould start again from scratch. > >>> > >>>> > >>>> -Joe > >>>> > >>>> _______________________________________________ > >>>> freebsd-fs@freebsd.org mailing list > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >>>> > >>> _______________________________________________ > >>> freebsd-fs@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >> > >=20 > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >=20 --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --9UV9rz0O2dU/yYYn Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdogyAAoJELK7NxCiBCPAT+MP/isQyDWA4tjDW8UZCsQ0QJmt gghjcskRnKsSqbGcEiS1lT/hY4rGIlErYG2XpKCjUsF9VxVnfd+ILtn4VB720vpd 3LAqutUR5VSkeYeFYNBlwUH07rauig6TXcLf1N8XSKaR+ppdCz8ff1GDtM52/nWM SirI8Au5SojToFb7QJnPoo4smlWwml48dSQ9ErEF9nR9h4YgPuGIdA+G4vS8jy+t zkGkLfmyQwl6BZfhRBW9xz2u2n1EvRGjnt0mObPL/3vulzA9yA1AKzZq0Zl28B47 w3GdkTPTgYSyikU0lYzG+vvnLKLgXkIN98Z8i+BLYWucleJOhsDmHwzfcYv/M6R+ SNiRHWt0ogQ5YdrNJf8mkSiR1uuQGc8OYbfdHj0jEBGH/rL+sCaE0k3UmebNYXN9 ZCGHhjTQH1n60H+iE+JcU051iDI8zNoKUlSNyOjoNC4f9VyyoqqpIDm9tJ7of0ab su5A/JOh3fxLg7xtvIBTC5ZnJLrdikopKQeBc6qKGDkLHRrqzLaVOqnv+2t6orta mxbFyfyjTrdcrosNzk3VEjdGS/AS+0KA722mcPkmCklSnyMWjfwwoLxAnLNwD2tG NeDwoCPHDbR3nkD5R3FFoaAjfvm26OIyifXkmJmBEv+45swhsyZ996NaNZyiKcAq +vfPMAnF7dhabhTHLpuQ =Ia4F -----END PGP SIGNATURE----- --9UV9rz0O2dU/yYYn-- From owner-freebsd-fs@freebsd.org Fri Jul 1 15:40:10 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77607B8E94D for ; Fri, 1 Jul 2016 15:40:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 673AF20EE for ; Fri, 1 Jul 2016 15:40:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u61Fe9S2073277 for ; Fri, 1 Jul 2016 15:40:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203864] ZFS deadlock between zfs send, zfs rename and ctrl-C Date: Fri, 01 Jul 2016 15:40:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: dogfood, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 15:40:10 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203864 --- Comment #10 from Cassiano Peixoto --- (In reply to roel from comment #9) Hi Roel, Can you send me the patch to my email address? i'd like to add it. I think would be a good idea to submit a patch to zrep port.=20 The worst thing with this issue is: when it happens i have to force server reboot and when it comes back i lost all my data since the beginning of iss= ue. It's unbelievable, i know, but it's happening. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jul 1 15:47:02 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7548EB8EB06 for ; Fri, 1 Jul 2016 15:47:02 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 31E152572 for ; Fri, 1 Jul 2016 15:47:02 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bJ0eh-000PuH-2T; Fri, 01 Jul 2016 16:46:59 +0100 Date: Fri, 1 Jul 2016 16:46:58 +0100 From: Gary Palmer To: Julien Cigar Cc: jg@internetx.com, freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701154658.GA70150@in-addr.com> References: <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> <20160701143917.GB41276@mordor.lan> <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> <4d13f123-de18-693a-f98b-d02c8864f02e@internetx.com> <20160701151146.GD41276@mordor.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160701151146.GD41276@mordor.lan> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 15:47:02 -0000 On Fri, Jul 01, 2016 at 05:11:47PM +0200, Julien Cigar wrote: > On Fri, Jul 01, 2016 at 04:44:24PM +0200, InterNetX - Juergen Gotteswinter wrote: > > dont get me wrong, what i try to say is that imho you are trying to > > reach something which looks great until something goes wrong. > > I agree..! :) > > > > > keep it simple, stupid simple, without much moving parts and avoid > > automagic voodoo wherever possible. > > > > to be honnest I've always been relunctant to "automatic failover", as I > think the problem is always not "how" to do it but "when".. and as Rick > said "The simpler/reliable way would be done manually be a sysadmin".. I agree. They can verify that the situation needs a fail over much better than any script. In a previous job I heard of a setup where the cluster manager software on the standby node decided that the active node was down so it did a force takeover of the disks. Since the active node was still up it somehow managed to wipe out the partition tables on the disks along with the vxvm configuration (Veritas Volume Manager) inside the partitions. They were restoring the partition tables and vxvm config from backups. >From what I remember the backups were printouts, which made it slow going as they had to be re-entered by hand. The system probably had dozens of disks (I don't know, but I know what role it was serving so I can guess) I'd rather not see that happen ever again (this was 15+ years ago FWIW, but the lesson is still applicable today) Gary > > > Am 01.07.2016 um 16:41 schrieb InterNetX - Juergen Gotteswinter: > > > Am 01.07.2016 um 16:39 schrieb Julien Cigar: > > >> On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen Gotteswinter wrote: > > >>> > > >>> > > >>> Am 01.07.2016 um 15:18 schrieb Joe Love: > > >>>> > > >>>>> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: > > >>>>> > > >>>>> Am 01.07.2016 um 12:57 schrieb Julien Cigar: > > >>>>>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotteswinter wrote: > > >>>>>> > > >>>>>> of course I'll test everything properly :) I don't have the hardware yet > > >>>>>> so ATM I'm just looking for all the possible "candidates", and I'm > > >>>>>> aware that a redundant storage is not that easy to implement ... > > >>>>>> > > >>>>>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), > > >>>>>> either zfs send|ssh zfs receive as you suggest (but it's > > >>>>>> not realtime), either a distributed FS (which I avoid like the plague..) > > >>>>> > > >>>>> zfs send/receive can be nearly realtime. > > >>>>> > > >>>>> external jbods with cross cabled sas + commercial cluster solution like > > >>>>> rsf-1. anything else is a fragile construction which begs for desaster. > > >>>> > > >>>> This sounds similar to the CTL-HA code that went in last year, for which I haven???t seen any sort of how-to. The RSF-1 stuff sounds like it has more scaling options, though. Which it probably should, given its commercial operation. > > >>> > > >>> rsf is what pacemaker / heartbeat tries to be, judge me for linking > > >>> whitepapers but in this case its not such evil marketing blah > > >>> > > >>> http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-PLUGIN-ZFS-STORAGE-CLUSTER.pdf > > >>> > > >>> > > >>> @ Julien > > >>> > > >>> seems like you take availability really serious, so i guess you also got > > >>> plans how to accomplish network problems like dead switches, flaky > > >>> cables and so on. > > >>> > > >>> like using multiple network cards in the boxes, cross cabling between > > >>> the hosts (rs232 and ethernet of course, using proved reliable network > > >>> switches in a stacked configuration for example cisco 3750 stacked). not > > >>> to forget redundant power feeds to redundant power supplies. > > >> > > >> the only thing that is not redundant (yet?) is our switch, an HP Pro > > >> Curve 2530-24G) .. it's the next step :) > > > > > > Arubas, okay, a quick view in the spec sheet does not seem to list > > > stacking option. > > > > > > what about power? > > > > > >> > > >>> > > >>> if not, i whould start again from scratch. > > >>> > > >>>> > > >>>> -Joe > > >>>> > > >>>> _______________________________________________ > > >>>> freebsd-fs@freebsd.org mailing list > > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > >>>> > > >>> _______________________________________________ > > >>> freebsd-fs@freebsd.org mailing list > > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > >> > > > > > > _______________________________________________ > > > freebsd-fs@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > > > -- > Julien Cigar > Belgian Biodiversity Platform (http://www.biodiversity.be) > PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 > No trees were killed in the creation of this message. > However, many electrons were terribly inconvenienced. From owner-freebsd-fs@freebsd.org Fri Jul 1 16:07:40 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC810B8F0DD for ; Fri, 1 Jul 2016 16:07:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C5A02D90 for ; Fri, 1 Jul 2016 16:07:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u61G7dbI068869 for ; Fri, 1 Jul 2016 16:07:40 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 203864] ZFS deadlock between zfs send, zfs rename and ctrl-C Date: Fri, 01 Jul 2016 16:07:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: dogfood, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: roel@qsp.nl X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 16:07:40 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203864 --- Comment #11 from roel@qsp.nl --- That's not that hard: *** /home/roel/zrep Wed Jan 13 22:12:27 2016 --- zrep Sun Feb 21 13:24:35 2016 *************** *** 1160,1164 **** if [[ $? -ne 0 ]] ; then zfs rename ${newsnap} ${newsnap}_unsent zrep_errquit Problem doing sync for $newsnap. renamed to ${newsnap}_unsent fi --- if [[ $? -ne 0 ]] ; then + echo "Error on send.. waiting 60 seconds to prevent deadloc= k" + sleep 60 zfs rename ${newsnap} ${newsnap}_unsent zrep_errquit Problem doing sync for $newsnap. renamed to ${newsnap}_unsent fi --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Jul 1 16:55:02 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2D9FB8FE10 for ; Fri, 1 Jul 2016 16:55:02 +0000 (UTC) (envelope-from bsdunix44@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 981A72C03; Fri, 1 Jul 2016 16:55:02 +0000 (UTC) (envelope-from bsdunix44@gmail.com) Received: by mail-it0-x236.google.com with SMTP id f6so21261790ith.0; Fri, 01 Jul 2016 09:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-transfer-encoding :message-id:cc:from:subject:date:to; bh=5dXRWQu/cVf64ay9o+j2fFSrPEU/eVZtsKQ3wZ49En0=; b=SndLyFTIl4QfubcgLneNQVE0vf8oYZeFn2V1nvB97da3x1O+LkupNd4oSEEBtbkhSr 4J47acLi0KTZ38tUERbKqMGGi7mWYxjLlamCw3UEvN0FLbyGbwYtzUx0EAz3/owAcHP5 +yQR+43ktRfpUKlFcO6kqGXlRwko0awJnu4LbieKqRJR4Vw/FQ0PxrcJTwlSF94Br51n fmSXUPSS1kq9l5zTgpvq9NVayoOk0EawGQxT+GM+IKqmTG/ysw1VYs3LIKGP5bXSDCZ7 1IXCv17Ffa2bFSC77eqgnC6/TkZjUQHHxU78Bmg53O0kvf6aFPZP7/jy1Thg7L50Wqsw M99w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=5dXRWQu/cVf64ay9o+j2fFSrPEU/eVZtsKQ3wZ49En0=; b=Gnszg7D0m1eTaPdKuPa242naR6Pw4YFjXYjMTznKbEpNYTkP+fpwW22la4GVjn/DSH Xz+EHDfHJYZ/oVTqeijAwezScPhSbim0pqnnaduI/19jddwHaBMQui+lfTkGXjk+KOBl bAgCWR4uAy5i9Ep9QtV5zs2yLnzhZutXijP6fOQdncCdrI46/qlYfKhyZEY45OycwOSv MuhfMjNoc7SrhEjAl9E4PwJahMeV+pM8MNGpM36LQ4xiv99XMdeRlVEU1ONwGEbI/Qxx 3cp/afJDh6qnxuN0ZF357GWVwkDY6qbLlvVjlIAGjGJZJQnCqEb3kNQ2Be3vq7OA+zrT IHNw== X-Gm-Message-State: ALyK8tJCCaJH4Ip5FzKHIfKvgu39w14Po4Azc4SvFkBLYmMq9tvK2Z2SX7XmL3t9coW7cw== X-Received: by 10.36.14.77 with SMTP id 74mr551153ite.98.1467392101801; Fri, 01 Jul 2016 09:55:01 -0700 (PDT) Received: from [100.93.55.189] ([172.56.13.148]) by smtp.gmail.com with ESMTPSA id d126sm6366721iog.20.2016.07.01.09.54.59 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Jul 2016 09:55:00 -0700 (PDT) References: <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <3d8c7c89-b24e-9810-f3c2-11ec1e15c948@internetx.com> <93E50E6B-8248-43B5-BE94-D94D53050E06@getsomewhere.net> <20160701143917.GB41276@mordor.lan> <01b8a61e-739e-c41e-45bc-a84af0a9d8ab@internetx.com> <4d13f123-de18-693a-f98b-d02c8864f02e@internetx.com> <20160701151146.GD41276@mordor.lan> <20160701154658.GA70150@in-addr.com> Mime-Version: 1.0 (1.0) In-Reply-To: <20160701154658.GA70150@in-addr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <7D9B465F-0248-4F6D-895D-7BC3684EC6F7@gmail.com> Cc: Julien Cigar , freebsd-fs@freebsd.org X-Mailer: iPhone Mail (13F69) From: Chris Watson Subject: Re: HAST + ZFS + NFS + CARP Date: Fri, 1 Jul 2016 11:54:58 -0500 To: Gary Palmer X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 16:55:02 -0000 Hi Gary! So I'll add another voice to the KISS camp. I'd rather have two boxes each w= ith two NICs attached to each other doing zfs replication from A to B. Addin= g more redundant hardware just adds more points of failure. NICs have no mov= ing parts so as long as they are thermally controlled they won't fail. This i= s simple and as safe as you can get. As for how to handle an actual failover= is really like to try out the ctl-ha option. Maybe this weekend.=20 Sent from my iPhone 5 > On Jul 1, 2016, at 10:46 AM, Gary Palmer wrote: >=20 >> On Fri, Jul 01, 2016 at 05:11:47PM +0200, Julien Cigar wrote: >>> On Fri, Jul 01, 2016 at 04:44:24PM +0200, InterNetX - Juergen Gotteswint= er wrote: >>> dont get me wrong, what i try to say is that imho you are trying to >>> reach something which looks great until something goes wrong. >>=20 >> I agree..! :) >>=20 >>>=20 >>> keep it simple, stupid simple, without much moving parts and avoid >>> automagic voodoo wherever possible. >>=20 >> to be honnest I've always been relunctant to "automatic failover", as I >> think the problem is always not "how" to do it but "when".. and as Rick >> said "The simpler/reliable way would be done manually be a sysadmin".. >=20 > I agree. They can verify that the situation needs a fail over much better= > than any script. In a previous job I heard of a setup where the cluster > manager software on the standby node decided that the active node was > down so it did a force takeover of the disks. Since the active node was > still up it somehow managed to wipe out the partition tables on the disks > along with the vxvm configuration (Veritas Volume Manager) inside the > partitions. >=20 > They were restoring the partition tables and vxvm config from backups. > =46rom what I remember the backups were printouts, which made it slow goin= g > as they had to be re-entered by hand. The system probably had dozens > of disks (I don't know, but I know what role it was serving so I can > guess) >=20 > I'd rather not see that happen ever again >=20 > (this was 15+ years ago FWIW, but the lesson is still applicable today) >=20 > Gary >=20 >>=20 >>>> Am 01.07.2016 um 16:41 schrieb InterNetX - Juergen Gotteswinter: >>>>> Am 01.07.2016 um 16:39 schrieb Julien Cigar: >>>>>> On Fri, Jul 01, 2016 at 03:44:36PM +0200, InterNetX - Juergen Gottesw= inter wrote: >>>>>>=20 >>>>>>=20 >>>>>>> Am 01.07.2016 um 15:18 schrieb Joe Love: >>>>>>>=20 >>>>>>>> On Jul 1, 2016, at 6:09 AM, InterNetX - Juergen Gotteswinter wrote: >>>>>>>>=20 >>>>>>>> Am 01.07.2016 um 12:57 schrieb Julien Cigar: >>>>>>>>> On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gott= eswinter wrote: >>>>>>>>>=20 >>>>>>>>> of course I'll test everything properly :) I don't have the hardwa= re yet >>>>>>>>> so ATM I'm just looking for all the possible "candidates", and I'm= =20 >>>>>>>>> aware that a redundant storage is not that easy to implement ... >>>>>>>>>=20 >>>>>>>>> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCS= I),=20 >>>>>>>>> either zfs send|ssh zfs receive as you suggest (but it's >>>>>>>>> not realtime), either a distributed FS (which I avoid like the pla= gue..) >>>>>>>>=20 >>>>>>>> zfs send/receive can be nearly realtime. >>>>>>>>=20 >>>>>>>> external jbods with cross cabled sas + commercial cluster solution l= ike >>>>>>>> rsf-1. anything else is a fragile construction which begs for desas= ter. >>>>>>>=20 >>>>>>> This sounds similar to the CTL-HA code that went in last year, for w= hich I haven???t seen any sort of how-to. The RSF-1 stuff sounds like it ha= s more scaling options, though. Which it probably should, given its commerc= ial operation. >>>>>>=20 >>>>>> rsf is what pacemaker / heartbeat tries to be, judge me for linking >>>>>> whitepapers but in this case its not such evil marketing blah >>>>>>=20 >>>>>> http://www.high-availability.com/wp-content/uploads/2013/01/RSF-1-HA-= PLUGIN-ZFS-STORAGE-CLUSTER.pdf >>>>>>=20 >>>>>>=20 >>>>>> @ Julien >>>>>>=20 >>>>>> seems like you take availability really serious, so i guess you also g= ot >>>>>> plans how to accomplish network problems like dead switches, flaky >>>>>> cables and so on. >>>>>>=20 >>>>>> like using multiple network cards in the boxes, cross cabling between= >>>>>> the hosts (rs232 and ethernet of course, using proved reliable networ= k >>>>>> switches in a stacked configuration for example cisco 3750 stacked). n= ot >>>>>> to forget redundant power feeds to redundant power supplies. >>>>>=20 >>>>> the only thing that is not redundant (yet?) is our switch, an HP Pro=20= >>>>> Curve 2530-24G) .. it's the next step :) >>>>=20 >>>> Arubas, okay, a quick view in the spec sheet does not seem to list >>>> stacking option. >>>>=20 >>>> what about power? >>>>=20 >>>>>=20 >>>>>>=20 >>>>>> if not, i whould start again from scratch. >>>>>>=20 >>>>>>>=20 >>>>>>> -Joe >>>>>>>=20 >>>>>>> _______________________________________________ >>>>>>> freebsd-fs@freebsd.org mailing list >>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org= " >>>>>> _______________________________________________ >>>>>> freebsd-fs@freebsd.org mailing list >>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"= >>>>=20 >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>=20 >> --=20 >> Julien Cigar >> Belgian Biodiversity Platform (http://www.biodiversity.be) >> PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 >> No trees were killed in the creation of this message. >> However, many electrons were terribly inconvenienced. >=20 >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri Jul 1 17:54:30 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC919B8EEA5 for ; Fri, 1 Jul 2016 17:54:30 +0000 (UTC) (envelope-from jkh@ixsystems.com) Received: from barracuda.ixsystems.com (barracuda.ixsystems.com [12.229.62.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AF32B2B91 for ; Fri, 1 Jul 2016 17:54:30 +0000 (UTC) (envelope-from jkh@ixsystems.com) X-ASG-Debug-ID: 1467395669-08ca0410fd27910001-3nHGF7 Received: from zimbra.ixsystems.com ([10.246.0.20]) by barracuda.ixsystems.com with ESMTP id yy6nPg2NMc0XjXCx (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 10:54:29 -0700 (PDT) X-Barracuda-Envelope-From: jkh@ixsystems.com X-Barracuda-RBL-Trusted-Forwarder: 10.246.0.20 X-ASG-Whitelist: Client Received: from localhost (localhost [127.0.0.1]) by zimbra.ixsystems.com (Postfix) with ESMTP id 985B9C81853; Fri, 1 Jul 2016 10:54:29 -0700 (PDT) Received: from zimbra.ixsystems.com ([127.0.0.1]) by localhost (zimbra.ixsystems.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 4ouYqCRZXAUe; Fri, 1 Jul 2016 10:54:28 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.ixsystems.com (Postfix) with ESMTP id C08D2C81B10; Fri, 1 Jul 2016 10:54:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at ixsystems.com Received: from zimbra.ixsystems.com ([127.0.0.1]) by localhost (zimbra.ixsystems.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id M-0RIxkVzxFg; Fri, 1 Jul 2016 10:54:28 -0700 (PDT) Received: from [172.20.0.34] (vpn.ixsystems.com [10.249.0.2]) by zimbra.ixsystems.com (Postfix) with ESMTPSA id 917DCC81853; Fri, 1 Jul 2016 10:54:28 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Jordan Hubbard X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP In-Reply-To: <20160630185701.GD5695@mordor.lan> Date: Fri, 1 Jul 2016 10:54:27 -0700 Cc: Chris Watson , freebsd-fs@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <678321AB-A9F7-4890-A8C7-E20DFDC69137@gmail.com> <20160630185701.GD5695@mordor.lan> To: Julien Cigar X-Mailer: Apple Mail (2.3124) X-Barracuda-Connect: UNKNOWN[10.246.0.20] X-Barracuda-Start-Time: 1467395669 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://10.246.0.26:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ixsystems.com X-Barracuda-BRTS-Status: 1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 17:54:30 -0000 > On Jun 30, 2016, at 11:57 AM, Julien Cigar = wrote: >=20 > It would be more than welcome indeed..! I have the feeling that HAST > isn't that much used (but maybe I am wrong) and it's difficult to find=20= > informations on it's reliability and concrete long-term use cases... This has been a long discussion so I=E2=80=99m not even sure where the = right place to jump in is, but just speaking as a storage vendor = (FreeNAS) I=E2=80=99ll say that we=E2=80=99ve considered HAST many times = but also rejected it many times for multiple reasons: 1. Blocks which are found to be corrupt by ZFS (fail checksum) get = replicated by HAST nonetheless since it has no idea - it=E2=80=99s below = that layer. This means that both good data and corrupt data are = replicated to the other pool, which isn=E2=80=99t a fatal flaw but = it=E2=80=99s a lot nicer to be replicating only *good* data at a higher = layer. 2. When HAST systems go split-brain, it=E2=80=99s apparently hilarious. = I don=E2=80=99t have any experience with that in production so I can=E2=80= =99t speak authoritatively about it, but the split-brain scenario has = been mentioned by some of the folks who are working on clustered = filesystems (glusterfs, ceph, etc) and I can easily imagine how that = might cause hilarity, given the fact that ZFS has no idea its underlying = block store is being replicated and also likes to commit changes in = terms of transactions (TXGs), not just individual block writes, and = writing a partial TXG (or potentially multiple outstanding TXGs with = varying degrees of completion) would Be Bad. 3. HAST only works on a pair of machines with a MASTER/SLAVE = relationship, which is pretty ghetto by today=E2=80=99s standards. HDFS = (Hadoop=E2=80=99s filesystem) can do block replication across multiple = nodes, as can DRDB (Distributed Replicated Block Device), so chasing = HAST seems pretty retro and will immediately set you up for = embarrassment when the inevitable =E2=80=9COK, that pair of nodes is = fine, but I=E2=80=99d like them both to be active and I=E2=80=99d also = like to add a 3rd node in this one scenario where I want even more = fault-tolerance - other folks can do that, how about you?=E2=80=9D = question comes up. In short, the whole thing sounds kind of MEH and that=E2=80=99s why = we=E2=80=99ve avoided putting any real time or energy into HAST. DRDB = sounds much more interesting, though of course it=E2=80=99s Linux-only. = This wouldn=E2=80=99t stop someone else from implementing a similar = scheme in a clean-room fashion, of course. And yes, of course one can layer additional things on top of iSCSI LUNs, = just as one can punch through LUNs from older SAN fabrics and put ZFS = pools on top of them (been there, done both of those things), though of = course the additional indirection has performance and debugging = ramifications of its own (when a pool goes sideways, you have additional = things in the failure chain to debug). ZFS really likes to =E2=80=9Cown = the disks=E2=80=9D in terms of providing block-level fault tolerance and = predictable performance characteristics given specific vdev topologies, = and once you start abstracting the disks away from it, making statements = about predicted IOPs for the pool becomes something of a =E2=80=9C???=E2=80= =9D exercise. - Jordan From owner-freebsd-fs@freebsd.org Fri Jul 1 18:23:58 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4E3EB8F517 for ; Fri, 1 Jul 2016 18:23:58 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6769328F2 for ; Fri, 1 Jul 2016 18:23:58 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id a66so39786704wme.0 for ; Fri, 01 Jul 2016 11:23:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=gCwMibvWJKs5cYUkO6Gr93C0gw1V5mEsComlCo+QkYE=; b=Kf+tP5BnfGJxr5DN4/CJk9BMeF0D33FFaZZ7PkLGQeELMCeB9U4r3tE5gFUWtR+0jQ dKWMKMIRtj0WB5c6QTcMUTxYcxj6sAxgDR9v+FshR0mrNeLBCBNigqRYZ29qA4hVxoKf 6BsHgcMoBiATyxIXDIU2nBp7koMTn0wot8qMy47jBBaT2N0nJfRMlBIC4ADiqj2fPpVr ZPCIPF6+myqrzKNLFXdIrzLQb9/g7v25XGy6ag+JintNijCO4+QP2J6vAq8nUQOsr3FP r59PKRsRz14Y9VxNowT5nNcLE0n1BGSGqSs6bvxuoH1tgD5TT8h4EpY3fhDK/O1Hb5NF hQlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=gCwMibvWJKs5cYUkO6Gr93C0gw1V5mEsComlCo+QkYE=; b=gvTn4RQg23M61OB6km2nP4CKlA++DPAFLzAhb2rTtfOU+Ja9pb2Z0W8DCbquhGOBKE iBWLg3Dbg5FVJp1r3/vJG5beha+cgHaMiRz1WoJfNi9gd+aM3hrP/0C24zDkzxVDc7n+ 0273ocOAEY98hcvr0QhIZ6rdg0XnJPmIjkpsby/DFX3IcX8vMw1ZG04WhRbeecBJmw5l bJITwcVszD/qGa1dQGMJeAgsvcQuBHogY2oUxrUiH9m+sY8mUKLbFo2/8CeN3xThLh6f /7bUNb5eATth1fYhzTDKmkjAs/slTddB+J1uX8VkAu/hI3ThEaKFAdE5GiFDlhCvr7Xo HFIg== X-Gm-Message-State: ALyK8tLalLBhEWGzsiB26Nt/KtQ9UG3ADD6p6kB/05hipMUOB5D8+d9IEhbkGvstO9ZaVA== X-Received: by 10.194.145.244 with SMTP id sx20mr5463983wjb.169.1467397436727; Fri, 01 Jul 2016 11:23:56 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id q63sm3661921wma.0.2016.07.01.11.23.55 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Jul 2016 11:23:55 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: Date: Fri, 1 Jul 2016 20:23:54 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <678321AB-A9F7-4890-A8C7-E20DFDC69137@gmail.com> <20160630185701.GD5695@mordor.lan> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 18:23:58 -0000 > On 01 Jul 2016, at 19:54, Jordan Hubbard wrote: > (...) > And yes, of course one can layer additional things on top of iSCSI = LUNs, just as one can punch through LUNs from older SAN fabrics and put = ZFS pools on top of them (been there, done both of those things), though = of course the additional indirection has performance and debugging = ramifications of its own (when a pool goes sideways, you have additional = things in the failure chain to debug). ZFS really likes to =E2=80=9Cown = the disks=E2=80=9D in terms of providing block-level fault tolerance and = predictable performance characteristics given specific vdev topologies, = and once you start abstracting the disks away from it, making statements = about predicted IOPs for the pool becomes something of a =E2=80=9C???=E2=80= =9D exercise. Would you say that giving an iSCSI disk to ZFS hides some details of the = raw disk to ZFS ? I though that iSCSI would have been a totally "transparent" layer, = transferring all ZFS requests to the raw disk, giving back the answers, = hiding anything. As you experienced iSCSI, any sad story with iSCSI disks given to ZFS ? Many thanks for your long feedback Jordan ! From owner-freebsd-fs@freebsd.org Sat Jul 2 00:26:11 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE1FAB8E5B7 for ; Sat, 2 Jul 2016 00:26:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE7492945 for ; Sat, 2 Jul 2016 00:26:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u620QBNY059008 for ; Sat, 2 Jul 2016 00:26:11 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 210731] kernel panic ZFS related Date: Sat, 02 Jul 2016 00:26:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 00:26:11 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210731 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jul 2 00:27:48 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 918F4B8E812 for ; Sat, 2 Jul 2016 00:27:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81A202B8A for ; Sat, 2 Jul 2016 00:27:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u620RmP8061392 for ; Sat, 2 Jul 2016 00:27:48 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 210721] Rerooting to an imported ZFS pool triggers panic Date: Sat, 02 Jul 2016 00:27:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 00:27:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210721 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jul 2 10:00:34 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22360B87D37 for ; Sat, 2 Jul 2016 10:00:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 11C902DBD for ; Sat, 2 Jul 2016 10:00:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u62A0X9a067706 for ; Sat, 2 Jul 2016 10:00:33 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 140068] [smbfs] [patch] smbfs does not allow semicolon in filenames, but should Date: Sat, 02 Jul 2016 10:00:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: easy, needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 10:00:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D140068 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ae@FreeBSD.org --- Comment #3 from Andrey V. Elsukov --- Looking in the Apple's smbfs sources we can find a bit more restrictions in= the smbfs_pathcheck() functions.=20 https://opensource.apple.com/source/smb/smb-551/kernel/smbfs/smbfs_vnops.c Also, it looks like our restricions doesn't have all characters according to Microsoft's documentation: https://msdn.microsoft.com/en-us/library/azure/dn167011.aspx --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jul 2 15:03:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B04BB8FDD5 for ; Sat, 2 Jul 2016 15:03:54 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D26F258C for ; Sat, 2 Jul 2016 15:03:54 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id r190so14470894wmr.0 for ; Sat, 02 Jul 2016 08:03:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=o+WW+om5pQdk8yH0RhhUPONN2L5rrNFED8SbArGBz70=; b=XAH5+/8oxaWDD4Mu5DI6PXwQepWr5MB+j6u7rMBHDHSiBlrtZm6YoCE58JJzQ4wxcF Sdd21AVIgnVXuUgVQO8RzAEv4HVHAZ91KgautPrzIU1GELgxYTp5rRuDHAIgKb9u0klr 89IK0NPvloZLhSKICwU9Yn/64FX0R9DDulLn+ZNQgsQc/62C2mVUKZnZqZ6hJBu4XwUY OsufVa3mnT7LFuGuYSDoLsd4LRooyKD/1YYenF+TRx5qNnW5N9AdQYaD30QOrkkgvJOi cHLLD5g/Aq3YIOKY6sCPxxMiChAyYKDo2dGn6lDGEjsicna9qIDdPVjsDouuU4xMAdKt oRyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=o+WW+om5pQdk8yH0RhhUPONN2L5rrNFED8SbArGBz70=; b=KgEd6IB5N1fKzUnKjn77sKHlLIXSLq2G8ejG8B/AL+FSDKKeW3nh8bJGYF5k6cszRY TnAbhgti0jrTzFvgUDIpsG7YG0IxCsBbsY/DMFe+jjnn0Jx0s9XutYSU5j5f0iuX+wJa uPMvz+9v+5hd+y178mru3UISBabBGyeHKsLGpwyxcbnl8zhQD0v1EgDredUCuebNJvyJ /LG9019adhflMpwl38pihwUNi2zlpA43MDEtu2PlMkZJvN3oWS/A0oykxznkHIULbkNr q2IAkRVB7NDL21GqoHi7SWCkWQTt+QSAFD8jIiUdAOZ2bWWT6CIg1PSoOCD20F2E8TGC s0XA== X-Gm-Message-State: ALyK8tK2xpP2jKPe60oLO7Grb+OJKVEEbOCzGI37SK2sIKLRLVxrYeWJGslbm3kyDDKb8w== X-Received: by 10.28.223.215 with SMTP id w206mr3095545wmg.61.1467471832360; Sat, 02 Jul 2016 08:03:52 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id n26sm3267499wmi.3.2016.07.02.08.03.50 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 02 Jul 2016 08:03:51 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: Date: Sat, 2 Jul 2016 17:03:50 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <868B90CB-828F-48E8-B678-50FC1BB52FAE@gmail.com> References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <678321AB-A9F7-4890-A8C7-E20DFDC69137@gmail.com> <20160630185701.GD5695@mordor.lan> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 15:03:54 -0000 > On 01 Jul 2016, at 20:23, Ben RUBSON wrote: >=20 >=20 >> On 01 Jul 2016, at 19:54, Jordan Hubbard wrote: >> (...) >> And yes, of course one can layer additional things on top of iSCSI = LUNs, just as one can punch through LUNs from older SAN fabrics and put = ZFS pools on top of them (been there, done both of those things), though = of course the additional indirection has performance and debugging = ramifications of its own (when a pool goes sideways, you have additional = things in the failure chain to debug). ZFS really likes to =E2=80=9Cown = the disks=E2=80=9D in terms of providing block-level fault tolerance and = predictable performance characteristics given specific vdev topologies, = and once you start abstracting the disks away from it, making statements = about predicted IOPs for the pool becomes something of a =E2=80=9C???=E2=80= =9D exercise. >=20 > Would you say that giving an iSCSI disk to ZFS hides some details of = the raw disk to ZFS ? > I though that iSCSI would have been a totally "transparent" layer, = transferring all ZFS requests to the raw disk, giving back the answers, = hiding anything. =46rom #openzfs : A: typically iSCSI disks still appear as "physical" disks to the OS = connecting to them. you can even get iSCSI servers that allow things = like SMART pass-thru Q: so ZFS will be as happy with iSCSI disks as if it used local disks ? = or will it miss something ? A: no, and ZFS isn't "unhappy" per se. but there are optimizations it = applies when it knows the disks belong to ZFS only Q: and using iSCSI disks, ZFS will not apply these optimizations (even = if these iSCSI disks are only given to ZFS) ? ie, will ZFS know these = iSCSI disks belong to ZFS only ? A: if it looks like a physical disk, if it quacks like a physical = disk...= From owner-freebsd-fs@freebsd.org Sat Jul 2 15:04:25 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB3F7B8FE24 for ; Sat, 2 Jul 2016 15:04:25 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x233.google.com (mail-wm0-x233.google.com [IPv6:2a00:1450:400c:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E2462635 for ; Sat, 2 Jul 2016 15:04:25 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x233.google.com with SMTP id r201so63279709wme.1 for ; Sat, 02 Jul 2016 08:04:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=IyE2mf+k+cmA39kJ6mLs7DmkSGvdavwBuYNk3P26Joo=; b=fK++Sh+omSOQIBQxoZugVl6XsQMFR/CD11R2ImB3srOmF4tzC63cAP4ubyaAqeCMww auAHGBKoAl3q8eX6yqDtLxZCBdRFGCMLCZoWbRKvtLvuvDUmcr7TVFoofN1DOQcxYv0T DvUyTJCKOhTlUOUNUgyzluUyubB4SFfwWhdaQDqILP/y/3z0hWZOEQ/6KBfeu5hDElxS hvWF4FcBwcs4SKqJUGgSqjcBG4lcUguDCp4KlN/2NHXyHok/4jRuZaab/mMv/fJyImGU HGITJ4UXcER3+sDmqdRIoIS1s9kVyHj8LJd4vrVUz9Vj64GKbLEPYO+N1rdzk1bh7nmE m+vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=IyE2mf+k+cmA39kJ6mLs7DmkSGvdavwBuYNk3P26Joo=; b=GRmbd2nscSu5Fqd76eT1ThDhqIsr6/wuVxczORLoLLyp15mGNRPktZvLRVhyJRcmyC 7HwMYNs1bP8v5svpvK34CtLKjwBLHJWfcKXer8kXDp+dhbZeGD9F/ixYdgLmGLqVvPjP 2qfKUycJHwK/8vTFk17Wg0UZ3oB5hyeMz1rLMlpMIU4WmSvXOSv5W9pRwy36xcqFuF2E IXuMDB4WpiKqTIz3nqHmmK7mttZBNqM2ULWreDf78IVU3B22+gS24hPbqLj8fKlUPmx9 8AQEXBvlOlcQZsOSx4ArAXbZx/wQ0A/lah+whQi1CWB2f847eRBR7Z3piNe8FiBNyfUB xtOg== X-Gm-Message-State: ALyK8tJu2BtR2V0YyJOFru2xLeID3l/ooD86s8aDvxLCLUQxdCHq+JM6xjem3+XN8MTogA== X-Received: by 10.28.107.23 with SMTP id g23mr3380231wmc.101.1467471863555; Sat, 02 Jul 2016 08:04:23 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id n26sm3267499wmi.3.2016.07.02.08.04.22 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 02 Jul 2016 08:04:22 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: <20160630185701.GD5695@mordor.lan> Date: Sat, 2 Jul 2016 17:04:22 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6035AB85-8E62-4F0A-9FA8-125B31A7A387@gmail.com> References: <20160630144546.GB99997@mordor.lan> <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <678321AB-A9F7-4890-A8C7-E20DFDC69137@gmail.com> <20160630185701.GD5695@mordor.lan> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 15:04:25 -0000 > On 30 Jun 2016, at 20:57, Julien Cigar wrote: >=20 > On Thu, Jun 30, 2016 at 11:32:17AM -0500, Chris Watson wrote: >>=20 >>=20 >> Sent from my iPhone 5 >>=20 >>>=20 >>>>=20 >>>> Yes that's another option, so a zpool with two mirrors (local +=20 >>>> exported iSCSI) ? >>>=20 >>> Yes, you would then have a real time replication solution (as HAST), = compared to ZFS send/receive which is not. >>> Depends on what you need :) >>>=20 >>>>=20 >>>>> ZFS would then know as soon as a disk is failing. >>=20 >> So as an aside, but related, for those watching this from the peanut = gallery and for the benefit of the OP perhaps those that run with this = setup might give some best practices and tips here in this thread on = making this a good reliable setup. I can see someone reading this thread = and tossing two crappy Ethernet cards in a box and then complaining it = doesn't work well.=20 >=20 > It would be more than welcome indeed..! I have the feeling that HAST > isn't that much used (but maybe I am wrong) and it's difficult to find=20= > informations on it's reliability and concrete long-term use cases... >=20 > Also the pros vs cons of HAST vs iSCSI I made further testing today. # serverA, serverB : kern.iscsi.ping_timeout=3D5 kern.iscsi.iscsid_timeout=3D5 kern.iscsi.login_timeout=3D5 kern.iscsi.fail_on_disconnection=3D1 # Preparation : - serverB : let's make 2 iSCSI targets : rem3, rem4. - serverB : let's start ctld. - serverA : let's create a mirror pool made of 4 disks : loc1, loc2, = rem3, rem4. - serverA : pool is healthy. # Test 1 : - serverA : put a lot of data into the pool ; - serverB : stop ctld ; - serverA : put a lot of data into the pool ; - serverB : start ctld ; - serverA : make all pool disks online : it works, pool is healthy. # Test 2 : - serverA : put a lot of data into the pool ; - serverA : export the pool ; - serverB : import the pool : it does not work, as ctld locks the disks = ! Good news, nice protection (both servers won't be able to access the = same disks at the same time). - serverB : stop ctld ; - serverB : import the pool : it works, 2 disks missing ; - serverA : let's make 2 iSCSI targets : rem1, rem2 ; - serverB : make all pool disks online : it works, pool is healthy. # Test 3 : - serverA : put a lot of data into the pool ; - serverB : stop ctld ; - serverA : put a lot of data into the pool ; - serverB : import the pool : it works, 2 disks missing ; - serverA : let's make 2 iSCSI targets : rem1, rem2 ; - serverB : make all pool disks online : it works, pool is healthy, but = of course data written at step3 is lost. # Test 4 : - serverA : put a lot of data into the pool ; - serverB : stop ctld ; - serverA : put a lot of data into the pool ; - serverA : export the pool ; - serverA : let's make 2 iSCSI targets : rem1, rem2 ; - serverB : import the pool : it works, pool is healthy, data written at = step3 is here. # Test 5 : - serverA : rsync a huge remote repo into the pool in the background ; - serverB : stop ctld ; - serverA : 2 disks missing, but rsync still runs flawlessly ; - serverB : start ctld ; - serverA : make all pool disks online : it works, pool is healthy. - serverB : ifconfig down ; - serverA : 2 disks missing, but rsync still runs flawlessly ; - serverB : ifconfig up ; - serverA : make all pool disks online : it works, pool is healthy. - serverB : power reset ! - serverA : 2 disks missing, but rsync still runs flawlessly ; - serverB : let's wait for server to be up ; - serverA : make all pool disks online : it works, pool is healthy. Quite happy with these tests actually :) Ben From owner-freebsd-fs@freebsd.org Sat Jul 2 18:28:19 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCF69B8F9B5 for ; Sat, 2 Jul 2016 18:28:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD2672E21 for ; Sat, 2 Jul 2016 18:28:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u62ISJjq041399 for ; Sat, 2 Jul 2016 18:28:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 139715] [zfs] vfs.numvnodes leak on busy zfs Date: Sat, 02 Jul 2016 18:28:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: swills@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 18:28:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D139715 Steve Wills changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |swills@FreeBSD.org --- Comment #9 from Steve Wills --- Is this related at all to the issue in 209158 ? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jul 2 18:28:51 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88CAAB8F9FF for ; Sat, 2 Jul 2016 18:28:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 78F712E9B for ; Sat, 2 Jul 2016 18:28:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u62ISpeE042012 for ; Sat, 2 Jul 2016 18:28:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 139715] [zfs] vfs.numvnodes leak on busy zfs Date: Sat, 02 Jul 2016 18:28:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: swills@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 18:28:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D139715 --- Comment #10 from Steve Wills --- (In reply to Steve Wills from comment #9) Sorry, PR 209158 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jul 2 19:00:27 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 12543B8F108 for ; Sat, 2 Jul 2016 19:00:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 027082AC1 for ; Sat, 2 Jul 2016 19:00:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u62J0NK9009335 for ; Sat, 2 Jul 2016 19:00:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 209158] node / npm triggering zfs rename deadlock Date: Sat, 02 Jul 2016 19:00:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 19:00:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209158 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mckusick@FreeBSD.org --- Comment #32 from Kirk McKusick --- Commits 301996 and 301997 made on 2016-06-17 to relieve pressure by ZFS on vfs.numvnodes should help moderate this problem. This is not to say that it will fully resolve the problem, so the ongoing work should continue. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jul 2 19:03:39 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83FD5B8F27E for ; Sat, 2 Jul 2016 19:03:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 743382D49 for ; Sat, 2 Jul 2016 19:03:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u62J3cGP049341 for ; Sat, 2 Jul 2016 19:03:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 139715] [zfs] vfs.numvnodes leak on busy zfs Date: Sat, 02 Jul 2016 19:03:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 19:03:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D139715 --- Comment #11 from Kirk McKusick --- I believe that the fix for this PR will be helpful for PR 209158 and have a= dded a comment to that effect in PR 209158. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Jul 2 19:05:15 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE52AB8F2D8 for ; Sat, 2 Jul 2016 19:05:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE7052DD7 for ; Sat, 2 Jul 2016 19:05:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u62J5Fqw051219 for ; Sat, 2 Jul 2016 19:05:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 139715] [zfs] vfs.numvnodes leak on busy zfs Date: Sat, 02 Jul 2016 19:05:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jul 2016 19:05:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D139715 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|In Progress |Closed --- Comment #12 from Kirk McKusick --- Several folks experiencing this bug report it to be fixed with 301996 and 301997. --=20 You are receiving this mail because: You are the assignee for the bug.=