From owner-freebsd-fs@freebsd.org Fri Jul 1 12:38:11 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93B99B87398 for ; Fri, 1 Jul 2016 12:38:11 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b02.edpnet.be (relay-b02.edpnet.be [212.71.1.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44DA0220A for ; Fri, 1 Jul 2016 12:38:10 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467376680-0a7b8d12103d727b0001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b02.edpnet.be with ESMTP id GIrk1I5C0l7xrKMp (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 14:38:01 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 14:38:00 +0200 From: Julien Cigar To: Ben RUBSON Cc: freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701123800.GA41276@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <5776569B.3050504@quip.cz> <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467376680 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.222:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2961 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30922 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 12:38:11 -0000 --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 01:58:42PM +0200, Ben RUBSON wrote: >=20 > > On 01 Jul 2016, at 13:40, Miroslav Lachman <000.fbsd@quip.cz> wrote: > >=20 > > Julien Cigar wrote on 07/01/2016 12:57: > >=20 > >>>> why...? I guess iSCSI is slower but should be safer than HAST, no? > >>>=20 > >>> do your testing, please. even with simulated short network cuts. 10-20 > >>> secs are way enaugh to give you a picture of what is going to happen > >>=20 > >> of course I'll test everything properly :) I don't have the hardware y= et > >> so ATM I'm just looking for all the possible "candidates", and I'm > >> aware that a redundant storage is not that easy to implement ... > >>=20 > >> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), > >> either zfs send|ssh zfs receive as you suggest (but it's > >> not realtime), either a distributed FS (which I avoid like the plague.= =2E) > >=20 > > When disaster comes you will need to restart NFS clients in almost all = cases (with CARP + ZFS + HAST|iSCSI) and you will lose some writes too. > > And if something bad happens with your mgmt scripts or network you can = end up with corrupted ZFS pool on master and slave too - you will need to r= ecovery from backups. For example in some split brain scenario when both no= des will try to import pool. >=20 > Of course you must take care that both nodes do not import the pool at th= e same time. > For the slave to import the pool, first stop iSCSI targets (ctld), and al= so put network replication interface down, to be sure. > Then, import the pool. > Once old master repaired, export its pool (if still imported), make its d= isks iSCSI targets and give them the old slave (promoted master just above). > Of course it implies some meticulous administration. I was thinking something like this also.. and I definitively think that=20 the switch from old save (promoted master) to "old master repaired" MUST be done manually! >=20 > > With ZFS send & receive you will lose some writes but the chance you wi= ll corrupt both pools are much lower than in the first case and the setup i= s much simpler and runtime error proof. I think losing some writes is somewhat unavoidable, corruption on the other hand is unacceptable >=20 > Only some ? > Depending on the write throughput, won't you loose a lot of data on the t= arget/slave ? > How do you make ZFS send/receive quite realtime ? > while [ 1 ] do ; snapshot ; send/receive ; delete old snapshots ; done ? >=20 > Thanks ! > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --wRRV7LY7NUeQGEoC Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdmQkAAoJELK7NxCiBCPAK2oQAOhKqx3mmjsfU7CPpQqym8zY NYMwPXKPOCYPZOe8jiq10j26JTdWgnjR+mXgEjuf8TioH7Xwk+O18w4i8P3X6XlX gvpxF2smrhf0ujV/roQh/8MUW3zl6EbC/MQl65pZPotiMFCeWzi6CUcFPXXBr8/9 GxRU9TTQMET3z/wAED/iFvehJIIHu97u+imRp3hmEIAJuWJBqdU9NkucQSsLjmWA 1PYUHjm9VVtvazrDG59UGSRp7vmQMf3v0vhMkTz1iFR2U1lHekySp5xfPQ2NvFar PaUGjN7oEsHYv0vLI5v3urfMirCa/WSIbvQM1xNuLFSitNwAzJCV1bK/0ORXwkca VjZ8x+dt0LIFUOFtQ1A32dtebJMSDZC/XWOBgyFv4hysUUhtcs1cUK510UzMDPmA lTz101Fs2mGTHlMXohTLHMV9vqDPBiCBw/b35APIy9be0ggtErHySCsha2cWIPU8 xTSAO2Sojn0JuzLIj04Qwfy9UGHtmj4PKF2XaRlQhdcLiOfQCFevJKo7c8PqqOhc q1fDmpVd/fRU7OQW3O4rvM5e5oCN/+yCUY6mBoBcTJaQvKT/Ayu/NYLptnN+Dh15 8t78F18hYJdf+2B2ZeB4XiLJU94vWjd2W7sc6IZS+jHEX2JmQpo2O7+4l4LzHyKP tUnBLu0eIaWdRkS1KlIU =RNKE -----END PGP SIGNATURE----- --wRRV7LY7NUeQGEoC--