From owner-freebsd-fs@freebsd.org Fri Jul 1 10:57:41 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7404FB8E430 for ; Fri, 1 Jul 2016 10:57:41 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b02.edpnet.be (relay-b02.edpnet.be [212.71.1.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26F3B209F for ; Fri, 1 Jul 2016 10:57:40 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467370655-0a7b8d1905c0a50001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b02.edpnet.be with ESMTP id ZHfirQ03CvbH3WzE (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 01 Jul 2016 12:57:36 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Fri, 1 Jul 2016 12:57:35 +0200 From: Julien Cigar To: InterNetX - Juergen Gotteswinter Cc: Ben RUBSON , freebsd-fs@freebsd.org Subject: Re: HAST + ZFS + NFS + CARP Message-ID: <20160701105735.GG5695@mordor.lan> X-ASG-Orig-Subj: Re: HAST + ZFS + NFS + CARP References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Qf1oXS95uex85X0R" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467370655 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.222:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 4964 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5000 1.0000 0.0100 X-Barracuda-Spam-Score: 0.01 X-Barracuda-Spam-Status: No, SCORE=0.01 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30921 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 10:57:41 -0000 --Qf1oXS95uex85X0R Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 01, 2016 at 12:18:39PM +0200, InterNetX - Juergen Gotteswinter = wrote: > Am 01.07.2016 um 12:15 schrieb Julien Cigar: > > On Fri, Jul 01, 2016 at 11:42:13AM +0200, InterNetX - Juergen Gotteswin= ter wrote: > >>> > >>> Thank you very much for those "advices", it is much appreciated!=20 > >>> > >>> I'll definitively go with iSCSI (for which I haven't that much=20 > >>> experience) over HAST. > >> > >> good luck, i rather cut one of my fingers than using something like th= is > >> in production. but its probably a quick way if one targets to find a n= ew > >> opportunity ;) > >=20 > > why...? I guess iSCSI is slower but should be safer than HAST, no? >=20 > do your testing, please. even with simulated short network cuts. 10-20 > secs are way enaugh to give you a picture of what is going to happen of course I'll test everything properly :) I don't have the hardware yet so ATM I'm just looking for all the possible "candidates", and I'm=20 aware that a redundant storage is not that easy to implement ... but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI),=20 either zfs send|ssh zfs receive as you suggest (but it's not realtime), either a distributed FS (which I avoid like the plague..) >=20 > >> > >>> > >>> Maybe a stupid question but, assuming on the MASTER with ada{0,1} the= =20 > >>> local disks and da{0,1} the exported iSCSI disks from the SLAVE, woul= d=20 > >>> you go with: > >>> > >>> $> zpool create storage mirror /dev/ada0s1 /dev/ada1s1 mirror /dev/da0 > >>> /dev/da1 > >>> > >>> or rather: > >>> > >>> $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > >>> /dev/da1 > >>> > >>> I guess the former is better, but it's just to be sure .. (or maybe i= t's > >>> better to iSCSI export a ZVOL from the SLAVE?) > >>> > >> > >> are you really sure you understand what you trying to do? even if its > >> currently so, i bet in a desaster case you will be lost. > >> > >> > >=20 > > well this is pretty new to me, but I don't see what could be wrong with: > >=20 > > $> zpool create storage mirror /dev/ada0s1 /dev/da0 mirror /dev/ada1s1 > > /dev/da1 > >=20 > > Let's take some use-cases: > > - MASTER and SLAVE are alive, the data is "replicated" on both > > nodes. As iSCSI is used, ZFS will see all the details of the > > underlying disks and we can be sure that no corruption will occur > > (contrary to HAST) > > - SLAVE die, correct me if I'm wrong the but pool is still available, > > fix the SLAVE, resilver and that's it ..? > > - MASTER die, CARP will notice it and SLAVE will take the VIP, the > > failover script will be executed with a $> zpool import -f > >=20 > >>> Correct me if I'm wrong but, from a safety point of view this setup i= s=20 > >>> also the safest as you'll get the "fullsync" equivalent mode of HAST > >>> (but but it's also the slowest), so I can be 99,99% confident that the > >>> pool on the SLAVE will never be corrupted, even in the case where the > >>> MASTER suddently die (power outage, etc), and that a zpool import -f > >>> storage will always work? > >> > >> 99,99% ? optimistic, very optimistic. > >=20 > > the only situation where corruption could occur is some sort of network > > corruption (bug in the driver, broken network card, etc), or a bug in > > ZFS ... but you'll have the same with a zfs send|ssh zfs receive > >=20 > >> >=20 > optimistic >=20 > >> we are playing with recovery of a test pool which has been imported on > >> two nodes at the same time. looks pretty messy > >> > >>> > >>> One last thing: this "storage" pool will be exported through NFS on t= he=20 > >>> clients, and when a failover occur they should, in theory, not notice > >>> it. I know that it's pretty hypothetical but I wondered if pfsync cou= ld > >>> play a role in this area (active connections)..? > >>> > >> > >> they will notice, and they will stuck or worse (reboot) > >=20 > > this is something that should be properly tested I agree.. > >=20 >=20 > do your testing, and keep your clients under load while testing. do > writes onto the nfs mounts and then cut. you will be surprised about the > impact. >=20 > >> >=20 >=20 >=20 > >>> Thanks! > >>> Julien > >>> > >>>> > >>>>>>>> ZFS would then know as soon as a disk is failing. > >>>>>>>> And if the master fails, you only have to import (-f certainly, = in case of a master power failure) on the slave. > >>>>>>>> > >>>>>>>> Ben > >>>> _______________________________________________ > >>>> freebsd-fs@freebsd.org mailing list > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >>> > >=20 >=20 --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --Qf1oXS95uex85X0R Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdkybAAoJELK7NxCiBCPAXy4QAKb/OswvhYZVN/0FcWRLdrPk X7oBaNKYTXnVa2+CWpkKWIXWzLuYjtU7mO8YpDeTRR97HwWaz39hQlC86Rtwvl+K bU91c5h/ykCD9pBe7IOA8bu86ejGJyW619C3VjbPFhRIpslNRULu5lGFaGSfka2S 7JiGXwicr8AhQd8TAj9DJCQcK/pr8u9k3KYgqU4MxxFrr14+++HzRKk9m/fFpQBy LhIK4DIzc32Wbta0bIfOeIhNS7ATJhXcyfoWEKHEy5JUq5j78p779An76UuyJLil 63H9s3pKT1QdEFiiVBjIEz02Y8XHDXEpqDgA9TRWLw+jGngXhGs79Syjdzo0RPlT DyV5TIgjcXoDyONvxFxcBsiEgI25LzKFwNQeTOWYEQzST1hAolzAgMEK0cKiQfde 3OBp3jvOocT0/7XfTE/Il8UWMdad2fZS6VAJGX0Ei9TXHCHCMhaTkCoqZ20py//7 4k4gBV3yeQpyMkHKHDlokRV63SnDON+V5/XyAyvbPS0df3zmhMVe2v0d0ODDxAis MQBkfnBtzQ6MT3OZWwL+xQgXA1u+iXkZsbWklGwcxTCy52wCLlBNtYjEyEoxX6Yg xEV1KG8m2u2e/U/uoBW9/aKKJC8wLmsDjMkss/ReO/aNIX+cZsTUQcq0dV73kW3O 6Z8kXmiz6O2xK5VmEksm =64Uc -----END PGP SIGNATURE----- --Qf1oXS95uex85X0R--