From owner-freebsd-fs@freebsd.org Thu Jun 30 15:04:01 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FE0CB87917 for ; Thu, 30 Jun 2016 15:04:01 +0000 (UTC) (envelope-from julien@perdition.city) Received: from relay-b03.edpnet.be (relay-b03.edpnet.be [212.71.1.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "edpnet.email", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5A9B2A1D for ; Thu, 30 Jun 2016 15:04:00 +0000 (UTC) (envelope-from julien@perdition.city) X-ASG-Debug-ID: 1467297946-0a88181ce55a0040001-3nHGF7 Received: from mordor.lan (213.219.165.225.bro01.dyn.edpnet.net [213.219.165.225]) by relay-b03.edpnet.be with ESMTP id esbEPwCyqcL6Li5F (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 30 Jun 2016 16:45:48 +0200 (CEST) X-Barracuda-Envelope-From: julien@perdition.city X-Barracuda-Effective-Source-IP: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Apparent-Source-IP: 213.219.165.225 Date: Thu, 30 Jun 2016 16:45:46 +0200 From: Julien Cigar To: freebsd-fs@freebsd.org Subject: HAST + ZFS + NFS + CARP Message-ID: <20160630144546.GB99997@mordor.lan> X-ASG-Orig-Subj: HAST + ZFS + NFS + CARP MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="1UWUbFP1cBYEclgG" Content-Disposition: inline User-Agent: Mutt/1.6.1 (2016-04-27) X-Barracuda-Connect: 213.219.165.225.bro01.dyn.edpnet.net[213.219.165.225] X-Barracuda-Start-Time: 1467297947 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://212.71.1.220:443/cgi-mod/mark.cgi X-Barracuda-Scan-Msg-Size: 2957 X-Virus-Scanned: by bsmtpd at edpnet.be X-Barracuda-BRTS-Status: 1 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4783 1.0000 0.0000 X-Barracuda-Spam-Score: 3.13 X-Barracuda-Spam-Status: No, SCORE=3.13 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests=BSF_SC0_MV0713, BSF_SC0_MV0713_2, SUBJ_ALL_CAPS, SUBJ_ALL_CAPS_2 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.30897 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 SUBJ_ALL_CAPS Subject is all capitals 1.62 SUBJ_ALL_CAPS_2 SUBJ_ALL_CAPS_2 0.50 BSF_SC0_MV0713 Custom rule MV0713 1.00 BSF_SC0_MV0713_2 BSF_SC0_MV0713_2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2016 15:04:01 -0000 --1UWUbFP1cBYEclgG Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, I'm always in the process of setting a redundant low-cost storage for=20 our (small, ~30 people) team here. I read quite a lot of articles/documentations/etc and I plan to use HAST with ZFS for the storage, CARP for the failover and the "good old NFS" to mount the shares on the clients. The hardware is 2xHP Proliant DL20 boxes with 2 dedicated disks for the shared storage. Assuming the following configuration: - MASTER is the active node and BACKUP is the standby node. - two disks in each machine: ada0 and ada1. - two interfaces in each machine: em0 and em1 - em0 is the primary interface (with CARP setup) - em1 is dedicated to the HAST traffic (crossover cable) - FreeBSD is properly installed in each machine. - a HAST resource "disk0" for ada0p2. - a HAST resource "disk1" for ada1p2. - a zpool create zhast mirror /dev/hast/disk0 /dev/hast/disk1 is created on MASTER A couple of questions I am still wondering: - If a disk dies on the MASTER I guess that zpool will not see it and will transparently use the one on BACKUP through the HAST ressource.. is it a problem? could this lead to some corruption? At this stage the common sense would be to replace the disk quickly, but imagine the worst case scenario where ada1 on MASTER dies, zpool will not see it=20 and will transparently use the one from the BACKUP node (through the=20 "disk1" HAST ressource), later ada0 on MASTER dies, zpool will not=20 see it and will transparently use the one from the BACKUP node=20 (through the "disk0" HAST ressource). At this point on MASTER the two=20 disks are broken but the pool is still considered healthy ... What if=20 after that we unplug the em0 network cable on BACKUP? Storage is down.. - Under heavy I/O the MASTER box suddently dies (for some reasons),=20 thanks to CARP the BACKUP node will switch from standy -> active and=20 execute the failover script which does some "hastctl role primary" for the ressources and a zpool import. I wondered if there are any situations where the pool couldn't be imported (=3D data corruption)? For example what if the pool hasn't been exported on the MASTER before it dies? - Is it a problem if the NFS daemons are started at boot on the standby node, or should they only be started in the failover script? What about stale files and active connections on the clients? - A catastrophic power failure occur and MASTER and BACKUP are suddently powered down. Later the power returns, is it possible that some problem occur (split-brain scenario ?) regarding the order in which the two machines boot up? - Other things I have not thought? Thanks! Julien --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --1UWUbFP1cBYEclgG Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXdTCWAAoJELK7NxCiBCPA6igQANRq1XO0qCQ40ZQv5xctdLNI 3NMR10J2Ghf2X59WmvLg0a7qqwxjn1e7Km9XYUfr05UgLFAr8OCXY6gDHiR8tO1l 9V3gKI3iqEjfOKdmzhrE22oow1gSzvjvA5pzTRGQDwmzNQbirtqH+xieRyP++mXN frXyR2SAQxChnJX4vPI7ysbRtR1V2l+QMyVT/aFoJFNTjZ/XxD9M4M7TQsW3zU46 bY18us9CN+inLs/RcPgccDQVqMpkGaDRqsrDhhrCWFklss0dNMYbU9N9uwFR+yaZ 23Li73SmJdlzCsr54z5ZN8iieFW2wCH3wALtaogc/1fdhTYClNHv6eBTEiuxdLK3 UkTFGdLHaeKe4ViIgq8gZfdGUryBfmThWiwsj+Aka3e4fMR78hqAjLCvmKOEwl75 Mak+vxP5p1L6xpmtwfk/oLVyua+eKMOnA+pEQwhttla1ACxOfketPnDamMf/BOEH nuSPuKD2AJrEQzXiONQkKEl25EHFiaF0RPWeWxT5g8jWdC/wsDhfoQZFVzMoqPzT skIfnLarO+JQuynPs2JxBCBPSgGpbv3xEhHfdEebUZlMqc2UI+CstqMt4v6+/kQv davNxLQg30/aMTq0GP31aryEmyq9TBVdTbol5lTFql9bSWmv/4RBe6fy3twip/CD 4RRWJkhwr5phiBWrp9Jf =6ezE -----END PGP SIGNATURE----- --1UWUbFP1cBYEclgG--