From owner-freebsd-fs@freebsd.org Fri Jul 1 11:58:46 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 492B9B8E9C0 for ; Fri, 1 Jul 2016 11:58:46 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE744287C for ; Fri, 1 Jul 2016 11:58:45 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22f.google.com with SMTP id v199so23639624wmv.0 for ; Fri, 01 Jul 2016 04:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=hKuGzqdo64rbu4luDd1331+OCPqVltngLBtk74XI72M=; b=TvTR9Q/oI/Qjp55Vngs3QGjOWGeVyV2E0QBNlFixXPmVbZMP9kaxvQZaPlBcCqfAaN o52ZmRAL2BwpACqNVhgiIVQcOuPY2xKXHyJ+sVq2tFGP3lXgWyGqBsj3veRJJyh4E0AX j2ro23Di0IPaA1LFyk+bk4+O0UrKak+xc8pEcj9ck3VqddmrhnHTgknjpDKi2nejnPu2 7BulKOFAlasKyM19zuPr4w95ziIwlJiOUrvxjRMZGpBixxZLEqrwiCIleI32xI3pt+oh qMxpxj+zSvW1+DqsBApojyULVY/hw5Y2OYnon75VxrT82xsx0YF7kIVuJy/AfjmTULsS Dprg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=hKuGzqdo64rbu4luDd1331+OCPqVltngLBtk74XI72M=; b=T+LSmkoXiMdE4C6xtXBlyfMki6DOtvlmBwGm6ewY7/2v2zmGRp4fXx6aa5LlQKGhzF s4vdpCP6d9BZYyEc8SSX236ftHp/juIXdBPARxohrmNwv6XInsbKyh8oCOA5itiSrs/W qWl2E6guhI+608S0P8177dh/q8TvBRN90Y7WlXPsbkA/y9uMeT11d0++OE0Hl8E9X+uo HezAN8pENC8AwlIBprzRD6vW4/AnxvLXg1GRJvXjZ2Kvitb067kU9KOu8IoCPrmVz3O2 bf3yZQXiN384xNjByG9AFVYRiFROuJcfhPijfRArFzv8tD5whxiiYe9AiZMi5UntQlkp 0BGQ== X-Gm-Message-State: ALyK8tJ3R779LJDEJjyI20Ahl2C9e1hXq2Es8u2ELd/aFQ+DAwfvptPM03L5Dr3QsB0vpg== X-Received: by 10.28.17.132 with SMTP id 126mr32796392wmr.90.1467374324029; Fri, 01 Jul 2016 04:58:44 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7077-85.w90-116.abo.wanadoo.fr. [90.116.246.85]) by smtp.gmail.com with ESMTPSA id u4sm2865318wjz.4.2016.07.01.04.58.43 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Jul 2016 04:58:43 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: HAST + ZFS + NFS + CARP From: Ben RUBSON In-Reply-To: <5776569B.3050504@quip.cz> Date: Fri, 1 Jul 2016 13:58:42 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <5F99508D-7532-468A-9121-7A76957A72DB@gmail.com> References: <71b8da1e-acb2-9d4e-5d11-20695aa5274a@internetx.com> <20160630153747.GB5695@mordor.lan> <63C07474-BDD5-42AA-BF4A-85A0E04D3CC2@gmail.com> <20160630163541.GC5695@mordor.lan> <50BF1AEF-3ECC-4C30-B8E1-678E02735BB5@gmail.com> <20160701084717.GE5695@mordor.lan> <47c7e1a5-6ae8-689c-9c2d-bb92f659ea43@internetx.com> <20160701101524.GF5695@mordor.lan> <20160701105735.GG5695@mordor.lan> <5776569B.3050504@quip.cz> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 11:58:46 -0000 > On 01 Jul 2016, at 13:40, Miroslav Lachman <000.fbsd@quip.cz> wrote: >=20 > Julien Cigar wrote on 07/01/2016 12:57: >=20 >>>> why...? I guess iSCSI is slower but should be safer than HAST, no? >>>=20 >>> do your testing, please. even with simulated short network cuts. = 10-20 >>> secs are way enaugh to give you a picture of what is going to happen >>=20 >> of course I'll test everything properly :) I don't have the hardware = yet >> so ATM I'm just looking for all the possible "candidates", and I'm >> aware that a redundant storage is not that easy to implement ... >>=20 >> but what solutions do we have? It's either CARP + ZFS + (HAST|iSCSI), >> either zfs send|ssh zfs receive as you suggest (but it's >> not realtime), either a distributed FS (which I avoid like the = plague..) >=20 > When disaster comes you will need to restart NFS clients in almost all = cases (with CARP + ZFS + HAST|iSCSI) and you will lose some writes too. > And if something bad happens with your mgmt scripts or network you can = end up with corrupted ZFS pool on master and slave too - you will need = to recovery from backups. For example in some split brain scenario when = both nodes will try to import pool. Of course you must take care that both nodes do not import the pool at = the same time. For the slave to import the pool, first stop iSCSI targets (ctld), and = also put network replication interface down, to be sure. Then, import the pool. Once old master repaired, export its pool (if still imported), make its = disks iSCSI targets and give them the old slave (promoted master just = above). Of course it implies some meticulous administration. > With ZFS send & receive you will lose some writes but the chance you = will corrupt both pools are much lower than in the first case and the = setup is much simpler and runtime error proof. Only some ? Depending on the write throughput, won't you loose a lot of data on the = target/slave ? How do you make ZFS send/receive quite realtime ? while [ 1 ] do ; snapshot ; send/receive ; delete old snapshots ; done ? Thanks !=