From owner-freebsd-stable@freebsd.org Fri Jun 3 16:38:10 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 236ACB68B9C for ; Fri, 3 Jun 2016 16:38:10 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 034A011EF for ; Fri, 3 Jun 2016 16:38:10 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 0297FB68B9A; Fri, 3 Jun 2016 16:38:10 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0239FB68B99 for ; Fri, 3 Jun 2016 16:38:10 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qg0-x233.google.com (mail-qg0-x233.google.com [IPv6:2607:f8b0:400d:c04::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B1C0511EE for ; Fri, 3 Jun 2016 16:38:09 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-qg0-x233.google.com with SMTP id 93so14352469qgx.2 for ; Fri, 03 Jun 2016 09:38:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=vfOhTqyPc8/PpsI4QfWODiqXk7xuJzq6Yhw3YdLjmTY=; b=ipQeABY4h68ecFBdNc9LX/DMRnpAkxQYhQqREpG79HZMBzJp3shfdrR2OvUyOxdARD 8KxYvB8UwAE3CweXSHKqbwlDFoclGaUjITB8MO2ZFVhOfEUOGIytcmGvTkeaQMDTSBBq RnpG2Iih6MsHcftOEN5ZoayDgDZWMbzPLLc/hOfkWI0dcjLDhoyRkJYgZuZ+0JlKjj0u BDNuJfdaOWKsHdHhuRQEj6kxDOBUGdi0qPGC1Z5zRJn0cmMNHHDTkg8M8+yVACip2xEz 1iLXdPbWg4Nozr+zqpUY+lKHtV8NIhTTE7w1QMtNcwqATbttniZoSf+QMoH9FD2sjq+w 0bnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=vfOhTqyPc8/PpsI4QfWODiqXk7xuJzq6Yhw3YdLjmTY=; b=MyoXLQKS/0pk2+q5j4lgF1fYP68+mMMwOsZrisK4LBIhhlnZfLduN1/pvf6sGAF0RD 7nt0g/C9MsQloGmDNu1RSqQvOF7jE3tfbb3UFteoShRxnw+5Tm8W4QfleCDRKBzgz+lH MRBxRXXtfDzY7XnJS/KUg/M7IyvNPdlIh1KJs+qQuZUPXRiXMdxeqsRunOgvfZEC84Zm ZaVoYhyAot7lGjsizd5vA/Y3CYUy6yuuhZ6rRVaDOC8MbXu3jVq0E/J4ohMEfn2AalVO 7B6nQXZyKFpPoC6xDogo8Z/i63LLvmJo7FMdwlnRVA4O/p74SQ/Nk7LY79rdMQ8h6o39 pAvg== X-Gm-Message-State: ALyK8tIhmIdulE/7tXhXAws/uBA8uW7aNQyLvErsJhLw+srEBF4IoPncuwVsBVsXP3vkwFx+EiWRdlQBCvbwMA== X-Received: by 10.140.91.53 with SMTP id y50mr3866947qgd.104.1464971888817; Fri, 03 Jun 2016 09:38:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.237.38.5 with HTTP; Fri, 3 Jun 2016 09:38:08 -0700 (PDT) In-Reply-To: <57514296.7090906@norma.perm.ru> References: <20160601132300.GA75625@zxy.spb.ru> <574FDC63.6090100@norma.perm.ru> <20160602145028.GC75625@zxy.spb.ru> <57514296.7090906@norma.perm.ru> From: Freddie Cash Date: Fri, 3 Jun 2016 09:38:08 -0700 Message-ID: Subject: Re: HAST, zfs and local mirroring To: "Eugene M. Zheganin" Cc: stable@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jun 2016 16:38:10 -0000 On Fri, Jun 3, 2016 at 1:40 AM, Eugene M. Zheganin wrote: > Hi. > > On 02.06.16 19:50, Slawa Olhovchenkov wrote: > > I am suggesting next setup: > > > > node0: > > own pool zroot0: mirror-0: local_disk0 > > remote-iscsi_disk1/1 > > local_disk1: exported by iscsi as remote-iscsi_disk0/1 to node1 > > > > node1: > > own pool zroot1: mirror-0: local_disk0 > > remote-iscsi_disk0/1 > > local_disk1: exported by iscsi as remote-iscsi_disk1/1 to node0 > > > > > > No HAST. > > Disks synced by ZFS over iSCSI. > But this way I will get two independent zfs pools (or I still didn't get > it), one half of each will be stored on another machine. And I need > something different - a continuously replicated disk resource, that > would be available on both machines in case either will crash. Cluster > filesystem would be fine, but as far as I know there's no such thing on > FreeBSD, so I accept it will be unavailable on slave while mounted as > read-write on the master. HAST looks like a thing thats fits my > requirement, only that I wank it to be redundant inside each host too, > and all the documentation shows examples without local redundancy. > =E2=80=8BWhich is exactly the setup that I showed you. Host1: diskA diskB Host2: diskC diskD You create two HAST resources: hast1: using diskA and diskC hast2: using diskB and diskD Then create a ZFS pool using hast1 and hast2 as a mirror vdev. That way, if you lose diskA, then diskC takes over, and ZFS never even notices. If you lose both diskA and diskC, you still have half the ZFS mirror vdev running, so the pool is intact. If you lose all of Host1, Host2 takes over, imports the pool, and carries on. HAST + CARP (to manage the fail-over and pool export/import) + ZFS gives you everything (well, mostly) that you want. There will be a smidgeon of downtime during the host-to-host fail-over that may or may not be workable in your setup. The other option is to investigate the Ceph clustered filesystem. I believe there's been work ongoing this year to get it working on top of ZFS on FreeBSD. There's also GlusterFS, which has seen a bunch of work to get it working on top of ZFS on FreeBSD. No idea how well those last two work. --=20 Freddie Cash fjwcash@gmail.com