From owner-freebsd-fs@freebsd.org Mon May 16 23:06:37 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48C8BB3E1A9 for ; Mon, 16 May 2016 23:06:37 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DEBB1111; Mon, 16 May 2016 23:06:36 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 2E8CC28426; Tue, 17 May 2016 01:00:40 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 29EC528412; Tue, 17 May 2016 01:00:39 +0200 (CEST) Message-ID: <573A5116.3090302@quip.cz> Date: Tue, 17 May 2016 01:00:38 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Palle Girgensohn , Borja Marcos CC: freebsd-fs@freebsd.org Subject: Re: Best practice for high availability ZFS pool References: <5E69742D-D2E0-437F-B4A9-A71508C370F9@FreeBSD.org> <89D73122-FAC7-4449-AAB3-C4BBE74B960A@FreeBSD.org> In-Reply-To: <89D73122-FAC7-4449-AAB3-C4BBE74B960A@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2016 23:06:37 -0000 Palle Girgensohn wrote on 05/17/2016 00:36: > >> 16 maj 2016 kl. 15:51 skrev Borja Marcos : >> >> >>> On 16 May 2016, at 12:08, Palle Girgensohn wrote: >>> >>> Hi, >>> >>> We need to set up a ZFS pool with redundance. The main goal is high availability - uptime. >>> >>> I can see a few of paths to follow. >>> >>> 1. HAST + ZFS >> >> Which means that a possible corruption causing bug in ZFS would vaporize the data of both replicas. >> >>> 3. ZFS replication (zfs snapshot + zfs send | ssh | zfs receive) >> >> If you don’t have a hard requirement for synchronous replication (and, in that case, I would opt for a more application >> aware approach) it’s the best method in my opinion. > > That was exactly my thought 18 months ago, and we set up two systems with zfs snapshot + zfs send | ssh | zfs receive. It works, but the problem is it just too slow and a complete sync takes like 10 minutes for all the file systems. We are forced to sync the file systems one at a time to get the kind of control and separation we need. Even if we could speed that up somehow, we are really looking for a more recilient system. Also, constant snapshotting and writing makes scrub very slow so we need to tune down the amount of syncing every fourth week-end to scrub. It's OK but not optimal, so we're pondering for something better. > > My first choice is really HAST at the moment, but I also dont find much written in the last couple of years, apart from some articles about setting it up in very minimal testbeds or posts about performance and stability troubles. This makes me wonder, is HAST actively maintained? Is it stable, used and loved by the community? I'd love to hear some success stories with farily large installations of at least 20 TB or so. I am not using HAST personally but I read about success with HAST and ZFS somewhere in FreeBSD mailing lists. I don't have a direct link / bookmark for it. Maybe you will find it thru search engine. Miroslav Lachman