From owner-freebsd-fs@freebsd.org Tue May 17 07:47:17 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4384B3E79D for ; Tue, 17 May 2016 07:47:17 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id 98EAB1DA6; Tue, 17 May 2016 07:47:17 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id A015F45FC0CD; Tue, 17 May 2016 09:41:50 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zDk391QzatOV; Tue, 17 May 2016 09:41:48 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 602374C4C754; Tue, 17 May 2016 09:41:48 +0200 (CEST) Subject: Re: Best practice for high availability ZFS pool References: <5E69742D-D2E0-437F-B4A9-A71508C370F9@FreeBSD.org> To: Palle Girgensohn , freebsd-fs@freebsd.org From: InterNetX - Juergen Gotteswinter Reply-To: jg@internetx.com Message-ID: Date: Tue, 17 May 2016 09:41:44 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <5E69742D-D2E0-437F-B4A9-A71508C370F9@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 May 2016 07:47:17 -0000 Hi, Am 5/16/2016 um 12:08 PM schrieb Palle Girgensohn: > Hi, > > We need to set up a ZFS pool with redundance. The main goal is high availability - uptime. > > I can see a few of paths to follow. > > 1. HAST + ZFS dont do this, this has already been discussed some time ago. afaik nothing changed until this https://lists.freebsd.org/pipermail/freebsd-fs/2014-October/020084.html > > 2. Some sort of shared storage, two machines sharing a JBOD box. take care when choosing sas hba and expander, avoid sata behind sas with dual expander jbods you will be able to build an ha setup, but i highly recommend to avoid any home brew solutions. go for rsf-1. > > 3. ZFS replication (zfs snapshot + zfs send | ssh | zfs receive) > > 4. using something else than ZFS, even a different OS if required. > > My main concern with HAST+ZFS is performance. Google offer some insights here, I find mainly unsolved problems. Please share any success stories or other experiences. > performance isnt the real problem, check the older discussion mentioned above. > Shared storage still has a single point of failure, the JBOD box. Apart from that, is there even any support for the kind of storage PCI cards that support dual head for a storage box? I cannot find any. > the jbods are just a dumb piece of metal with an expander mounted. so far, i never had a broken one. > We are running with ZFS replication today, but it is just too slow for the amount of data. > replicate more often to keep the delta between each snapshot as small as possible? maybe even 10G crosslink if possible? > We prefer to keep ZFS as we already have a rather big (~30 TB) pool and also tools, scripts, backup all is using ZFS, but if there is no solution using ZFS, we're open to alternatives. Nexenta springs to mind, but I believe it is using shared storage for redundance, so it does have single points of failure? > > Any other suggestions? Please share your experience. :) > > Palle >