From owner-freebsd-questions@FreeBSD.ORG Thu Jun 21 12:39:14 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52E031065675 for ; Thu, 21 Jun 2012 12:39:14 +0000 (UTC) (envelope-from freebsd@pki2.com) Received: from btw.pki2.com (btw.pki2.com [IPv6:2001:470:a:6fd::2]) by mx1.freebsd.org (Postfix) with ESMTP id 03E6B8FC08 for ; Thu, 21 Jun 2012 12:39:13 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by btw.pki2.com (8.14.5/8.14.5) with ESMTP id q5LCd2t0074087; Thu, 21 Jun 2012 05:39:02 -0700 (PDT) (envelope-from freebsd@pki2.com) From: Dennis Glatting To: Hooman Fazaeli In-Reply-To: <4FE2CE38.9000100@gmail.com> References: <4FE2CE38.9000100@gmail.com> Content-Type: text/plain; charset="ISO-8859-1" Date: Thu, 21 Jun 2012 05:39:02 -0700 Message-ID: <1340282342.19296.47.camel@btw.pki2.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-yoursite-MailScanner-Information: Dennis Glatting X-yoursite-MailScanner-ID: q5LCd2t0074087 X-yoursite-MailScanner: Found to be clean X-MailScanner-From: freebsd@pki2.com Cc: FreeBSD Questions Subject: Re: Is ZFS production ready? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jun 2012 12:39:14 -0000 On Thu, 2012-06-21 at 12:03 +0430, Hooman Fazaeli wrote: > Dear community > > In the past, I built a 8TB ZFS log server on freebsd 7.4. > However, the system experienced instablility after long up times. > My main motive to use ZFS was UFS inability to support large > file systems. > > Now, I want to the same thing on 8.3 and wanted to know > your opinion on ZFS stability. Is there any success story using > ZFS in 24x7, large volume, heavy duty servers? Is there any > other option other than ZFS to build larger than 2TB file systems? > System 1: 32 cores, Interlagos, 64GB, 18TB RAIDz1 System 2: 64 cores, Interlagos, 128GB, 15TB RAIDz1 System 3: 8 cores, Bulldozer, 16GB, 27TB RAIDz2 Those are the main volumes on those systems. There are smaller ZFS volumes. I have other systems also ZFS (total of seven systems), typically with one or more 5-10TB RAIDz1 volumes. All systems RELENG_9. Stable? Yes. Be sure you have up-to-date FreeBSD kernel and your HBA firmware is up-to-date. Generally I use LSI 9211 cards. That said, the weak point is the drives. For example, one system has 6 Hitachi 4TB drives and there are three more in other systems -- 30% failure rate within one year. I've also had several failures with Seagate drives across two years. Zero failures with WD drives (12 drives in one ZFS array, IIRC) however those are slower, cheap drives. I am also working with compressed volumes because my data is very large and highly compressable. Compressed volumes, as you would expect, has a significant kernel performance impact depending on what you are doing.