From owner-freebsd-questions@FreeBSD.ORG Tue Jul 16 17:49:47 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 51545EED for ; Tue, 16 Jul 2013 17:49:47 +0000 (UTC) (envelope-from aurfalien@gmail.com) Received: from mail-pb0-x235.google.com (mail-pb0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 2C0D3814 for ; Tue, 16 Jul 2013 17:49:47 +0000 (UTC) Received: by mail-pb0-f53.google.com with SMTP id xb12so940319pbc.40 for ; Tue, 16 Jul 2013 10:49:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=kqC0kVuth5ABtxd3YD9PcBtZ8WRb/3GGbNVgE59gp7c=; b=VIFp1a9DteiCwx238wzh/ojLm3ymPUbDyb7xmstfX1Qd1vH5xgcKbEM+XUfEjjEo5S xyQFuccmed8ARXOGhsGOJC80LG7vCODBrTyoTWKHEPDDOvK2Uw4pa0USZUPg/yZI6bLR z6Y3/DH6ummMyCjM9vfnoQlubRZ28igwCKZ8v+2eMkuCVoXjktvfHdDimWt+NL6D8qw6 UfdbNpu6M6jzcsJuHWCEfn76EpFu2E3HIf+FVQoBZAh7uu/WcFbqVY0YWgC5mU2aTXlD x06e1BMM+Cg9l0CHQeHcO+omEVQM+5aI7z6ngEms7eVNHYwvUv9J3eaDF8b7eK069b1m opvg== X-Received: by 10.66.145.4 with SMTP id sq4mr3819721pab.46.1373996986989; Tue, 16 Jul 2013 10:49:46 -0700 (PDT) Received: from briankrusicw.logan.tv ([64.17.255.138]) by mx.google.com with ESMTPSA id jb2sm3158087pbc.8.2013.07.16.10.49.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 16 Jul 2013 10:49:45 -0700 (PDT) Subject: Re: to gmirror or to ZFS Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: aurfalien In-Reply-To: <51E51558.50302@ShaneWare.Biz> Date: Tue, 16 Jul 2013 10:49:42 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4DFBC539-3CCC-4B9B-AB62-7BB846F18530@gmail.com> <976836C5-F790-4D55-A80C-5944E8BC2575@gmail.com> <51E51558.50302@ShaneWare.Biz> To: Shane Ambler X-Mailer: Apple Mail (2.1085) Cc: freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 17:49:47 -0000 On Jul 16, 2013, at 2:41 AM, Shane Ambler wrote: > On 16/07/2013 14:41, aurfalien wrote: >>=20 >> On Jul 15, 2013, at 9:23 PM, Warren Block wrote: >>=20 >>> On Mon, 15 Jul 2013, aurfalien wrote: >>>=20 >>>> ... thats the question :) >>>>=20 >>>> At any rate, I'm building a rather large 100+TB NAS using ZFS. >>>>=20 >>>> However for my OS, should I also ZFS or simply gmirror as I've a >>>> dedicated pair of 256GB SSD drives for it. I didn't ask for SSD >>>> sys drives, this system just came with em. >>>>=20 >>>> This is more of a best practices q. >>>=20 >>> ZFS has data integrity checking, gmirror has low RAM overhead. >>> gmirror is, at present, restricted to MBR partitioning due to >>> metadata conflicts with GPT, so 2TB is the maximum size. >>>=20 >>> Best practices... depends on your use. gmirror for the system >>> leaves more RAM for ZFS. >>=20 >> Perfect, thanks Warren. >>=20 >> Just what I was looking for. >=20 > I doubt that you would save any ram having the os on a non-zfs drive = as > you will already be using zfs chances are that non-zfs drives would = only > increase ram usage by adding a second cache. zfs uses it's own cache > system and isn't going to share it's cache with other system managed > drives. I'm not actually certain if the system cache still sits above > zfs cache or not, I think I read it bypasses the traditional drive = cache. >=20 > For zfs cache you can set the max usage by adjusting vfs.zfs.arc_max > that is a system wide setting and isn't going to increase if you have > two zpools. >=20 > Tip: set the arc_max value - by default zfs will use all physical ram > for cache, set it to be sure you have enough ram left for any services > you want running. >=20 > Have you considered using one or both SSD drives with zfs? They can be > added as cache or log devices to help performance. > See man zpool under Intent Log and Cache Devices. This is a very interesting point. In terms if SSDs for cache, I was planning on using a pair of Samsung = Pro 512GB SSDs for this purpose (which I haven't bought yet). But I tire of buying stuff, so I have a pair of 40GB Intel SSDs for use = as sys disks and several Intel 160GB SSDs lying around that I can = combine with the existing 256GB SSDs for a cache. Then use my 36x3TB for the beasty NAS. - aurf