From owner-freebsd-questions@FreeBSD.ORG Tue Jul 16 09:42:03 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0695DA0D for ; Tue, 16 Jul 2013 09:42:03 +0000 (UTC) (envelope-from FreeBSD@shaneware.biz) Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [IPv6:2001:44b8:8060:ff02:300:1:2:6]) by mx1.freebsd.org (Postfix) with ESMTP id 7194FA8D for ; Tue, 16 Jul 2013 09:42:02 +0000 (UTC) Received: from ppp247-71.static.internode.on.net (HELO leader.local) ([203.122.247.71]) by ipmail06.adl2.internode.on.net with ESMTP; 16 Jul 2013 19:11:48 +0930 Message-ID: <51E51558.50302@ShaneWare.Biz> Date: Tue, 16 Jul 2013 19:11:44 +0930 From: Shane Ambler User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130516 Thunderbird/17.0.6 MIME-Version: 1.0 To: aurfalien Subject: Re: to gmirror or to ZFS References: <4DFBC539-3CCC-4B9B-AB62-7BB846F18530@gmail.com> <976836C5-F790-4D55-A80C-5944E8BC2575@gmail.com> In-Reply-To: <976836C5-F790-4D55-A80C-5944E8BC2575@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jul 2013 09:42:03 -0000 On 16/07/2013 14:41, aurfalien wrote: > > On Jul 15, 2013, at 9:23 PM, Warren Block wrote: > >> On Mon, 15 Jul 2013, aurfalien wrote: >> >>> ... thats the question :) >>> >>> At any rate, I'm building a rather large 100+TB NAS using ZFS. >>> >>> However for my OS, should I also ZFS or simply gmirror as I've a >>> dedicated pair of 256GB SSD drives for it. I didn't ask for SSD >>> sys drives, this system just came with em. >>> >>> This is more of a best practices q. >> >> ZFS has data integrity checking, gmirror has low RAM overhead. >> gmirror is, at present, restricted to MBR partitioning due to >> metadata conflicts with GPT, so 2TB is the maximum size. >> >> Best practices... depends on your use. gmirror for the system >> leaves more RAM for ZFS. > > Perfect, thanks Warren. > > Just what I was looking for. I doubt that you would save any ram having the os on a non-zfs drive as you will already be using zfs chances are that non-zfs drives would only increase ram usage by adding a second cache. zfs uses it's own cache system and isn't going to share it's cache with other system managed drives. I'm not actually certain if the system cache still sits above zfs cache or not, I think I read it bypasses the traditional drive cache. For zfs cache you can set the max usage by adjusting vfs.zfs.arc_max that is a system wide setting and isn't going to increase if you have two zpools. Tip: set the arc_max value - by default zfs will use all physical ram for cache, set it to be sure you have enough ram left for any services you want running. Have you considered using one or both SSD drives with zfs? They can be added as cache or log devices to help performance. See man zpool under Intent Log and Cache Devices.