From owner-freebsd-questions@FreeBSD.ORG Fri Jul 19 18:25:33 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6A7DB52C for ; Fri, 19 Jul 2013 18:25:33 +0000 (UTC) (envelope-from aurfalien@gmail.com) Received: from mail-pb0-x235.google.com (mail-pb0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 45A73A9A for ; Fri, 19 Jul 2013 18:25:33 +0000 (UTC) Received: by mail-pb0-f53.google.com with SMTP id xb12so4739805pbc.12 for ; Fri, 19 Jul 2013 11:25:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=tt3cdMbTd7EYQyJNDN9P028OelhlertT+np+YhQM+k8=; b=IcETJrt0OwCLKQh5FDvfk9I26sd0jhz1O2+KFT+tpdDZBrxUPUtLB+2wKwaaDbMLQy phwrqX0Iv4tcTFP6MIq3wzx1qNK4iVNQDsyngyd08WRkqCD25+rolVTZsp/r7006ZIVW W34QCm+QERhNv5HwpXaiG+Zo74PvtKfjY2aGZnG37vg+1jDksLKp2hjw3dhk3S/O/U3m CVofx3rFVGb+rVK9TTbReJAp3LwT+BsV2cC5DrfRCjjS/oIl4B+ly4qabRIHNexa8aki Gm6LMt7HxaxGk5pW8EyXnFqWS8irvipawEcbE27GnhK4Zt1lxkFiqSCPCbj76VXeDh24 jMoA== X-Received: by 10.68.224.228 with SMTP id rf4mr18588823pbc.50.1374258332993; Fri, 19 Jul 2013 11:25:32 -0700 (PDT) Received: from briankrusicw.logan.tv ([64.17.255.138]) by mx.google.com with ESMTPSA id ib9sm20801418pbc.43.2013.07.19.11.25.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 19 Jul 2013 11:25:31 -0700 (PDT) Subject: Re: to gmirror or to ZFS Mime-Version: 1.0 (Apple Message framework v1085) Content-Type: text/plain; charset=us-ascii From: aurfalien In-Reply-To: Date: Fri, 19 Jul 2013 11:25:28 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <069F4A27-A7A2-4215-A815-468F436B331F@gmail.com> References: <4DFBC539-3CCC-4B9B-AB62-7BB846F18530@gmail.com> <976836C5-F790-4D55-A80C-5944E8BC2575@gmail.com> <51E51558.50302@ShaneWare.Biz> To: Warren Block X-Mailer: Apple Mail (2.1085) Cc: freebsd-questions@freebsd.org, Shane Ambler X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2013 18:25:33 -0000 On Jul 16, 2013, at 11:42 AM, Warren Block wrote: > On Tue, 16 Jul 2013, aurfalien wrote: >> On Jul 16, 2013, at 2:41 AM, Shane Ambler wrote: >>>=20 >>> I doubt that you would save any ram having the os on a non-zfs drive = as >>> you will already be using zfs chances are that non-zfs drives would = only >>> increase ram usage by adding a second cache. zfs uses it's own cache >>> system and isn't going to share it's cache with other system managed >>> drives. I'm not actually certain if the system cache still sits = above >>> zfs cache or not, I think I read it bypasses the traditional drive = cache. >>>=20 >>> For zfs cache you can set the max usage by adjusting vfs.zfs.arc_max >>> that is a system wide setting and isn't going to increase if you = have >>> two zpools. >>>=20 >>> Tip: set the arc_max value - by default zfs will use all physical = ram >>> for cache, set it to be sure you have enough ram left for any = services >>> you want running. >>>=20 >>> Have you considered using one or both SSD drives with zfs? They can = be >>> added as cache or log devices to help performance. >>> See man zpool under Intent Log and Cache Devices. >>=20 >> This is a very interesting point. >>=20 >> In terms if SSDs for cache, I was planning on using a pair of Samsung = Pro 512GB SSDs for this purpose (which I haven't bought yet). >>=20 >> But I tire of buying stuff, so I have a pair of 40GB Intel SSDs for = use as sys disks and several Intel 160GB SSDs lying around that I can = combine with the existing 256GB SSDs for a cache. >>=20 >> Then use my 36x3TB for the beasty NAS. >=20 > Agreed that 256G mirrored SSDs are kind of wasted as system drives. = The 40G mirror sounds ideal. Update; I went with ZFS as I didn't want to confuse the toolset needed to = support this server. Although gmirror is not hard to figure out, I = wanted consistency in systems. So I've a booted 9.1 rel using a mirrored ZFS system disk. The drives do support TRIM but am unsure how this plays with ZFS. I did = the standard partition scheme of; root@kronos:/root # gpart show =3D> 34 78165293 da0 GPT (37G) 34 128 1 freebsd-boot (64k) 162 6 - free - (3.0k) 168 8388608 2 freebsd-swap (4.0G) 8388776 69776544 3 freebsd-zfs (33G) 78165320 7 - free - (3.5k) =3D> 34 78165293 da1 GPT (37G) 34 128 1 freebsd-boot (64k) 162 6 - free - (3.0k) 168 8388608 2 freebsd-swap (4.0G) 8388776 69776544 3 freebsd-zfs (33G) 78165320 7 - free - (3.5k) At any rate, thank you for the replies, very much appreciate it. Especially since building a rather large production worthy NAS not = knowing a lick of freeBSD. The reasons going with freeBSD are 2 fold; ZFS stability,seems a better marriage then ZOL. Correctly provides NFS pre attributes on write reply; mtime. Linux does = not. While its a steep learning curve, the 2 points above require the use of = freeBSD or alike. - aurf=