From owner-freebsd-stable@FreeBSD.ORG Wed Dec 29 00:40:38 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03DD61065670 for ; Wed, 29 Dec 2010 00:40:38 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta06.emeryville.ca.mail.comcast.net (qmta06.emeryville.ca.mail.comcast.net [76.96.30.56]) by mx1.freebsd.org (Postfix) with ESMTP id 9E27D8FC18 for ; Wed, 29 Dec 2010 00:40:37 +0000 (UTC) Received: from omta15.emeryville.ca.mail.comcast.net ([76.96.30.71]) by qmta06.emeryville.ca.mail.comcast.net with comcast id oj5j1f0051Y3wxoA6ogcu8; Wed, 29 Dec 2010 00:40:36 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta15.emeryville.ca.mail.comcast.net with comcast id ooga1f00U2tehsa8bogb8y; Wed, 29 Dec 2010 00:40:35 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 912279B427; Tue, 28 Dec 2010 16:40:34 -0800 (PST) Date: Tue, 28 Dec 2010 16:40:34 -0800 From: Jeremy Chadwick To: Freek van Hemert Message-ID: <20101229004034.GA18233@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: slow ZFS on FreeBSD 8.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Dec 2010 00:40:38 -0000 On Wed, Dec 29, 2010 at 12:56:57AM +0100, Freek van Hemert wrote: > Hello everyone, > > This is my first mail on the mailinglist and I very much appreciate this > option of getting some help. > > I have a question regarding zfs on freebsd. > (I'm making a home server) > This afternoon I did a "zpool create data mirror ad4 ad6" Now I'm copying > things from my ufs2 disk into the 2TB zpool, it is very slow. I'm on freebsd > 8.1 amd64 on an atom n330 with 2 sata disks, gstat tells me I'm going at > around 2 mbps at near 100 %busy while the ufs2 drives are near 0. Also, > ufs2 to ufs2 was much faster (I estimate about 10 times faster). How do I > tune? The wiki is not helpful for amd64 users stating that the defaults > should be optimal. I'm using the 8.1-stable version which has just been > installed this afternoon from a minimal install iso. > > I'm copying from a single ufs2 pata drive into a sata zpool mirror. I have > 2GB of ram. > > Top tells me: 761 MB Active, 790 Inactive and there is hardly any cpu usage > (96-98% idle). vfs.numvnodes: around 12500 now (after several hours of > copying) and stil slowly rising. For your system. /boot/loader.conf: # Increase vm.kmem_size to allow for ZFS ARC to utilise more memory. vm.kmem_size="2048M" vfs.zfs.arc_max="1024M" # Disable ZFS prefetching # http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html # Increases overall speed of ZFS, but when disk flushing/writes occur, # system is less responsive (due to extreme disk I/O). # NOTE: Systems with 8GB of RAM or more have prefetch enabled by # default. vfs.zfs.prefetch_disable="1" # Decrease ZFS txg timeout value from 30 (default) to 5 seconds. This # should increase throughput and decrease the "bursty" stalls that # happen during immense I/O with ZFS. # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html vfs.zfs.txg.timeout="5" /etc/sysctl.conf: # Increase number of vnodes; we've seen vfs.numvnodes reach 115,000 # at times. Default max is a little over 200,000. Playing it safe... kern.maxvnodes=250000 # Set TXG write limit to a lower threshold. This helps "level out" # the throughput rate (see "zpool iostat"). A value of 256MB works well # for systems with 4GB of RAM, while 1GB works well for us w/ 8GB. # We're using 1GB here because of the ada3 disk which can really handle # a massive amount of write speed. vfs.zfs.txg.write_limit_override=1073741824 Note that the last entry in sysctl.conf may need to be adjusted, or commented out entirely. It's up to you. You can mess with this in real-time using sysctl. Google for vfs.zfs.txg.write_limit_override to get an idea of how to adjust it, if you want to. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |