From owner-freebsd-fs@FreeBSD.ORG Mon Jul 12 12:47:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 752CE106566B for ; Mon, 12 Jul 2010 12:47:49 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.westchester.pa.mail.comcast.net (qmta09.westchester.pa.mail.comcast.net [76.96.62.96]) by mx1.freebsd.org (Postfix) with ESMTP id 1F4988FC15 for ; Mon, 12 Jul 2010 12:47:48 +0000 (UTC) Received: from omta09.westchester.pa.mail.comcast.net ([76.96.62.20]) by qmta09.westchester.pa.mail.comcast.net with comcast id h0SQ1e0040SCNGk590npju; Mon, 12 Jul 2010 12:47:49 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta09.westchester.pa.mail.comcast.net with comcast id h0no1e0013LrwQ23V0noBa; Mon, 12 Jul 2010 12:47:49 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id C58A19B425; Mon, 12 Jul 2010 05:47:46 -0700 (PDT) Date: Mon, 12 Jul 2010 05:47:46 -0700 From: Jeremy Chadwick To: S M Message-ID: <20100712124746.GA9328@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: zfs on 4k sector disks X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2010 12:47:49 -0000 On Mon, Jul 12, 2010 at 12:25:02PM +0000, S M wrote: > > I found that playing with either: > > (sysctl) vfs.zfs.txg.write_limit_override > > or > > (boot/loader.conf) vfs.zfs.txg.timeout=5 > (boot/loader.conf) vfs.zfs.txg.synctime=5 > > Helped with the write problem. What seems to happen is that zfs' calculations go slightly wrong if you have a lot of RAM and therefore zfs buffers too much, forcing so many writes that you lose all read IO for a few seconds.... This has little to do with 4KB sector disks, as it applies to 512-byte sector disks too. Some references: http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html The values chosen (in your case, "5") greatly depend on the model and type of hard disk used. I stress the word "greatly". Others reading this mail should take that into consideration. The default values (meaning out-of-the-box) for these tunables is as follows (taken from RELENG_8's source as of a few minutes ago). I'm also including what source files were used to verify said claims. For a description of what these do, see sysctl -d. vfs.zfs.txg.synctime = 5 - src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c - src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c vfs.zfs.txg.timeout = 30 - src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c vfs.zfs.txg.write_limit_override = 0 - src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c - src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c The defaults may vary per person, based on what source branch/tag they use (ex. RELENG_8 vs. RELENG_8_0 vs. RELENG_7) and what build date your kernel is (since what's in /usr/src might not match the running kernel). To see if these values have changed over time (e.g. in an older release they may have been different), one should refer to cvsweb or equivalent. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |