From owner-freebsd-stable@FreeBSD.ORG Tue Mar 8 11:48:12 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EFF341065672 for ; Tue, 8 Mar 2011 11:48:12 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta06.emeryville.ca.mail.comcast.net (qmta06.emeryville.ca.mail.comcast.net [76.96.30.56]) by mx1.freebsd.org (Postfix) with ESMTP id D79FE8FC1D for ; Tue, 8 Mar 2011 11:48:12 +0000 (UTC) Received: from omta04.emeryville.ca.mail.comcast.net ([76.96.30.35]) by qmta06.emeryville.ca.mail.comcast.net with comcast id Gbno1g0030lTkoCA6boCHj; Tue, 08 Mar 2011 11:48:12 +0000 Received: from koitsu.dyndns.org ([98.248.33.18]) by omta04.emeryville.ca.mail.comcast.net with comcast id GboB1g0060PUQVN8QboBGJ; Tue, 08 Mar 2011 11:48:12 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 61A849B422; Tue, 8 Mar 2011 03:48:10 -0800 (PST) Date: Tue, 8 Mar 2011 03:48:10 -0800 From: Jeremy Chadwick To: "Patrick M. Hausen" Message-ID: <20110308114810.GA37554@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable Subject: Re: ZFS performance as the FS fills up? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Mar 2011 11:48:13 -0000 On Tue, Mar 08, 2011 at 12:26:49PM +0100, Patrick M. Hausen wrote: > we use a big JBOD and ZFS with raidz2 as the target > for our nightly Amanda backups. > > I already suspected that the fact that the FS was > 90% full might > be the cause of our backup performance continously decreasing. > > I just added another vdev - 6 disks of 750 GB each, raidz2 and the > FS usage is back to 71% currently. This was while backups were > running and write performance instantly skyrocketed compared to > the values before. > > So, is it possible to name a reasonable amount of free space to > keep on a raidz2 volume? On last year's EuroBSDCon I got > the impression that with recent (RELENG_8) ZFS merges > I could get away with using around 90%. I'm in no way attempting to dissuade you from your efforts to figure out a good number for utilisation, but when I hear of disks -- no matter how many -- being 90% full, I immediately conclude performance is going to suck simply because the outer "tracks" on a disk contains more sectors than the inner "tracks". This is the reason for performance degradation as the seek offset increases, resulting in graphs like this: http://img171.imageshack.us/img171/4776/1tb2.png Given this info, it shouldn't come as too much of a surprise that adding another vdev (effectively adding more disks to the pool) greatly helps in relieving this issue. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |