From owner-freebsd-stable@FreeBSD.ORG Fri Jan 22 11:44:56 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACF2910656A6 for ; Fri, 22 Jan 2010 11:44:56 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 36CBD8FC20 for ; Fri, 22 Jan 2010 11:44:56 +0000 (UTC) Received: from outgoing.leidinger.net (pD954F1B8.dip.t-dialin.net [217.84.241.184]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id B2A8884425B; Fri, 22 Jan 2010 12:44:39 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 2D2467EE75; Thu, 21 Jan 2010 03:32:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1264041142; bh=xJTljPu0kpgaGROK69sGr2HU5gvbQ6ZxriQntnG0V0A=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=z9lqYUATaGwm6gYMVK4ry7zo/GGK6xhfhkaqAMuGd4jsflRFcce77KfFbSSkPj3Kz Sj8o98QafHZvRecXUX8tygs6882nV7A/PladxrYE068+zsvsZsVENIrysnZpua3hQy dJeQ8hTIJN6pOCNDnsNwIgyzy3tjLGugGJN8fZfT6dH8kDcIRExRqwgXIUBHRP7Gmf KWxFjTTLGWSfBMM0AggXPqWR8pfdLVVCk2jJcf5n7HV9jfpP9YTX1CB9DGNByYkfpF Z+ZL4a55sf0jDp34xNqatGL4GBp9jsZidO5JvIvHbAfJvASOGhYi4SYI2a3y8s1Etp Q8PVviDMhwPEQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o0KAlUGw061051; Wed, 20 Jan 2010 11:47:30 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 20 Jan 2010 11:47:30 +0100 Message-ID: <20100120114730.916917ef0eu668n4@webmail.leidinger.net> Date: Wed, 20 Jan 2010 11:47:30 +0100 From: Alexander Leidinger To: Jeremy Chadwick References: <7346c5c61001030842r7dc76199y51e4c1c90a3eea6e@mail.gmail.com> <7346c5c61001091706m45a3a2a5k3ca8bb0c4bec5ea8@mail.gmail.com> <7346c5c61001171521w1ca4738w98e8fcca24643cda@mail.gmail.com> <201001180829.48126.npapke@acm.org> <7346c5c61001190840k31466754i32b2ae833390b79b@mail.gmail.com> <20100119170101.GA80917@icarus.home.lan> In-Reply-To: <20100119170101.GA80917@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: B2A8884425B.65725 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1264765483.82864@0Ktajn9IaGqLN1pEkwB5Ww X-EBL-Spam-Status: No Cc: freebsd-stable@FreeBSD.org Subject: Re: ZFS performance degradation over time X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 11:44:56 -0000 Quoting Jeremy Chadwick (from Tue, 19 Jan 2010 09:01:01 -0800): > On Tue, Jan 19, 2010 at 11:40:50AM -0500, Garrett Moore wrote: >> I've been watching my memory usage and I have no idea what is consuming >> memory as 'Active'. >> >> Last night I had around 6500MB 'Active' again, 1500MB Wired, no inact, ~30MB >> buf, no free, and ~100MB swap used. My performance copying ZFS->ZFS was >> again slow (<1MB/s). I tried killing rTorrent and no significant amount of >> memory was reclaimed - maybe 100MB. `ps aux` showed no processes using any >> significant amount of memory, and I was definitely nowhere near 6500MB >> usage. >> >> I tried running the perl oneliner again to hog a bunch of memory, and almost >> all of the Active memory was IMMEDIATELY marked as Free, and my performance >> was excellent again. >> >> I'm not sure what in userland could be causing the issue. The only things >> I've installed are rTorrent, lighttpd, samba, smartmontools, vim, bash, >> Python, Perl, and SABNZBd. There is nothing that *should* be consuming any >> serious amount of memory. > > I've two recommendations: > > 1) Have you considered "upgrading" to RELENG_8 (e.g. 8.0-STABLE) instead > of sticking with 8.0-RELEASE? There's been a recent MFC to RELENG_8 > which pertain to ARC drainage. I'm referring to the commit labelled > revision 1.22.2.2 (RELENG_8): > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c This patch can be merged stand-alone if necessary, no need to go to RELENG_8 if there are reservations. > 2) Have you tried using vfs.zfs.arc_max in loader.conf to limit the ARC > size? I'd recommend picking something like 1GB as a cap (your machine Or even less... to be determined by experimenting. > has 8GB total at present, if I remember right). I believe long ago > someone said this isn't an explicit hard limit on the maximum size of > the ARC, but I believe this was during the RELENG_7 days and the ARC > "stuff" on FreeBSD has changed since then. I wish the tunables were > better documented, or at least explained in detail (hello Wiki!). The commit you refer to above is just doing this: limiting the arc more to the arc_max than it was the case before. This patch is in 7-stable too (in case someone is interested). Bye, Alexander. -- Johnson's First Law: When any mechanical contrivance fails, it will do so at the most inconvenient possible time. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137