From owner-freebsd-fs@FreeBSD.ORG Mon Dec 31 23:48:53 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CE6A16A41B for ; Mon, 31 Dec 2007 23:48:53 +0000 (UTC) (envelope-from davidt@yadt.co.uk) Received: from outcold.yadt.co.uk (outcold.yadt.co.uk [81.187.204.178]) by mx1.freebsd.org (Postfix) with ESMTP id 0FD2F13C455 for ; Mon, 31 Dec 2007 23:48:52 +0000 (UTC) (envelope-from davidt@yadt.co.uk) Received: from localhost (localhost [127.0.0.1]) by outcold.yadt.co.uk (Postfix) with ESMTP id 7078B23BD; Mon, 31 Dec 2007 23:30:15 +0000 (GMT) X-Virus-Scanned: amavisd-new at yadt.co.uk Received: from outcold.yadt.co.uk ([127.0.0.1]) by localhost (outcold.yadt.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5ZopmFTzvQm2; Mon, 31 Dec 2007 23:30:13 +0000 (GMT) Received: by outcold.yadt.co.uk (Postfix, from userid 1001) id CED166CF5; Mon, 31 Dec 2007 23:23:19 +0000 (GMT) Date: Mon, 31 Dec 2007 23:23:19 +0000 From: David Taylor To: freebsd-fs@freebsd.org Message-ID: <20071231232319.GA90972@outcold.yadt.co.uk> Mail-Followup-To: David Taylor , freebsd-fs@freebsd.org, pjd@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) Cc: pjd@freebsd.org Subject: [PATCH] ZFS not caching on i386 with kmem_size >1GB X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2007 23:48:53 -0000 Hi, About 2 months ago I reported that I found ZFS extremely slow for some tasks (specifically upgrading ports). This was because ZFS was only using the absolute minimum cache size at all times. The problem is here in /sys/contrib/opensolaris/uts/common/fs/zfs/arc.c: static int arc_reclaim_needed(void) { ... if (kmem_used() > (kmem_size() * 4) / 5) return (1); } I'm running on i386 with kmem_size set to 1GB. As a result, the multiplication overflows and the test becomes (kmem_used() > 0). ZFS then always tries to shrink the cache, and never grows it above the absolute minimum size (about 30MB for each of c and p) The patch I have attached fixes the problem for me, although there is probably a better way to avoid the overflow (without calling kmem_size() twice). Best of all, portupgrade is now an order of magnitude faster! Of course, I'm now worried that my previously rock-solid settings will actually trigger the kmem_map too small panics when the cache actually fills up. -- David Taylor