From owner-freebsd-fs@FreeBSD.ORG  Wed Jan  2 11:09:46 2008
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id B81F016A41A;
	Wed,  2 Jan 2008 11:09:46 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id BC84713C458;
	Wed,  2 Jan 2008 11:09:45 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Message-ID: <477B70F9.8070903@FreeBSD.org>
Date: Wed, 02 Jan 2008 12:09:45 +0100
From: Kris Kennaway <kris@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031)
MIME-Version: 1.0
To: David Taylor <davidt@yadt.co.uk>, freebsd-fs@freebsd.org, pjd@freebsd.org
References: <20071231232319.GA90972@outcold.yadt.co.uk>
In-Reply-To: <20071231232319.GA90972@outcold.yadt.co.uk>
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Cc: 
Subject: Re: [PATCH] ZFS not caching on i386 with kmem_size >1GB
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 02 Jan 2008 11:09:46 -0000

David Taylor wrote:
> Hi,
> 
> About 2 months ago I reported that I found ZFS extremely slow for
> some tasks (specifically upgrading ports).  This was because ZFS
> was only using the absolute minimum cache size at all times.
> 
> The problem is here in /sys/contrib/opensolaris/uts/common/fs/zfs/arc.c:
> 
> static int
> arc_reclaim_needed(void)
> {
> ...
>         if (kmem_used() > (kmem_size() * 4) / 5)
>                 return (1);
> }
> 
> I'm running on i386 with kmem_size set to 1GB.  As a result, the
> multiplication overflows and the test becomes (kmem_used() > 0).  ZFS then
> always tries to shrink the cache, and never grows it above the absolute
> minimum size (about 30MB for each of c and p)
> 
> The patch I have attached fixes the problem for me, although there is probably
> a better way to avoid the overflow (without calling kmem_size() twice).
> Best of all, portupgrade is now an order of magnitude faster!
> 
> Of course, I'm now worried that my previously rock-solid settings will actually
> trigger the kmem_map too small panics when the cache actually fills up.
> 

FYI, kmem_size > 1GB makes no sense unless you also increase KVA_PAGES 
since the entire kernel only has 1GB of address space on i386.

Kris