From owner-freebsd-current@FreeBSD.ORG Sun Nov 4 13:18:37 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA8DB16A469 for ; Sun, 4 Nov 2007 13:18:37 +0000 (UTC) (envelope-from davidt@yadt.co.uk) Received: from outcold.yadt.co.uk (outcold.yadt.co.uk [81.187.204.178]) by mx1.freebsd.org (Postfix) with ESMTP id 08C0413C4B6 for ; Sun, 4 Nov 2007 13:18:36 +0000 (UTC) (envelope-from davidt@yadt.co.uk) Received: from localhost (localhost [127.0.0.1]) by outcold.yadt.co.uk (Postfix) with ESMTP id 4F46E68C7 for ; Sun, 4 Nov 2007 13:18:19 +0000 (GMT) X-Virus-Scanned: amavisd-new at yadt.co.uk Received: from outcold.yadt.co.uk ([127.0.0.1]) by localhost (outcold.yadt.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PzFS2rJ8SyFo for ; Sun, 4 Nov 2007 13:18:09 +0000 (GMT) Received: by outcold.yadt.co.uk (Postfix, from userid 1001) id C974868BB; Sun, 4 Nov 2007 13:18:08 +0000 (GMT) Date: Sun, 4 Nov 2007 13:18:08 +0000 From: David Taylor To: freebsd-current@freebsd.org Message-ID: <20071104131808.GA21441@outcold.yadt.co.uk> Mail-Followup-To: freebsd-current@freebsd.org References: <200711021208.25913.Thomas.Sparrevohn@btinternet.com> <20071103164231.GB23714@outcold.yadt.co.uk> <200711040948.25732.peter.schuller@infidyne.com> <20071104110521.GA12145@outcold.yadt.co.uk> <472DB78E.5030103@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <472DB78E.5030103@FreeBSD.org> User-Agent: Mutt/1.4.2.3i Subject: Re: ZFS slowness (not using cache?) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2007 13:18:37 -0000 On Sun, 04 Nov 2007, Kris Kennaway wrote: > David Taylor wrote: >> >>Hmm. That prompted me to have a look at the arcstats, and I'm now rather >>confused. It seems to have plenty of cache free, but not actually using >>it properly. >> >>I'm running i386 for now (on an amd64 motherboard), with 4GB of RAM >>(~3.5Gb usable) and the following settings in loader.conf: [snip] >> >>kern.maxdsiz="900M" >>vm.kmem_size=1450M >>vfs.zfs.arc_max=500M >>vfs.root.mountfrom="zfs:tank/i386_root" >> >>This results in (sysctl): >> >>vm.kmem_size_scale: 3 >>vm.kmem_size_max: 335544320 # I have just noticed this is about 300MB, >> # far lower than vm.kmem_size. Is that a >> problem? > > No but you can't set kmem to 1500MB without first increasing the kernel > address space (1GB on i386) to some suitably larger value via KVA_PAGES. Sorry, I should have mentioned that I have KVA_PAGES=512. >>vm.kmem_size_min: 0 >>vm.kmem_size: 1520435200 >> >>vfs.zfs.arc_min: 47513600 >>vfs.zfs.arc_max: 524288000 >>kstat.zfs.misc.arcstats.c_min: 47513600 >>kstat.zfs.misc.arcstats.c_max: 524288000 >>kstat.zfs.misc.arcstats.size: 86967808 >> >>Regardless of what I do, I can't seem to get arcstats.size to exceed >>100MB. It initially goes up with disk usage, but then starts to >>drop again, causing it to hold steady around 70-90MB. Something >>seems to be agressively pushing data out of the cache, despite it >>being <20% full, which seems rather fishy. >>Does anyone know what's going on? > > Try increasing kern.maxvnodes. Increasing this from 100k to 1M seems to have made no difference. vfs.numvnodes is now growing to 150k and beyond, but arcstats.size is still sitting at ~70MB. -- David Taylor