From owner-freebsd-fs@FreeBSD.ORG Thu Dec 30 14:37:40 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23BD7106564A for ; Thu, 30 Dec 2010 14:37:40 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EC3748FC0A for ; Thu, 30 Dec 2010 14:37:39 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A64AC46B06 for ; Thu, 30 Dec 2010 09:37:39 -0500 (EST) Received: from John-Baldwins-Macbook-Pro.local (c-68-36-150-83.hsd1.nj.comcast.net [68.36.150.83]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 45E9D8A009 for ; Thu, 30 Dec 2010 09:37:38 -0500 (EST) Message-ID: <4D1C9931.2060209@FreeBSD.org> Date: Thu, 30 Dec 2010 09:37:37 -0500 From: John Baldwin User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20101229221801.GA48661@cabstand.com> In-Reply-To: <20101229221801.GA48661@cabstand.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Thu, 30 Dec 2010 09:37:38 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-0.9 required=4.2 tests=BAYES_00,RDNS_DYNAMIC autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Subject: Re: how big can kern.maxvnodes get? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Dec 2010 14:37:40 -0000 Chris Peiffer wrote: > I have a backend server running 8-2-PRERELEASE with lots of > independent files that randomly grow and then get truncated to > zero. (Think popserver.) > > Roughly 2.5 million inodes on each of 4 Intel SSD disks. 24 gb of RAM > in the system. I want to maximize the buffer cache in RAM. > > I doubled kern.maxvnodes to 942108 and reads/second went down and > memory use went up, (as I expected) but right now there's still about > 15g RAM marked as free. > > vfs.numvnodes crept up to 821704 and has hovered there. The file > sizes range to 1 mb but most are in the range 0-10k. Since the server > operations are so simple kern.openfiles hovers in the range 100-200. > > Obviously, all things being equal I'd like to give the filesystem > buffer cache access to that free RAM by allowing more vnodes to stay > cached. > > Can I increase kern.maxvnodes by another factor of 2? more? Are there > any known problems with stepping it up, besides general memory > exhaustion? With so much free RAM I'd like to turn the dial a little > bit but I wonder if there are other linked things I should watch out > for. You can increase it, but if numvnodes is less than maxvnodes then it won't help you as the system hasn't had to recycle any vnodes yet. It is already caching all the vnodes you have accessed in that case. If the files are frequently truncated then you may end up with a lot of free RAM simply because there isn't enough recently used data to cache. The VM system will cache everything that is accessed until either 1) the pages become invalid (e.g. due to an unlink or truncate) or 2) free memory runs low enough to trigger pagedaemon to harvest some inactive pages. If you have 15G of free RAM, then 2) isn't happening and your working set is less than your RAM. -- John Baldwin