From owner-freebsd-current@FreeBSD.ORG Tue Nov 18 13:00:22 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0FE716A4CE for ; Tue, 18 Nov 2003 13:00:22 -0800 (PST) Received: from smtp.mho.com (smtp.mho.net [64.58.4.6]) by mx1.FreeBSD.org (Postfix) with SMTP id 24C5843FE3 for ; Tue, 18 Nov 2003 13:00:20 -0800 (PST) (envelope-from scottl@freebsd.org) Received: (qmail 87877 invoked by uid 1002); 18 Nov 2003 21:00:19 -0000 Received: from unknown (HELO ?10.4.1.5?) (64.58.1.252) by smtp.mho.net with SMTP; 18 Nov 2003 21:00:19 -0000 Date: Tue, 18 Nov 2003 14:02:41 -0700 (MST) From: Scott Long X-X-Sender: scottl@pooker.samsco.home To: =?iso-8859-1?q?Claus=20Guttesen?= In-Reply-To: <20031118193749.22575.qmail@web14107.mail.yahoo.com> Message-ID: <20031118135038.T34253@pooker.samsco.home> References: <20031118193749.22575.qmail@web14107.mail.yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@freebsd.org Subject: Re: FreeBSD current, apache and php4 woes X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Nov 2003 21:00:23 -0000 On Tue, 18 Nov 2003, [iso-8859-1] Claus Guttesen wrote: > Hi. > > > > > > >>>panic: kmem_malloc(4096): kmem_map too small: > > >>>275251200 total allocated cpuid = 0; lapic.id = > > >>>00000000 > > >> > > You'll either want to raise the size of the kmem_map > > pool or decrease the maximum number of vnodes > allowed > > (vnodes get allocated out of the kmem_map and are > > likely depleating it > > Add one of the two lines to /boot/loader.conf: > > > > kern.vn.kmem.size=350000000 > > or > > kern.maxvnodes=150000 > > > > The first one is probably the better choice for you > > since > > the very nature of what you are doing demands that > > you touch a lot of vnodes. > > > > Scott > > It seems that your advice helpted cure the patient. I > did two things: > > 1. added kern.vm.kmem.size=450000000 > 2. clean up tmp-files older than 4 hours every hour > (previous was files older than 12 h.). > > Now the servers has been quite stable, no reboot in > almost two days! My problem appears to be too many > files in /tmp and /var/tmp (50.000 or more) which made > the kernel puke. I forgot to mention in the last email that kern.maxvnodes will still scale upwards as you increase kern.vm.kmem.size. So you might want to set a hard limit on it so you don't continue to run into problems. A value of 200,000 is probably good in your case. > > I guess this is a scenario which we will see more > often. Would it be possible to output this situation > to the message-log before the server simply reboots? It might be useful for this particular panic message to print out the value of maxvnodes, numvnodes, and/or other metrics to help with debugging. We need to also review the scaling algorithm and tweak it back into line. A more complex solution would be to create a way for the vfs system to get feedback on KVA and kmem_map pressure and auto-tune itself. Scott