From owner-freebsd-stable Sat Aug 31 18:16:18 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0DA0637B400; Sat, 31 Aug 2002 18:16:14 -0700 (PDT) Received: from earth.hub.org (earth.hub.org [64.49.215.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id B275F43E72; Sat, 31 Aug 2002 18:16:13 -0700 (PDT) (envelope-from scrappy@hub.org) Received: from earth.hub.org (earth.hub.org [64.49.215.11]) by earth.hub.org (Postfix) with ESMTP id EEE162CCA07; Sat, 31 Aug 2002 22:16:07 -0300 (ADT) Date: Sat, 31 Aug 2002 22:16:07 -0300 (ADT) From: "Marc G. Fournier" To: Matthew Dillon Cc: Arnvid Karstad , , Subject: Re: Problems with FreeBSD - causing zalloc to return 0 ?! In-Reply-To: <200208312003.g7VK37aS002117@apollo.backplane.com> Message-ID: <20020831220710.D14642-100000@mail1.hub.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Okay, so is this what you are looking for? jupiter# cat /root/run_gdb #!/bin/tcsh while(1) gdb -x /root/gdb_commands -batch -k /root/kernel.debug /dev/mem | grep \$1 | logger -ip local7.info sleep 60 end jupiter# cat gdb_commands print kernel_vm_end jupiter# tail -f /var/log/server_watch Aug 31 20:12:46 jupiter root[8902]: $1 = 4055891968 Aug 31 20:13:47 jupiter root[9548]: $1 = 4060086272 Anything I might want to add to that script, or is the kernel_vm_end the only thing that is useful at this point? On Sat, 31 Aug 2002, Matthew Dillon wrote: > > : > : This message is in MIME format. The first part should be readable text, > : while the remaining parts are likely unreadable without MIME-aware tools. > : Send mail to mime@docserver.cac.washington.edu for more info. > : > :--0-1838173444-1030814090=:14642 > :Content-Type: TEXT/PLAIN; charset=US-ASCII > : > : > :Morning ... as I said, shouldn't take long :) > : > :I setup the script to log in 1 second intervals, to a log.`date +%S` file, > :hoping that you only wanted the last few seconds of data? If not, I can > :set it up differently, but here are the result of the last minute of its > :apparent existence ... > : > :First, the kernel panic of: > > Well Marc, there's nothing totally obvious here but it is still quite > possible that the machine is running out of KVM. > > Try this. You can run gdb on a running machine as follows: > > gdb -k kernel.debug /dev/mem > > You can then print out elements of the kernel's memory in real time: > > gdb> print kernel_vm_end > > Could you post your kernel config? And repeat your test but run gdb > in a shell and print kernel_vm_end at the start of your test and > every so often while the test is running. > > Other things you can try that will reduce KVM usage: in > /boot/loader.conf reduce the amount of physical memory, e.g. reduce > from 4G to 2G with: > > hw.physmem="2048m" > > and see if you can repeat the problem. Oddly enough if you cannot > repeat the problem the issue is almost certainly the kernel running > out of KVM. You can also try reducing the number of mbuf clusters > and reducing maxvnodes, though the values I saw in your logs seem > reasonable. > > If you only have one swap device you can reducing the number of swap > devices by setting the NSWAPDEV kernel config variable to 1 > (it defaults to 4). This will reduce the KVM reservation for > the swap bitmap. You can also reduce the kernel reservation for > swap block data by setting the kern.maxswzone boot environment > variable. This is in bytes, e.g. in /boot/loader.conf > > kern.maxswzone="32m" > > All of these ideas are designed to reduce KVM usage. I am extremely > interested in knowing if they have an effect on the crashes you are > seeing. If you can stop the crashes from occuring then we are going > to have to get more conservative in our KVM allocations. > > -Matt > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message