From owner-freebsd-hackers@FreeBSD.ORG Wed Sep 26 16:55:41 2007 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EE8F16A417 for ; Wed, 26 Sep 2007 16:55:41 +0000 (UTC) (envelope-from benjie@addgene.org) Received: from el-out-1112.google.com (el-out-1112.google.com [209.85.162.176]) by mx1.freebsd.org (Postfix) with ESMTP id 79B7C13C45A for ; Wed, 26 Sep 2007 16:55:41 +0000 (UTC) (envelope-from benjie@addgene.org) Received: by el-out-1112.google.com with SMTP id r27so439361ele for ; Wed, 26 Sep 2007 09:55:40 -0700 (PDT) Received: by 10.114.131.9 with SMTP id e9mr1253822wad.1190825739631; Wed, 26 Sep 2007 09:55:39 -0700 (PDT) Received: by 10.114.15.16 with HTTP; Wed, 26 Sep 2007 09:55:39 -0700 (PDT) Message-ID: Date: Wed, 26 Sep 2007 12:55:39 -0400 From: "Benjie Chen" To: "Ivan Voras" , "Kris Kennaway" In-Reply-To: <9bbcef730709241211o479e3e5dwf64382596c837dfb@mail.gmail.com> MIME-Version: 1.0 References: <9bbcef730709241211o479e3e5dwf64382596c837dfb@mail.gmail.com> X-Mailman-Approved-At: Thu, 27 Sep 2007 12:09:16 +0000 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: Kernel panic on PowerEdge 1950 under certain stress load X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Sep 2007 16:55:41 -0000 We got this bt after setting kernel dump stuff up. For some reason kgdb did not find debugging symbols, even though the kernel was compiled with -g. Hopefully this helps and I will go dig some more to find why we don't have a debug symbol. #0 0xc0674fae in doadump () #1 0xc067550a in boot () #2 0xc0675831 in panic () #3 0xc088e29c in trap_fatal () #4 0xc088da0e in trap () #5 0xc0879d4a in calltrap () #6 0xc066c731 in _mtx_lock_sleep () #7 0xc06bba96 in unp_gc () #8 0xc06961b3 in taskqueue_run () #9 0xc0696696 in taskqueue_thread_loop () #10 0xc065ec4d in fork_exit () #11 0xc0879dac in fork_trampoline () On 9/24/07, Ivan Voras wrote: > > On 24/09/2007, Benjie Chen wrote: > > Ivan and Kris, > > > > I will try to get a kernel trace -- it may not happen for awhile since I > am > > not in the office and working remotely for awhile so it may not be easy > to > > get a trace... but I will check. > > It's fairly easy: > > 1) add lines like the following in rc.conf: > dumpdev="/dev/amrd0s1b" > dumpdir="/storage1/crashdumps" > > (dumpdev is your swap partition, which must be larger than your RAM, > dumpdir is where the crash dumps will be saved, also needs to be > larger than RAM) > > 2) add these lines to sysctl.conf: > > debug.debugger_on_panic=0 > debug.trace_on_panic=1 > > (These will cause the panic message and backtrace to be automatically > recorded in the message buffer saved to the kernel crash dump. If not > told otherwise, the machine will then reboot and at the next boot > generate a crash dump in your dumpdir. When you get the kernel crash > dump, run "kgdb vmcore.0 /boot/kernel/kernel" as root and then you can > inspect the trace, core dump, etc. You don't need any special > knowledge of (k)gdb for this. The one useful command to you may be > "bt" - generate a backtrace). > > > It looks like the problem reported by that link, and some of the links > from > > there though... > > I'm trying to solve that one with Craig Rodrigues. So far it's been > without success but we know where the problem is. > -- Benjie Chen, Ph.D. Addgene, a better way to share plasmids www.addgene.org