From owner-freebsd-current@FreeBSD.ORG Tue May 26 15:58:51 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8AA5A106564A; Tue, 26 May 2009 15:58:51 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) by mx1.freebsd.org (Postfix) with ESMTP id 5DF9D8FC1D; Tue, 26 May 2009 15:58:51 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from 76-205-169-61.lightspeed.austtx.sbcglobal.net ([76.205.169.61]:21853 helo=borg) by thebighonker.lerctr.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1M8ysb-000Gso-Vj; Tue, 26 May 2009 10:47:55 -0500 Date: Tue, 26 May 2009 10:47:37 -0500 (CDT) From: Larry Rosenman Sender: ler@borg To: Florian Smeets In-Reply-To: <4A1BE241.5030706@kasimir.com> Message-ID: References: <3c1674c90905242253n544c3f0cqb10952f349391ce7@mail.gmail.com> <454b8cc37c60ab7af2663ba70ddbfd59.squirrel@webmail.lerctr.org> <5a9a181a12e9e4ef864d23ae063f7277.squirrel@webmail.lerctr.org> <4A1BE241.5030706@kasimir.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Spam-Score: -2.3 (--) X-LERCTR-Spam-Score: -2.3 (--) X-Spam-Report: SpamScore (-2.3/5.0) ALL_TRUSTED=-1.8, BAYES_00=-2.599, SARE_SUB_OBFU_OTHER=0.135, TVD_RCVD_IP=1.931 X-LERCTR-Spam-Report: SpamScore (-2.3/5.0) ALL_TRUSTED=-1.8, BAYES_00=-2.599, SARE_SUB_OBFU_OTHER=0.135, TVD_RCVD_IP=1.931 DomainKey-Status: no signature Cc: freebsd-current@freebsd.org, Kip Macy Subject: Re: ZFS Crash X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2009 15:58:51 -0000 On Tue, 26 May 2009, Florian Smeets wrote: > On 26.05.09 14:04, Larry Rosenman wrote: >> On Mon, 25 May 2009, Larry Rosenman wrote: >> >>> On Mon, 25 May 2009, Larry Rosenman wrote: >>> >>>> after looking at the code, never mind the "don't call doadump", so we'll >>>> get the textdump. >>>> >>>> Thanks rwatson for the textdump stuff! >>>> >>> Here is current stats before we crash. Does any of this look totally >>> out of line? >>> >> It crashed again, but did *NOT* make it into ddb enough to do the textdump. >> >> It was hung with the backtrace (looks like the same, but I couldn't >> scroll the screen back). >> >> Ideas? >> >> I'm really concerned that there is a problem. >> > > Hi i guess I'm having a similar/the same problem. The panic looks the same, > and the trace is a little different, I've had a lot of these panics on this > machine, and the stack trace looked different quite often. > > The panic and show uma and show malloc can be found here: > http://webmail.solomo.de/~flo/panic.txt > > I have a coredump written to swap, but this panic is very easy to trigger and > it panics every time savecore runs... It is also easy to trigger this with a > buildworld -j5. > > This is a quadcore Xeon with a a slow SATA disk and only 1GB of mem. (It's > only a test machine, so i can test any suggestion.) This box used to run > stable with kmem max set to 768M. > > Anything i can do? > I've started running the below script every minute from cron. If anyone wants to look at the data let me know (it's on a Resi DSL line, so I don't want to run an http server). I'm willing to give SSH accounts to @FreeBSD.org devs to look around. script: #!/bin/sh DATE=`date +%Y%m%d.%H%M%S` (echo "Uptime:";uptime;echo "vmstat -m:";vmstat -m echo "vmstat -z:";vmstat -z) >/home/ler/stats/${DATE}.stats -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893