From owner-freebsd-current@FreeBSD.ORG Fri May 29 17:52:29 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F6AD106566C; Fri, 29 May 2009 17:52:29 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [192.147.25.65]) by mx1.freebsd.org (Postfix) with ESMTP id 2D3DC8FC13; Fri, 29 May 2009 17:52:29 +0000 (UTC) (envelope-from ler@lerctr.org) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=lerami; d=lerctr.org; h=Received:Date:From:To:cc:Subject:In-Reply-To:Message-ID:References:User-Agent:MIME-Version:Content-Type:X-Spam-Score:X-LERCTR-Spam-Score:X-Spam-Report:X-LERCTR-Spam-Report:DomainKey-Status; b=QjavlWg43Y9yKGO2+ntKOvgJiRc26r/KP/lLI4U2hvZIBxWwwH7ozkiiBkWfgoDs0GB8O0NVFSc4Abd0vUac5eor/S3t1j1SuOyzaAsG7pGJn0W/l89RJb9CKiGIFLlak3eBqa8bguwNUwR39p9W1r88VESWBMxeRp0IWvqDPqw=; Received: from thebighonker.lerctr.org ([192.147.25.65]:53180) by thebighonker.lerctr.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1MA6Fm-000KOF-Q1; Fri, 29 May 2009 12:52:28 -0500 Date: Fri, 29 May 2009 12:52:24 -0500 (CDT) From: Larry Rosenman To: Kip Macy In-Reply-To: Message-ID: References: <3c1674c90905242253n544c3f0cqb10952f349391ce7@mail.gmail.com> <454b8cc37c60ab7af2663ba70ddbfd59.squirrel@webmail.lerctr.org> <5a9a181a12e9e4ef864d23ae063f7277.squirrel@webmail.lerctr.org> <3c1674c90905280055h740bce23p33b18fefacf31196@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Spam-Score: -4.3 (----) X-LERCTR-Spam-Score: -4.3 (----) X-Spam-Report: SpamScore (-4.3/5.0) ALL_TRUSTED=-1.8, BAYES_00=-2.599, SARE_SUB_OBFU_OTHER=0.135 X-LERCTR-Spam-Report: SpamScore (-4.3/5.0) ALL_TRUSTED=-1.8, BAYES_00=-2.599, SARE_SUB_OBFU_OTHER=0.135 DomainKey-Status: no signature Cc: freebsd-current@freebsd.org Subject: Re: ZFS Crash X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 May 2009 17:52:29 -0000 On Fri, 29 May 2009, Larry Rosenman wrote: > > Ok, it just crashed. Unfortunately, I'm at work and the box is at home. > > I did have my script running every minute of that entire boot. > > What I saw was a full backup running, and then we started paging, and then > the backup jobs got pager errors, and were killed. > > I'm not sure what else went on, so I restarted the bacula daemons that > got killed, and was in the bacula console when it died. > > I'll see if I can get a cell-phone camera shot of the console. > > I'll also tar up the vmstat outputs and put them on my web server. > > What other forensics should I get? Bear in mind the system is probably > locked up with no dump taken :( One other "interesting" thing is the IPMI card seems to also be locked up. I.E. if I try to login to it, it just hangs after giving id/pw. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893