From owner-svn-src-all@FreeBSD.ORG Tue May 31 22:40:56 2011 Return-Path: Delivered-To: svn-src-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7C191065672; Tue, 31 May 2011 22:40:56 +0000 (UTC) (envelope-from ken@kdm.org) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) by mx1.freebsd.org (Postfix) with ESMTP id 6542D8FC14; Tue, 31 May 2011 22:40:56 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.2/8.14.2) with ESMTP id p4VMeti5060162; Tue, 31 May 2011 16:40:55 -0600 (MDT) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.2/8.14.2/Submit) id p4VMetrV060161; Tue, 31 May 2011 16:40:55 -0600 (MDT) (envelope-from ken) Date: Tue, 31 May 2011 16:40:55 -0600 From: "Kenneth D. Merry" To: mdf@FreeBSD.org Message-ID: <20110531224055.GA59252@nargothrond.kdm.org> References: <201105311729.p4VHTwrZ033296@svn.freebsd.org> <20110531214618.GA57498@nargothrond.kdm.org> <20110531221122.GA58836@nargothrond.kdm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2i Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org Subject: Re: svn commit: r222537 - in head/sys: kern sys X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 May 2011 22:40:56 -0000 On Tue, May 31, 2011 at 15:16:05 -0700, mdf@FreeBSD.org wrote: > On Tue, May 31, 2011 at 3:11 PM, Kenneth D. Merry wrote: > > On Tue, May 31, 2011 at 15:02:37 -0700, mdf@FreeBSD.org wrote: > >> On Tue, May 31, 2011 at 2:46 PM, Kenneth D. Merry wrote: > >> > On Tue, May 31, 2011 at 14:00:18 -0700, mdf@FreeBSD.org wrote: > >> >> On Tue, May 31, 2011 at 10:29 AM, Kenneth D. Merry wrote: > >> >> > Author: ken > >> >> > Date: Tue May 31 17:29:58 2011 > >> >> > New Revision: 222537 > >> >> > URL: http://svn.freebsd.org/changeset/base/222537 > >> >> > > >> >> > Log: > >> >> > ?Fix apparent garbage in the message buffer. > >> >> > > >> >> > ?While we have had a fix in place (options PRINTF_BUFR_SIZE=128) to fix > >> >> > ?scrambled console output, the message buffer and syslog were still getting > >> >> > ?log messages one character at a time. ?While all of the characters still > >> >> > ?made it into the log (courtesy of atomic operations), they were often > >> >> > ?interleaved when there were multiple threads writing to the buffer at the > >> >> > ?same time. > >> >> > >> >> This seems to panic my box with "lock "msgbuf" 0xfffffe0127ffffe0 > >> >> already initialized". > >> >> > >> >> Unfortunately, though I booted with a fresh CURRENT this morning > >> >> successfully, both /boot/kernel and /boot/kernel.old give this panic. > >> >> To add insult to injury, when the kernel drops into the debugger, my > >> >> keyboard input no longer works so I can't get a stack, etc. > >> > > >> > Uh-oh! > >> > > >> >> So: > >> >> > >> >> 1) Is there anything else I can do to help debug this? > >> >> 2) how can I resurrect this box without a reinstall? > >> >> > >> >> I will try to repro on a virtual machine so I have a snapshot to come back to. > >> > > >> > My guess is that this is an issue with the message buffer reinitialization > >> > path. ?lock_init() (called by mtx_init()) has an assert to make sure that > >> > the lock is initialized, and that is just a flag check. > >> > > >> > Since the spin lock is part of the message buffer structure, if it is held > >> > over from a previous boot, the LO_INITIALIZED flag may still be set. > >> > > >> > Try power cycling the machine. ?If it is an issue with re-initialization, > >> > that should clear the memory and allow you to boot. > >> > >> Hmm, apparently my previous presses of the power button weren't long > >> enough. ?I let it sit off for 20 seconds and it boots okay now. > > > > Okay, so it probably is the re-initialization code. ?Can you try this patch > > and see if it survives a warm boot? ?I also changed the initialization > > path, so we don't get tripped up by garbage left in memory. > > This patch survived a warm reboot (shutdown -r now). Cool! I just checked in the patch. > > Also, does the debugger work now that it has booted successfully? > > The debugger (or rather, my keyboard in the debugger) works on a > successful boot. I used sysctl debug.kdb.enter=1 to test it. Ahh, glad things are working there too. Let me know if you (or anyone else) run into any more issues. Thanks for testing this! Ken -- Kenneth Merry ken@FreeBSD.ORG