From owner-freebsd-current@FreeBSD.ORG Tue Feb 20 15:45:42 2007 Return-Path: X-Original-To: current@FreeBSD.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5BE8D16A535; Tue, 20 Feb 2007 15:45:42 +0000 (UTC) (envelope-from john@utzweb.net) Received: from marley.grokthis.net (marley.grokthis.net [69.93.78.26]) by mx1.freebsd.org (Postfix) with ESMTP id 336B613C471; Tue, 20 Feb 2007 15:45:42 +0000 (UTC) (envelope-from john@utzweb.net) Received: from minime.local (c-71-231-84-94.hsd1.or.comcast.net [71.231.84.94]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by marley.grokthis.net (Postfix) with ESMTP id B5E93F1018; Tue, 20 Feb 2007 10:22:44 -0500 (EST) Date: Tue, 20 Feb 2007 07:21:39 -0800 From: John Utz To: Robert Watson Message-ID: <20070220072139.78d85ca5@minime.local> In-Reply-To: <20070215095248.P55760@fledge.watson.org> References: <20070215095248.P55760@fledge.watson.org> Organization: The Utz Family X-Mailer: Claws Mail 2.7.2 (GTK+ 2.8.18; i386-unknown-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: current@FreeBSD.org Subject: Re: sbflush() panics, new socket-related DDB commands (was: cvs commit: src/sys/conf files src/sys/kern uipc_debug.c (fwd)) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 15:45:42 -0000 On Thu, 15 Feb 2007 10:12:59 +0000 (GMT) Robert Watson wrote: > > Dear All, > > Of late we've seen an increase in the number of reports of panics in > sbflush(). These occur due to an invariant check when closing > sockets, which in effect confirms that the amount of actual data in a > socket buffer matches the socket buffer's cached length from that > data. Typically, these panics are a symptom of mbuf chain/mbuf > packet queue corruption, and in the past have been symptomatic > primarily of device driver bugs (i.e., the device driver touches mbuf > fields after passing the mbuf into the stack, etc). However, > debugging these problems is very difficult, as sbflush() is called > only when the socket is closed, and the corruption frequently occurs > much earlier. It's possibly worth pointing out that i can get either the same or a remarkably similar panic 100% reproducibly by using usbaudio. As soon as mpg321 finishes the song and closes the device i get a panic. 100% reproducible. the panic does *not* occur if i use the snd_ich stuff instead of usbaudio. i last synced at the following: minime# uname -a FreeBSD minime.local 7.0-CURRENT FreeBSD 7.0-CURRENT #20: Mon Feb 12 07:49:10 PST 2007 spaz@minime.local:/usr/src/sys/i386/compile/MINIME i386 I havent reported it previously because i thought it was my own little weird problem and i've been attempting to get a more specific failure point beyond 'device close' > If you are experiencing these panics, please compile your kernel with > "options SOCKBUF_DEBUG", which adds more frequent invariant checks of > socket buffer consistency. These checks may have a substantial > performance impact, but unless we can catch the problem(s) closer to > when it(they) occur(s), the chances of tracking the source are low. > We could be looking at one or more bugs in some combination of the > socket, protocol, and network interface code leading to a recent > increase. Perhaps socket buffer auto-sizing has increased the > chances of it occuring, or perhaps we just have increase parallelism > in test hardware. Dont think it's network code because it get's tickled by closing usbaudio. > I've added some new commands to DDB to make it easier to understand > the state of the system following panic. While the same information > can be extracted with kgdb, core dumps are not reliably available in > all environments, and certain other information (such as lock > information) is most easily obtained using DDB. The commands are: > > show socket > show sockbuf > show protosw > show domain > > Only the first two of these are likely to be used directly, and they > invoke protosw and domain printing as required. It would be helpful > if people debugging network panics involving sockets could use the > above (and especially show socket) on socket arguments in stack > traces. For example, in an sbflush() panic, the first argument to > soclose() in the stack trace is the socket pointer, so is ideal to > pass to "show socket" :-). > > FYI, in general, the first argument to sofoo() functions is the > socket pointer. Likewise, the first argument to sbfoo() functions is > the socket buffer pointer. This is not the case of soo_foo() > functions, which take a file descriptor, or filt_sofoo() functions, > which accept a knote. In general, if you print the socket contents > with "show socket", then it isn't necessary to print socket buffers > separately; "show socket" prints not just the contents of the socket > buffers, but also the pointers to them so that those pointers can be > compared with other arguments in the stack. I will endeavor to resync and rebuild and see if i can provide a more specific failure point with the usb stuff, but i will not be able to tackle it before the weekend because this laptop is my only computer (and i run -current on it, suprisingly well for the most part!) > Robert N M Watson > Computer Laboratory > University of Cambridge > > ---------- Forwarded message ---------- > Date: Thu, 15 Feb 2007 01:28:23 +0000 (UTC) > From: Robert Watson > To: src-committers@FreeBSD.org, cvs-src@FreeBSD.org, > cvs-all@FreeBSD.org Subject: cvs commit: src/sys/conf files > src/sys/kern uipc_debug.c > > rwatson 2007-02-15 01:28:22 UTC > > FreeBSD src repository > > Modified files: > sys/conf files > Added files: > sys/kern uipc_debug.c > Log: > Teach DDB how to print sockets, socket buffers, protosw's, and > domain structures given pointers to them. > > Revision Changes Path > 1.1177 +1 -0 src/sys/conf/files > 1.1 +530 -0 src/sys/kern/uipc_debug.c (new) > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" >