From owner-freebsd-stable@FreeBSD.ORG Fri Aug 19 00:38:07 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DC74106566B for ; Fri, 19 Aug 2011 00:38:07 +0000 (UTC) (envelope-from sterling@camdensoftware.com) Received: from wh1.interactivevillages.com (ca.2e.7bae.static.theplanet.com [174.123.46.202]) by mx1.freebsd.org (Postfix) with ESMTP id F39218FC0C for ; Fri, 19 Aug 2011 00:38:06 +0000 (UTC) Received: from 184-78-197-203.war.clearwire-wmx.net ([184.78.197.203] helo=_HOSTNAME_) by wh1.interactivevillages.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69) (envelope-from ) id 1QuD61-0004Fs-P0; Thu, 18 Aug 2011 17:37:38 -0700 Received: by _HOSTNAME_ (sSMTP sendmail emulation); Thu, 18 Aug 2011 17:38:00 -0700 Date: Thu, 18 Aug 2011 17:37:59 -0700 From: Chip Camden To: Hiroki Sato Message-ID: <20110819003759.GC54831@libertas.local.camdensoftware.com> Mail-Followup-To: Hiroki Sato , attilio@FreeBSD.org, kostikbel@gmail.com, freebsd-stable@FreeBSD.org, avg@FreeBSD.org, mdtansca@FreeBSD.org, Nick Esborn References: <20110818.091600.831954331552558249.hrs@allbsd.org> <20110818025550.GA1971@libertas.local.camdensoftware.com> <20110819.092811.1087267565626420460.hrs@allbsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="f+W+jCU1fRNres8c" Content-Disposition: inline In-Reply-To: <20110819.092811.1087267565626420460.hrs@allbsd.org> User-Agent: Mutt/1.4.2.3i Company: Camden Software Consulting URL: http://camdensoftware.com X-PGP-Key: http://pgp.mit.edu:11371/pks/lookup?search=0xD6DBAF91 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - wh1.interactivevillages.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - camdensoftware.com X-Source: X-Source-Args: X-Source-Dir: Cc: freebsd-stable@FreeBSD.org, avg@FreeBSD.org, attilio@FreeBSD.org, Nick Esborn , kostikbel@gmail.com, mdtansca@FreeBSD.org Subject: Re: panic: spin lock held too long (RELENG_8 from today) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 00:38:07 -0000 --f+W+jCU1fRNres8c Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Quoth Hiroki Sato on Friday, 19 August 2011: > Chip Camden wrote > in <20110818025550.GA1971@libertas.local.camdensoftware.com>: >=20 > st> Quoth Attilio Rao on Thursday, 18 August 2011: > st> > In callout_cpu_switch() if a low priority thread is migrating the > st> > callout and gets preempted after the outcoming cpu queue lock is le= ft > st> > (and scheduled much later) we get this problem. > st> > > st> > In order to fix this bug it could be enough to use a critical secti= on, > st> > but I think this should be really interrupt safe, thus I'd wrap them > st> > up with spinlock_enter()/spinlock_exit(). Fortunately > st> > callout_cpu_switch() should be called rarely and also we already do > st> > expensive locking operations in callout, thus we should not have > st> > problem performance-wise. > st> > > st> > Can the guys I also CC'ed here try the following patch, with all the > st> > initial kernel options that were leading you to the deadlock? (thus > st> > revert any debugging patch/option you added for the moment): > st> > http://www.freebsd.org/~attilio/callout-fixup.diff > st> > > st> > Please note that this patch is for STABLE_8, if you can confirm the > st> > good result I'll commit to -CURRENT and then backmarge as soon as > st> > possible. > st> > > st> > Thanks, > st> > Attilio > st> > > st> > st> Thanks, Attilio. I've applied the patch and removed the extra debug > st> options I had added (though keeping debug symbols). I'll let you kno= w if > st> I experience any more panics. >=20 > No panic for 20 hours at this moment, FYI. For my NFS server, I > think another 24 hours would be sufficient to confirm the stability. > I will see how it works... >=20 > -- Hiroki Likewise: $ uptime 5:37PM up 21:45, 5 users, load averages: 0.68, 0.45, 0.63 So far, so good (knocks on head). --=20 =2EO. | Sterling (Chip) Camden | http://camdensoftware.com =2E.O | sterling@camdensoftware.com | http://chipsquips.com OOO | 2048R/D6DBAF91 | http://chipstips.com --f+W+jCU1fRNres8c Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iQEcBAEBAgAGBQJOTbBnAAoJEIpckszW26+RT+AIAIRMa07BhoVaRBq3lz1dVcsq zh+G7945FXqbD+0hhv+/4T75mbtzSG4l72dhlwGWNUZg70hZKqEUfNzQs3meSquR wmVCi3NH0cu5jIAZqvDWCvU8BigBn2GRjN/sXl5GCsGrZFi50kZXWKmgzTyDVrIM iwva8366ceK36QfodupVgxSs7ifDt8Jl3tLSdXHdacf17BceW2mETwOVvmd13LXQ BVOxFE7Qmk7xYXqrt3dj+E/gtO21R31EL3XJYx7prev534eNF99pn1GZCaj2By1Q B1iG4SfXMgYtzHpqSGniENX8RAhaCJmpFZDrIebnawel2rPMPFHuzJLc5hKp6eE= =lxLO -----END PGP SIGNATURE----- --f+W+jCU1fRNres8c--