From owner-freebsd-stable@FreeBSD.ORG Fri Aug 19 00:29:01 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AFFD106564A; Fri, 19 Aug 2011 00:29:01 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id C36848FC15; Fri, 19 Aug 2011 00:29:00 +0000 (UTC) Received: from alph.allbsd.org (p3028-ipbf608funabasi.chiba.ocn.ne.jp [125.175.94.28]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id p7J0SOkw048646; Fri, 19 Aug 2011 09:28:34 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id p7J0SK8W078115; Fri, 19 Aug 2011 09:28:21 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Fri, 19 Aug 2011 09:28:11 +0900 (JST) Message-Id: <20110819.092811.1087267565626420460.hrs@allbsd.org> To: attilio@FreeBSD.org From: Hiroki Sato In-Reply-To: <20110818025550.GA1971@libertas.local.camdensoftware.com> References: <20110818.091600.831954331552558249.hrs@allbsd.org> <20110818025550.GA1971@libertas.local.camdensoftware.com> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.3 on Emacs 23.1 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Fri_Aug_19_09_28_11_2011_956)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [133.31.130.32]); Fri, 19 Aug 2011 09:28:40 +0900 (JST) X-Spam-Status: No, score=-102.6 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT,DIRECTOCNDYN,RCVD_IN_RP_RNBL,SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: freebsd-stable@FreeBSD.org, sterling@camdensoftware.com, avg@FreeBSD.org, Nick Esborn , kostikbel@gmail.com, mdtansca@FreeBSD.org Subject: Re: panic: spin lock held too long (RELENG_8 from today) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Aug 2011 00:29:01 -0000 ----Security_Multipart(Fri_Aug_19_09_28_11_2011_956)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Chip Camden wrote in <20110818025550.GA1971@libertas.local.camdensoftware.com>: st> Quoth Attilio Rao on Thursday, 18 August 2011: st> > In callout_cpu_switch() if a low priority thread is migrating the st> > callout and gets preempted after the outcoming cpu queue lock is left st> > (and scheduled much later) we get this problem. st> > st> > In order to fix this bug it could be enough to use a critical section, st> > but I think this should be really interrupt safe, thus I'd wrap them st> > up with spinlock_enter()/spinlock_exit(). Fortunately st> > callout_cpu_switch() should be called rarely and also we already do st> > expensive locking operations in callout, thus we should not have st> > problem performance-wise. st> > st> > Can the guys I also CC'ed here try the following patch, with all the st> > initial kernel options that were leading you to the deadlock? (thus st> > revert any debugging patch/option you added for the moment): st> > http://www.freebsd.org/~attilio/callout-fixup.diff st> > st> > Please note that this patch is for STABLE_8, if you can confirm the st> > good result I'll commit to -CURRENT and then backmarge as soon as st> > possible. st> > st> > Thanks, st> > Attilio st> > st> st> Thanks, Attilio. I've applied the patch and removed the extra debug st> options I had added (though keeping debug symbols). I'll let you know if st> I experience any more panics. No panic for 20 hours at this moment, FYI. For my NFS server, I think another 24 hours would be sufficient to confirm the stability. I will see how it works... -- Hiroki ----Security_Multipart(Fri_Aug_19_09_28_11_2011_956)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk5NrhsACgkQTyzT2CeTzy1O/ACeJPyJpjyI8X68PscHDXRU7iXu 8M0An23TY3RL9ZPaL1R+FCLHmhe9Mqi7 =FHX7 -----END PGP SIGNATURE----- ----Security_Multipart(Fri_Aug_19_09_28_11_2011_956)----