From owner-freebsd-stable@FreeBSD.ORG Wed Sep 7 00:48:31 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E33C1106566C; Wed, 7 Sep 2011 00:48:31 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper-int.allbsd.org [IPv6:2001:2f0:104:e002::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5124F8FC17; Wed, 7 Sep 2011 00:48:21 +0000 (UTC) Received: from alph.allbsd.org ([IPv6:2001:2f0:104:e010:862b:2bff:febc:8956]) (authenticated bits=128) by mail.allbsd.org (8.14.4/8.14.4) with ESMTP id p870lpDU023337; Wed, 7 Sep 2011 09:48:01 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.14.4/8.14.4) with ESMTP id p870lkCs065537; Wed, 7 Sep 2011 09:47:46 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Wed, 07 Sep 2011 09:47:17 +0900 (JST) Message-Id: <20110907.094717.2272609566853905102.hrs@allbsd.org> To: attilio@FreeBSD.org From: Hiroki Sato In-Reply-To: <20110903104410.GX17489@deviant.kiev.zoral.com.ua> References: <20110820.105229.834911491934932780.hrs@allbsd.org> <20110903.071908.971549835606878048.hrs@allbsd.org> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.3.51 on Emacs 23.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Wed_Sep__7_09_47_17_2011_726)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (mail.allbsd.org [IPv6:2001:2f0:104:e001::32]); Wed, 07 Sep 2011 09:48:03 +0900 (JST) X-Spam-Status: No, score=-104.6 required=13.0 tests=BAYES_00, CONTENT_TYPE_PRESENT, RDNS_NONE, SPF_SOFTFAIL, USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on gatekeeper.allbsd.org Cc: pjd@FreeBSD.org, sterling@camdensoftware.com, avg@FreeBSD.org, nick@desert.net, freebsd-stable@FreeBSD.org, kib@FreeBSD.org Subject: Re: panic: spin lock held too long (RELENG_8 from today) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Sep 2011 00:48:32 -0000 ----Security_Multipart(Wed_Sep__7_09_47_17_2011_726)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Attilio Rao wrote in : at> This should be enough for someone NFS-aware to look into it. at> at> Were you also able to get a core? Yes. But as kib@ pointed out it seems a deadlock in ZFS. Some experiments I did showed that this deadlock can be triggered at least by doing "rm -rf" against a local directory that has a large number of files/sub-directories. Then, I updated the kernel with the latest 8-STABLE + WITNESS option because a fix for LOR of spa_config lock was committed and tracking locks without WITNESS was hard. The deadlock can still be triggered after that. During this investigation an disk has to be replaced and resilvering it is now in progress. A deadlock and a forced reboot after that make recovering of the zfs datasets take a long time (for committing logs, I think), so I will try to reproduce the deadlock and get a core dump after it finished. If the old kernel and core of the deadlock I reported on Saturday are still useful for debugging, I can put them to somewhere you can access. -- Hiroki ----Security_Multipart(Wed_Sep__7_09_47_17_2011_726)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEABECAAYFAk5mvxUACgkQTyzT2CeTzy14sACfRMSxYlKQktsuL8w/10/89VUU Q6sAnAu5mnWcMUzllH3eTzEXr04g3s2j =iR8M -----END PGP SIGNATURE----- ----Security_Multipart(Wed_Sep__7_09_47_17_2011_726)----