From owner-freebsd-fs@freebsd.org Thu Nov 19 17:16:28 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4CBFA32889 for ; Thu, 19 Nov 2015 17:16:28 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (yugo.yamagi.org [212.48.122.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8AC971D77 for ; Thu, 19 Nov 2015 17:16:27 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from p57b5dddb.dip0.t-ipconnect.de ([87.181.221.219] helo=kosei.home.yamagi.org) by mail1.yamagi.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1ZzSE6-000NCT-Rb; Thu, 19 Nov 2015 17:38:27 +0100 Date: Thu, 19 Nov 2015 17:38:21 +0100 From: Yamagi Burmeister To: rs@bytecamp.net Cc: freebsd-fs@freebsd.org Subject: Re: filesystem deadlock, process in vodead state Message-Id: <20151119173821.b3da4b7d92571b723d4c5e5f@yamagi.org> In-Reply-To: <564D9930.1080509@bytecamp.net> References: <564D9930.1080509@bytecamp.net> X-Mailer: Sylpheed 3.4.3 (GTK+ 2.24.28; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Nov 2015 17:16:28 -0000 Hello, last week I've observed a similar deadlock on a FreeBSD 10.2 machine with UFS filesystem. I haven't saved the procstat output because I suspected broken hardware (The SSD wearout indicator was down to 22. There weren't any ATA errors in dmesg), but as far as I remember the kernel stacks looked similar. ufs_open() instead of zfs_freebsd_open() of course. Regards, Yamagi On Thu, 19 Nov 2015 10:41:04 +0100 Robert Schulze wrote: > Hi, > > on an SSD-only mirrored pool which was idle for about 30 days, I noticed > a stuck chksetuid run from periodic. The "find" process is stuck on > vodead state. > > I found out which directory is affected via procstat -f and tried to ls > it, now this ls is stuck in above state, too (no surprise). > > procstat -kk output is: > > 1816 102485 find - mi_switch+0xe1 > sleepq_wait+0x3a _sleep+0x287 vnode_create_vobject+0x100 > zfs_freebsd_open+0xf5 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 > vn_open_cred+0x33e kern_openat+0x26f amd64_syscall+0x33a Xfast_syscall+0xfb > > 71376 102400 ls - mi_switch+0xe1 > sleepq_wait+0x3a _sleep+0x287 vnode_create_vobject+0x100 > zfs_freebsd_open+0xf5 VOP_OPEN_APV+0xa1 vn_open_vnode+0x234 > vn_open_cred+0x33e kern_openat+0x26f amd64_syscall+0x33a Xfast_syscall+0xfb > > The processes are not killable. > So what is this state and how to fix the deadlocked filesystem? I'm able > to give any debugging information, just give me some pointers. > > # uname: FreeBSD 10.2-RELEASE-p5 #1 r289218 > > # zpool status > > pool: home > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > home ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/mirror0-a ONLINE 0 0 0 > gpt/mirror0-b ONLINE 0 0 0 > mirror-1 ONLINE 0 0 0 > gpt/mirror1-a ONLINE 0 0 0 > gpt/mirror1-b ONLINE 0 0 0 > mirror-2 ONLINE 0 0 0 > gpt/mirror2-a ONLINE 0 0 0 > gpt/mirror2-b ONLINE 0 0 0 > mirror-3 ONLINE 0 0 0 > gpt/mirror3-a ONLINE 0 0 0 > gpt/mirror3-b ONLINE 0 0 0 > > errors: No known data errors > > > with kind regards, > Robert Schulze > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Homepage: www.yamagi.org XMPP: yamagi@yamagi.org GnuPG/GPG: 0xEFBCCBCB