From owner-freebsd-stable@freebsd.org Wed Nov 9 14:03:53 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B55CC38648 for ; Wed, 9 Nov 2016 14:03:53 +0000 (UTC) (envelope-from hlh@restart.be) Received: from tignes.restart.be (tignes.restart.be [IPv6:2001:41d0:8:bdbe:0:1::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tignes.restart.be", Issuer "CA master" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D391F9F for ; Wed, 9 Nov 2016 14:03:52 +0000 (UTC) (envelope-from hlh@restart.be) X-Comment: SPF check N/A for local connections - client-ip=2001:41d0:8:bdbe:1:1::; helo=restart.be; envelope-from=hlh@restart.be; receiver=freebsd-stable@freebsd.org DKIM-Filter: OpenDKIM Filter v2.10.3 tignes.restart.be 3tDSZt6tf1zrMw DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=restart.be; s=tignes; t=1478700231; bh=A9Ck4EsrMnRrqHY8F5TKcK631mpfMR4waqCBYOY8Z8E=; h=Subject:To:References:From:Date:In-Reply-To; z=Subject:=20Re:=20Freebsd=2011.0=20RELEASE=20-=20ZFS=20deadlock|To :=20freebsd-stable@freebsd.org|References:=20<0c223160-b76f-c635-b b15-4a068ba7efe7@restart.be>=0D=0A=20|From:=20Henri=20Hennebert=20 |Date:=20Wed,=209=20Nov=202016=2015:03:49=20+0100|In-Reply-To:=20< aaf2df40-b0df-2141-9ed8-5b947d8d5a33@FreeBSD.org>; b=vODNpb10yLsXE9n1fVXx+4Cm/PkD0T8hkHnXJtWU99Rg+ZmXPbRbuKRcG44uhRfqt Q2eLM1fPTb/idNN0LhTlEfBc5TfKc7WcNfvY5cEKXFgohoPgJ0FgJUVUP/FzBbUe6N cxvsF87NjSU0nUG97h08Hh+ulAELGVHH6YXJ95Rp/WXhHiOxs/C6XorUmzEgYrari3 +4Va3V+3/YuW7HgpkJhjMjYNSA4rITr6nEF8v8s09dAdzgyMtMkASyCYDdmKvdjzbU Z93opA9qyrDzHaQNIP1R9eYvMQJALkLnU7YGxMP04pyeIiZVvp8UHQXrQZiOZbT/RH aRmulwk2DnTkA== Received: from restart.be (avoriaz.restart.be [IPv6:2001:41d0:8:bdbe:1:1::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.restart.be", Issuer "CA master" (verified OK)) by tignes.restart.be (Postfix) with ESMTPS id 3tDSZt6tf1zrMw for ; Wed, 9 Nov 2016 15:03:50 +0100 (CET) Received: from chamonix.restart.bel (chamonix.restart.bel [IPv6:2001:41d0:8:bdbe:1:9:0:0]) (authenticated bits=0) by restart.be (8.15.2/8.15.2) with ESMTPSA id uA9E3neh002019 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 9 Nov 2016 15:03:49 +0100 (CET) (envelope-from hlh@restart.be) Subject: Re: Freebsd 11.0 RELEASE - ZFS deadlock To: freebsd-stable@freebsd.org References: <0c223160-b76f-c635-bb15-4a068ba7efe7@restart.be> From: Henri Hennebert Message-ID: Date: Wed, 9 Nov 2016 15:03:49 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2016 14:03:53 -0000 On 11/09/2016 14:58, Eric van Gyzen wrote: > On 11/09/2016 07:48, Henri Hennebert wrote: >> I encounter a strange deadlock on >> >> FreeBSD avoriaz.restart.bel 11.0-RELEASE-p3 FreeBSD 11.0-RELEASE-p3 #0 r308260: >> Fri Nov 4 02:51:33 CET 2016 >> root@avoriaz.restart.bel:/usr/obj/usr/src/sys/AVORIAZ amd64 >> >> This system is exclusively running on zfs. >> >> After 3 or 4 days, `periodic daily` is locked in the directory /usr/local/news/bin >> >> [root@avoriaz ~]# ps xa|grep find >> 85656 - D 0:01.13 find / ( ! -fstype local -o -fstype rdonly ) -prune >> -o ( -name [#,]* -o -name .#* -o -name a.out -o -nam >> 462 1 S+ 0:00.00 grep find >> [root@avoriaz ~]# procstat -f 85656 >> PID COMM FD T V FLAGS REF OFFSET PRO NAME >> 85656 find text v r r------- - - - /usr/bin/find >> 85656 find cwd v d r------- - - - /usr/local/news/bin >> 85656 find root v d r------- - - - / >> 85656 find 0 v c r------- 3 0 - /dev/null >> 85656 find 1 p - rw------ 1 0 - - >> 85656 find 2 v r -w------ 7 17 - - >> 85656 find 3 v d r------- 1 0 - /home/root >> 85656 find 4 v d r------- 1 0 - /home/root >> 85656 find 5 v d r----n-- 1 533545184 - /usr/local/news/bin >> [root@avoriaz ~]# >> >> If I try `ls /usr/local/news/bin` it is also locked. >> >> After `shutdown -r now` the system remain locked after the line '0 0 0 0 0 0' >> >> After a reset and reboot I can access /usr/local/news/bin. >> >> I delete this directory and reinstall the package `portupgrade -fu news/inn` >> >> 5 days later `periodic daily`is locked on the same directory :-o >> >> Any idea? > I can't help with the deadlock, but someone who _can_ help will probably ask for > the output of "procstat -kk PID" with the PID of the "find" process. > > Eric [root@avoriaz ~]# procstat -kk 85656 PID TID COMM TDNAME KSTACK 85656 101112 find - mi_switch+0xd2 sleepq_wait+0x3a sleeplk+0x1b4 __lockmgr_args+0x356 vop_stdlock+0x3c VOP_LOCK1_APV+0x8d _vn_lock+0x43 vget+0x47 cache_lookup+0x679 vfs_cache_lookup+0xac VOP_LOOKUP_APV+0x87 lookup+0x591 namei+0x572 kern_statat+0xa8 sys_fstatat+0x2c amd64_syscall+0x4ce Xfast_syscall+0xfb Henri