From owner-freebsd-stable@freebsd.org Thu Nov 10 15:55:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C12DCC3A864 for ; Thu, 10 Nov 2016 15:55:19 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id F1A2EAD1 for ; Thu, 10 Nov 2016 15:55:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA20896; Thu, 10 Nov 2016 17:55:16 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1c4rh6-0006pu-2e; Thu, 10 Nov 2016 17:55:16 +0200 Subject: Re: Freebsd 11.0 RELEASE - ZFS deadlock To: Henri Hennebert , freebsd-stable@FreeBSD.org References: <0c223160-b76f-c635-bb15-4a068ba7efe7@restart.be> <43c9d4d4-1995-5626-d70a-f92a5b456629@FreeBSD.org> <9d1f9a76-5a8d-6eca-9a50-907d55099847@FreeBSD.org> <6bc95dce-31e1-3013-bfe3-7c2dd80f9d1e@restart.be> From: Andriy Gapon Message-ID: Date: Thu, 10 Nov 2016 17:54:39 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <6bc95dce-31e1-3013-bfe3-7c2dd80f9d1e@restart.be> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 15:55:19 -0000 On 10/11/2016 17:20, Henri Hennebert wrote: > On 11/10/2016 15:00, Andriy Gapon wrote: >> Interesting. I can not spot any suspicious thread that would hold the vnode >> lock. Could you please run kgdb (just like that, no arguments), then execute >> 'bt' command and then select a frame when _vn_lock is called with 'fr N' >> command. Then please 'print *vp' and share the result. >> > I Think I miss something in your request: Oh, sorry! The very first step should be 'tid 101112' to switch to the correct context. > [root@avoriaz ~]# kgdb > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > done. > > --- clip --- > > Loaded symbols for /boot/kernel/accf_data.ko > Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from > /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. > done. > Loaded symbols for /boot/kernel/daemon_saver.ko > #0 sched_switch (td=0xfffff8001131da00, newtd=0xfffff800762a8500, flags= optimized out>) > at /usr/src/sys/kern/sched_ule.c:1973 > 1973 cpuid = PCPU_GET(cpuid); > (kgdb) bt > #0 sched_switch (td=0xfffff8001131da00, newtd=0xfffff800762a8500, flags= optimized out>) > at /usr/src/sys/kern/sched_ule.c:1973 > #1 0xffffffff80566b15 in tc_fill_vdso_timehands32 (vdso_th32=0x0) at > /usr/src/sys/kern/kern_tc.c:2121 > #2 0xffffffff80555227 in timekeep_push_vdso () at > /usr/src/sys/kern/kern_sharedpage.c:174 > #3 0xffffffff80566226 in tc_windup () at /usr/src/sys/kern/kern_tc.c:1426 > #4 0xffffffff804eaa41 in hardclock_cnt (cnt=1, usermode=) > at /usr/src/sys/kern/kern_clock.c:589 > #5 0xffffffff808fac74 in handleevents (now=, fake=0) at > /usr/src/sys/kern/kern_clocksource.c:223 > #6 0xffffffff808fb1d7 in timercb (et=0xffffffff8100cf20, arg= out>) at /usr/src/sys/kern/kern_clocksource.c:352 > #7 0xfffff800b6429a00 in ?? () > #8 0xffffffff81051080 in vm_page_array () > #9 0xffffffff81051098 in vm_page_queue_free_mtx () > #10 0xfffffe0101818920 in ?? () > #11 0xffffffff805399c0 in __mtx_lock_sleep (c=, tid=Error > accessing memory address 0xffffffffffffffac: Bad add\ > ress. > ) at /usr/src/sys/kern/kern_mutex.c:590 > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > (kgdb) q > [root@avoriaz ~]# > > I don't find the requested frame > > Henri -- Andriy Gapon