From owner-freebsd-stable@freebsd.org Thu Nov 10 17:34:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F75CC39D28 for ; Thu, 10 Nov 2016 17:34:47 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4FF1EBD4 for ; Thu, 10 Nov 2016 17:34:45 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA21051; Thu, 10 Nov 2016 19:34:43 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1c4tFL-0006up-9s; Thu, 10 Nov 2016 19:34:43 +0200 Subject: Re: Freebsd 11.0 RELEASE - ZFS deadlock To: Henri Hennebert , freebsd-stable@FreeBSD.org References: <0c223160-b76f-c635-bb15-4a068ba7efe7@restart.be> <43c9d4d4-1995-5626-d70a-f92a5b456629@FreeBSD.org> <9d1f9a76-5a8d-6eca-9a50-907d55099847@FreeBSD.org> <6bc95dce-31e1-3013-bfe3-7c2dd80f9d1e@restart.be> <23a66749-f138-1f1a-afae-c775f906ff37@restart.be> From: Andriy Gapon Message-ID: <8e7547ef-87f7-7fab-6f45-221e8cea1989@FreeBSD.org> Date: Thu, 10 Nov 2016 19:33:47 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <23a66749-f138-1f1a-afae-c775f906ff37@restart.be> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 17:34:47 -0000 On 10/11/2016 18:12, Henri Hennebert wrote: > On 11/10/2016 16:54, Andriy Gapon wrote: >> On 10/11/2016 17:20, Henri Hennebert wrote: >>> On 11/10/2016 15:00, Andriy Gapon wrote: >>>> Interesting. I can not spot any suspicious thread that would hold the vnode >>>> lock. Could you please run kgdb (just like that, no arguments), then execute >>>> 'bt' command and then select a frame when _vn_lock is called with 'fr N' >>>> command. Then please 'print *vp' and share the result. >>>> >>> I Think I miss something in your request: >> >> Oh, sorry! The very first step should be 'tid 101112' to switch to the correct >> context. >> > > (kgdb) fr 7 > #7 0xffffffff8063c5b3 in _vn_lock (vp=, flags=2121728, "value optimized out" - not good > file=, > line=) at vnode_if.h:859 > 859 vnode_if.h: No such file or directory. > in vnode_if.h > (kgdb) print *vp I am not sure if this output is valid, because of the message above. Could you please try to navigate to nearby frames and see if vp itself has a valid value there. If you can find such a frame please do *vp there. > $1 = {v_tag = 0xffffffff80faeb78 "â~\231\200ÿÿÿÿ", v_op = 0xfffff80009a41000, > v_data = 0x0, v_mount = 0xfffff80009a41010, > v_nmntvnodes = {tqe_next = 0x0, tqe_prev = 0xffffffff80edc088}, v_un = > {vu_mount = 0x0, vu_socket = 0x0, vu_cdev = 0x0, > vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0xfffff80009466e90, le_prev = > 0x0}, v_cache_src = {lh_first = 0xfffffe010186d768}, > v_cache_dst = {tqh_first = 0x0, tqh_last = 0xfffffe0000b8a7c0}, v_cache_dd = > 0xfffff8000284f000, v_lock = {lock_object = { > lo_name = 0xfffff8002c00ee80 "", lo_flags = 0, lo_data = 0, lo_witness = > 0xfffff800068bd480}, > lk_lock = 18446735277770268056, lk_exslpfail = 153715840, lk_timo = -2048, > lk_pri = 0}, v_interlock = {lock_object = { > lo_name = 0x18af8ffffffff address>, lo_flags = 0, lo_data = 0, > lo_witness = 0x0}, mtx_lock = 0}, v_vnlock = 0x0, v_actfreelist = > {tqe_next = 0x0, tqe_prev = 0xfffff80009ba05c0}, > v_bufobj = {bo_lock = {lock_object = {lo_name = 0xfffff80009a41000 "", > lo_flags = 1, lo_data = 0, lo_witness = 0x4000000ff}, > rw_lock = 2}, bo_ops = 0x100000000, bo_object = 0xfffff80049c2c068, > bo_synclist = {le_next = 0xffffffff813be535, > le_prev = 0x1000000000000}, bo_private = 0x0, __bo_vnode = 0x0, bo_clean = > {bv_hd = {tqh_first = 0x0, tqh_last = 0x0}, > bv_root = {pt_root = 0}, bv_cnt = 0}, bo_dirty = {bv_hd = {tqh_first = > 0xfffff80088ac8d00, tqh_last = 0xfffff8003cc5b600}, > bv_root = {pt_root = 2553161591}, bv_cnt = -1741805705}, bo_numoutput = > 31, bo_flag = 0, bo_bsize = 0}, v_pollinfo = 0x0, > v_label = 0x0, v_lockf = 0x0, v_rl = {rl_waiters = {tqh_first = 0xf88, > tqh_last = 0x19cc}, rl_currdep = 0x3f8}, > v_cstart = 16256, v_lasta = 679, v_lastw = 0, v_clen = 0, v_holdcnt = 0, > v_usecount = 2369, v_iflag = 0, v_vflag = 0, > v_writecount = 0, v_hash = 0, v_type = VNON} > (kgdb) > > Thanks for your time > > Henri -- Andriy Gapon