From owner-freebsd-fs@freebsd.org Sun Dec 4 17:55:33 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C658C67963 for ; Sun, 4 Dec 2016 17:55:33 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E92291BB0 for ; Sun, 4 Dec 2016 17:55:32 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from moby.local ([95.119.248.22]) by mail.gmx.com (mrgmx101 [212.227.17.174]) with ESMTPSA (Nemesis) id 0M9wrU-1cOcZI2Miu-00B4Co for ; Sun, 04 Dec 2016 18:55:24 +0100 From: Nikos Vassiliadis Subject: unionfs/zfs panic To: freebsd-fs@FreeBSD.org Message-ID: <1394db91-ea6e-4d24-41c1-d0ad762c493f@gmx.com> Date: Sun, 4 Dec 2016 18:55:23 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:0jjkkrW9iTPooAhn1WnJJtxHNmr6i2MN9OPNHHraOF5PANN798L kTM3siMjrqaM2CzNEcKGEDO9TwWvaj8Jg1puSO2ksrdB7EIWmx1kKXfzsASTChs/prw3YJD /V31ZMLKrVhk2HvYkx/E63ENZIe0OqsmGcDQ4TzBnkYnzLp5VlCgWcS2/zbxks/MJcs1KiN 5OFCSHchUvK0HMtadB7FA== X-UI-Out-Filterresults: notjunk:1;V01:K0:C9rBp7uH0yI=:44oUY/97b8G1aXiQcojT+x KJVgkmZzCQcZufbEf0Ojk/aX85ZgV3HGvk2PYQWmiXJAfdu2hsBiLc1LsmQ5MSuyVka/zMF01 oGWNiS+JzLZNcvhWR4M1MCW3HizngjTnvJdJt2cygSXKAqGtW2yj31vPq2EczeWxnlFLSY9hd zwrxhbXfqQmKYCAyQXpkd/KvwqnvM3Pp7LUSrt6xCkHTbzX23TsGqRfXvqnRDVXH4dZ5EEX0a UGfI1NmI2fPQJexjw6SoJyjrDoJ50HHFlnA54xqNsOYB5+s0OXLQ9dy+har7CEqLs62wi56kn yl/5py+VwSDv0qFvqJoeBZHa21hL+294Jq9n3Zfx62u9s7D6MmAtJctm66Ur/6eRdgv8LOIoO xC5bVJHp2aUyYd7sl6OSZ/74dQyUfCDXUY4Xmg0rJvb9bIrietKepjlO2gKWyElWDOV5fagmP ROaJ2EfJ3fmyO5W93w3NynxtXjOuK/nl2DwiYs1zWm/2ZP7Mt6xAzfN/uNtnsqVRzyz2nmnhm AshV53XU5ynHotlKWCE8tjVFZAUr1mMCilmaG9QiW2LO059Zixz8hjkRcMIRLZevFvzh8imFr 9MrjFBtxMc3nFudE7l6OdtOixEGs9A0ja4n3YndrDRXAW1O8Bd10MCNDMNpeHLxGPvupMpHmq 5bqzd067kKxjK7dXg77RL5ppURDc888eU4VFq27IODSgPop/ee6/oi1Ymyxy3FtyB+JvEgHAJ h7dh9UakWLc8fDhioGhu45jAva50H4dP3T9cuCOh8hjAJWMNa/yjnzguA1czVw75DoH/4XuAV SAlGStg X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Dec 2016 17:55:33 -0000 Hi, I got a panic while trying to use a zfs / unionfs combination for jails. The following script can be used to reproduce the panic: basejail=/jail mkdir -p /tmp/i24420/n0 mkdir -p /tmp/i24420/upper/n0 mount_nullfs -o ro $basejail /tmp/i24420/n0 mount_unionfs -o noatime /tmp/i24420/upper/n0 /tmp/i24420/n0 mkdir -p /tmp/i24420/n1 mkdir -p /tmp/i24420/upper/n1 mount_nullfs -o ro $basejail /tmp/i24420/n1 mount_unionfs -o noatime /tmp/i24420/upper/n1 /tmp/i24420/n1 mkdir -p /tmp/i24420/n2 mkdir -p /tmp/i24420/upper/n2 mount_nullfs -o ro $basejail /tmp/i24420/n2 mount_unionfs -o noatime /tmp/i24420/upper/n2 /tmp/i24420/n2 mkdir -p /tmp/i24420/n3 mkdir -p /tmp/i24420/upper/n3 mount_nullfs -o ro $basejail /tmp/i24420/n3 mount_unionfs -o noatime /tmp/i24420/upper/n3 /tmp/i24420/n3 chroot /tmp/i24420/n0 rm -rv /var Any ideas how to debug this further? TIA, Nikos > panic: solaris assert: cnp->cn_namelen < sizeof(nm), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c, line: 4945 > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > lock order reversal: > 1st 0xfffff8000eb6b7c8 zfs (zfs) @ /usr/src/sys/kern/vfs_lookup.c:1080 > 2nd 0xfffff8000eba85f0 unionfs (unionfs) @ /usr/src/sys/modules/unionfs/../../fs/unionfs/union_subr.c:359 > stack backtrace: > #0 0xffffffff80ac0ed0 at witness_debugger+0x70 > #1 0xffffffff80ac0dc4 at witness_checkorder+0xe54 > #2 0xffffffff80a381f2 at __lockmgr_args+0x4c2 > #3 0xffffffff828208ef at unionfs_noderem+0x19f > #4 0xffffffff82825fd4 at unionfs_reclaim+0x14 > #5 0xffffffff810663b0 at VOP_RECLAIM_APV+0xe0 > #6 0xffffffff80b2901e at vgonel+0x20e > #7 0xffffffff80b2955e at vrecycle+0x3e > #8 0xffffffff82825fbc at unionfs_inactive+0x1c > #9 0xffffffff81066250 at VOP_INACTIVE_APV+0xe0 > #10 0xffffffff80b27e56 at vinactive+0xc6 > #11 0xffffffff80b2839e at vputx+0x30e > #12 0xffffffff80b31d41 at kern_rmdirat+0x1e1 > #13 0xffffffff80ef8d44 at amd64_syscall+0x314 > #14 0xffffffff80ed84ab at Xfast_syscall+0xfb > panic: solaris assert: cnp->cn_namelen < sizeof(nm), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c, line: 4945 > cpuid = 0 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00952092a0 > vpanic() at vpanic+0x182/frame 0xfffffe0095209320 > panic() at panic+0x43/frame 0xfffffe0095209380 > assfail() at assfail+0x1a/frame 0xfffffe0095209390 > zfs_freebsd_lookup() at zfs_freebsd_lookup+0x47/frame 0xfffffe00952094d0 > VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xda/frame 0xfffffe0095209500 > vfs_cache_lookup() at vfs_cache_lookup+0xd6/frame 0xfffffe0095209560 > VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xda/frame 0xfffffe0095209590 > relookup() at relookup+0xa1/frame 0xfffffe00952095f0 > unionfs_relookup() at unionfs_relookup+0xfa/frame 0xfffffe0095209650 > unionfs_remove() at unionfs_remove+0xc8/frame 0xfffffe0095209710 > VOP_REMOVE_APV() at VOP_REMOVE_APV+0xe0/frame 0xfffffe0095209740 > kern_unlinkat() at kern_unlinkat+0x208/frame 0xfffffe0095209930 > amd64_syscall() at amd64_syscall+0x314/frame 0xfffffe0095209ab0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0095209ab0 > --- syscall (10, FreeBSD ELF64, sys_unlink), rip = 0x80097915a, rsp = 0x7fffffffea58, rbp = 0x7fffffffeb20 --- > KDB: enter: panic > #0 doadump (textdump=-5) at pcpu.h:222 > 222 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=-5) at pcpu.h:222 > #1 0xffffffff80397db6 in db_fncall (dummy1=, > dummy2=, dummy3=, > dummy4=) at /usr/src/sys/ddb/db_command.c:581 > #2 0xffffffff80397919 in db_command (cmd_table=) > at /usr/src/sys/ddb/db_command.c:453 > #3 0xffffffff80397674 in db_command_loop () > at /usr/src/sys/ddb/db_command.c:506 > #4 0xffffffff8039a72f in db_trap (type=, > code=) at /usr/src/sys/ddb/db_main.c:248 > #5 0xffffffff80aa2db3 in kdb_trap (type=, > code=, tf=) > at /usr/src/sys/kern/subr_kdb.c:654 > #6 0xffffffff80ef7cc4 in trap (frame=0xfffffe00952091d0) > at /usr/src/sys/amd64/amd64/trap.c:537 > #7 0xffffffff80ed81c1 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:236 > #8 0xffffffff80aa245b in kdb_enter (why=0xffffffff814339c1 "panic", > msg=0x80
) at cpufunc.h:63 > #9 0xffffffff80a609bf in vpanic (fmt=, > ap=0xfffffe0095209360) at /usr/src/sys/kern/kern_shutdown.c:752 > #10 0xffffffff80a60a23 in panic (fmt=0xffffffff81c23810 "\004") > at /usr/src/sys/kern/kern_shutdown.c:690 > #11 0xffffffff825cb23a in assfail (a=, > f=, l=) > at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81 > #12 0xffffffff8231f957 in zfs_freebsd_lookup (ap=0xfffffe0095209518) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4945 > #13 0xffffffff81063caa in VOP_CACHEDLOOKUP_APV (vop=, > a=) at vnode_if.c:195 > #14 0xffffffff80b12786 in vfs_cache_lookup (ap=) > at vnode_if.h:80 > #15 0xffffffff81063b4a in VOP_LOOKUP_APV (vop=, > a=) at vnode_if.c:127 > #16 0xffffffff80b1c611 in relookup (dvp=0xfffff8000eb6ab10, > vpp=0xfffffe00952096c0, cnp=0xfffffe0095209680) at vnode_if.h:54 > #17 0xffffffff82820f9a in unionfs_relookup (dvp=0xfffff8000eb6ab10, > vpp=0xfffffe00952096c0, cnp=0xfffffe0095209890, cn=0xfffffe0095209680, > td=0xfffff8000e95e000, path=, pathlen=10240, > nameiop=) > at /usr/src/sys/modules/unionfs/../../fs/unionfs/union_subr.c:544 > #18 0xffffffff82824cf8 in unionfs_remove (ap=0xfffffe00952098e8) > at /usr/src/sys/modules/unionfs/../../fs/unionfs/union_vnops.c:992 > #19 0xffffffff810656d0 in VOP_REMOVE_APV (vop=, > a=) at vnode_if.c:1400 > #20 0xffffffff80b31ab8 in kern_unlinkat (td=, fd=-100, > path=0x800e20748
, > pathseg=UIO_USERSPACE, oldinum=) at vnode_if.h:575 > #21 0xffffffff80ef8d44 in amd64_syscall (td=0xfffff8000e95e000, traced=0) > at subr_syscall.c:135 > #22 0xffffffff80ed84ab in Xfast_syscall () > at /usr/src/sys/amd64/amd64/exception.S:396 > #23 0x000000080097915a in ?? () > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > (kgdb)