From owner-freebsd-fs@FreeBSD.ORG Mon Sep 9 16:30:03 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 89F7FCA9 for ; Mon, 9 Sep 2013 16:30:03 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from dss.incore.de (dss.incore.de [195.145.1.138]) by mx1.freebsd.org (Postfix) with ESMTP id 2E5A822F2 for ; Mon, 9 Sep 2013 16:30:03 +0000 (UTC) Received: from inetmail.dmz (inetmail.dmz [10.3.0.3]) by dss.incore.de (Postfix) with ESMTP id 65D7C5DCCD for ; Mon, 9 Sep 2013 18:22:03 +0200 (CEST) X-Virus-Scanned: amavisd-new at incore.de Received: from dss.incore.de ([10.3.0.3]) by inetmail.dmz (inetmail.dmz [10.3.0.3]) (amavisd-new, port 10024) with LMTP id AGbQQ0dfFeKV for ; Mon, 9 Sep 2013 18:22:02 +0200 (CEST) Received: from mail.incore (fwintern.dmz [10.0.0.253]) by dss.incore.de (Postfix) with ESMTP id E67475DCCC for ; Mon, 9 Sep 2013 18:22:01 +0200 (CEST) Received: from bsdlo.incore (bsdlo.incore [192.168.0.84]) by mail.incore (Postfix) with ESMTP id DBAF350B65 for ; Mon, 9 Sep 2013 18:22:01 +0200 (CEST) Message-ID: <522DF5A9.4070103@incore.de> Date: Mon, 09 Sep 2013 18:22:01 +0200 From: Andreas Longwitz User-Agent: Thunderbird 2.0.0.19 (X11/20090113) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: zfs panic during find(1) on zfs snapshot directory Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Sep 2013 16:30:03 -0000 Hello, I run Freebsd 8.4-Stable r253040 completed with adapted r244795, r244925 and r245286 from head. During an amanda backup which created the zfs snapshot /backup/jail/db1/.zfs/snapshot/amanda-mpool_jail_db1_backup-current the command find /backup -name 'sysout.*' -type f -mtime +100 -exec rm {} caused a panic on the lstat system call for the name of the snapshot directory. Console output: panic: __lockmgr_args: recursing on non recursive lockmgr zfs @ /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/gfs.c:451 cpuid = 2 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x1ce __lockmgr_args() at __lockmgr_args+0xb68 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x70 _vn_lock() at _vn_lock+0x47 gfs_lookup_dot() at gfs_lookup_dot+0xa9 gfs_dir_lookup() at gfs_dir_lookup+0x49 zfsctl_snapshot_inactive() at zfsctl_snapshot_inactive+0x81 VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0x68 vinactive() at vinactive+0x71 vputx() at vputx+0x2d8 traverse() at traverse+0xa3 zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x1bb VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x62 lookup() at lookup+0x44c namei() at namei+0x53d kern_statat_vnhook() at kern_statat_vnhook+0x8f kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a amd64_syscall() at amd64_syscall+0x1f4 Xfast_syscall() at Xfast_syscall+0xfc --- syscall (190, FreeBSD ELF64, lstat), rip = 0x18073597c, rsp = 0x7fffffffea68, rbp = 0x180a2d350 --- KDB: enter: panic 0xffffff0091bf6588: tag zfs, type VDIR usecount 5, writecount 0, refcount 5 mountedhere 0 flags () lock type zfs: EXCL by thread 0xffffff01aabf6470 (pid 29384) 0xffffff01940fc1d8: tag zfs, type VDIR usecount 0, writecount 0, refcount 1 mountedhere 0 flags (VI_DOINGINACT) lock type zfs: EXCL by thread 0xffffff01aabf6470 (pid 29384) >From kerneldump: (kgdb) bt #0 doadump () at /usr/src/sys/kern/kern_shutdown.c:266 #1 0xffffffff801f87fc in db_fncall (dummy1=, dummy2=, dummy3=, dummy4=) at /usr/src/sys/ddb/db_command.c:548 #2 0xffffffff801f8aad in db_command (last_cmdp=0xffffffff8086bdc0, cmd_table=, dopager=0) at /usr/src/sys/ddb/db_command.c:445 #3 0xffffffff801fd163 in db_script_exec (scriptname=0xffffff8245eeec00 "kdb.enter.panic", warnifnotfound=0) at /usr/src/sys/ddb/db_script.c:302 #4 0xffffffff801fd232 in db_script_kdbenter (eventname=) at /usr/src/sys/ddb/db_script.c:324 #5 0xffffffff801fae44 in db_trap (type=, code=) at /usr/src/sys/ddb/db_main.c:230 #6 0xffffffff80432e61 in kdb_trap (type=3, code=0, tf=0xffffff8245eeee30) at /usr/src/sys/kern/subr_kdb.c:654 #7 0xffffffff805dc82f in trap (frame=0xffffff8245eeee30) at /usr/src/sys/amd64/amd64/trap.c:574 #8 0xffffffff805c2ba4 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #9 0xffffffff804328fb in kdb_enter (why=0xffffffff8069543a "panic", msg=0xa
) at cpufunc.h:63 #10 0xffffffff803ff367 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:616 #11 0xffffffff803e7df8 in __lockmgr_args (lk=0xffffff0091bf6620, flags=0, ilk=0xffffff0091bf6648, wmesg=, pri=982528, timo=1173286416, file=0xffffffff80b68978 "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/gfs.c", line=451) at /usr/src/sys/kern/kern_lock.c:704 #12 0xffffffff804833e9 in vop_stdlock (ap=) at lockmgr.h:94 #13 0xffffffff8063a870 in VOP_LOCK1_APV (vop=0xffffffff8082af60, a=0xffffff8245eef150) at vnode_if.c:2052 #14 0xffffffff804a3137 in _vn_lock (vp=0xffffff0091bf6588, flags=525312, file=0xffffffff80b68978 "/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/gfs.c", line=451) at vnode_if.h:859 #15 0xffffffff80a66489 in gfs_lookup_dot (vpp=0xffffff8245eef2c8, dvp=0xffffff0091bf6588, pvp=0xffffff0091bf6588, nm=0xffffffff80b7ab31 "..") at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/gfs.c:451 #16 0xffffffff80a664e9 in gfs_dir_lookup (dvp=0xffffff01940fc1d8, nm=0xffffffff80b7ab31 "..", vpp=0xffffff8245eef2c8, cr=0xffffff01e58b6300, flags=0, direntflags=0x0, realpnp=0x0) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/gfs.c:920 #17 0xffffffff80af4ad1 in zfsctl_snapshot_inactive (ap=) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:1470 #18 0xffffffff80639638 in VOP_INACTIVE_APV (vop=0xffffffff80b83f60, a=0xffffff8245eef340) at vnode_if.c:1923 #19 0xffffffff80491671 in vinactive (vp=0xffffff01940fc1d8, td=0xffffff01aabf6470) at vnode_if.h:807 #20 0xffffffff80496038 in vputx (vp=0xffffff01940fc1d8, func=1) at /usr/src/sys/kern/vfs_subr.c:2347 #21 0xffffffff80a63d73 in traverse (cvpp=0xffffff8245eef930, lktype=525312) at /usr/src/sys/modules/zfs/../../cddl/compat/opensolaris/kern/opensolaris_lookup.c:98 #22 0xffffffff80af4dbb in zfsctl_snapdir_lookup (ap=) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ctldir.c:1007 #23 0xffffffff8063a372 in VOP_LOOKUP_APV (vop=0xffffffff80b845a0, a=0xffffff8245eef7c0) at vnode_if.c:126 #24 0xffffffff8048898c in lookup (ndp=0xffffff8245eef900) at vnode_if.h:54 #25 0xffffffff80489b0d in namei (ndp=0xffffff8245eef900) at /usr/src/sys/kern/vfs_lookup.c:269 #26 0xffffffff8049a47f in kern_statat_vnhook (td=0xffffff01aabf6470, flag=, fd=, path=, pathseg=, sbp=0xffffff8245eefa80, hook=0) at /usr/src/sys/kern/vfs_syscalls.c:2339 #27 0xffffffff8049a645 in kern_statat (td=, flag=, fd=, path=, pathseg=, sbp=) at /usr/src/sys/kern/vfs_syscalls.c:2320 #28 0xffffffff8049a70a in lstat (td=, uap=0xffffff8245eefbb0) at /usr/src/sys/kern/vfs_syscalls.c:2383 #29 0xffffffff805db824 in amd64_syscall (td=0xffffff01aabf6470, traced=0) at subr_syscall.c:114 #30 0xffffffff805c2e9c in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 #31 0x000000018073597c in ?? () (kgdb) f 25 #25 0xffffffff80489b0d in namei (ndp=0xffffff8245eef900) at /usr/src/sys/kern/vfs_lookup.c:269 269 error = lookup(ndp); (kgdb) l 264 VREF(dp); 265 } 266 if (vfslocked) 267 ndp->ni_cnd.cn_flags |= GIANTHELD; 268 ndp->ni_startdir = dp; 269 error = lookup(ndp); 270 if (error) { 271 uma_zfree(namei_zone, cnp->cn_pnbuf); 272 #ifdef DIAGNOSTIC 273 cnp->cn_pnbuf = NULL; (kgdb) p *ndp $23 = {ni_dirp = 0x180a2d3c8
, ni_segflg = UIO_USERSPACE, ni_startdir = 0x0, ni_rootdir = 0xffffff0002c493b0, ni_topdir = 0x0, ni_dirfd = -100, ni_vp = 0xffffff01940fc1d8, ni_dvp = 0xffffff0091bf6588, ni_pathlen = 1, ni_next = 0xffffff0026476424 "", ni_loopcnt = 0, ni_cnd = {cn_nameiop = 0, cn_flags = 83935492, cn_thread = 0xffffff01aabf6470, cn_cred = 0xffffff01e58b6300, cn_lkflags = 2097152, cn_pnbuf = 0xffffff0026476400 "amanda-mpool_jail_db1_backup-current", cn_nameptr = 0xffffff0026476400 "amanda-mpool_jail_db1_backup-current", cn_namelen = 36, cn_consume = 0}} I would like to know if this panic is a known issue and can give more information from the kerneldump. -- Andreas Longwitz