From owner-freebsd-current@freebsd.org Tue Dec 29 19:48:38 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9978BA559BB for ; Tue, 29 Dec 2015 19:48:38 +0000 (UTC) (envelope-from 0xf10e@fsfe.org) Received: from mail-2.alumni.tu-berlin.de (mail-2.alumni.tu-berlin.de [130.149.5.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 629D61804 for ; Tue, 29 Dec 2015 19:48:38 +0000 (UTC) (envelope-from 0xf10e@fsfe.org) X-tubIT-Incoming-IP: 89.204.139.41 Received: from [89.204.139.41] (helo=[192.168.43.66]) by mailbox.alumni.tu-berlin.de (exim-4.76) with esmtpsa [UNKNOWN:AES256-SHA:256] for id 1aE0Fy-0008Tw-8P; Tue, 29 Dec 2015 20:48:30 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Subject: Stack backtrace on `zpool export` From: Florian Ermisch <0xf10e@fsfe.org> Date: Tue, 29 Dec 2015 20:48:27 +0100 To: freebsd-current@freebsd.org Message-ID: <5B1F1E9E-7651-4C1D-807F-A875EDEF8705@fsfe.org> X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Dec 2015 19:48:38 -0000 Hi *, since I've upgraded my laptop from 10.2-RELEASE to 11-CURRENT (r292536, now r292755) I see this stack backtrace when a zpool is exported: Dec 27 18:44:02 fuchi-cyber220 kernel: lock order reversal: Dec 27 18:44:02 fuchi-cyber220 kernel: 1st 0xfffff800c7472418 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1224 Dec 27 18:44:02 fuchi-cyber220 kernel: 2nd 0xfffff800c73fad50 zfs_gfs (zfs_gfs) @ /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/gfs.c:494 Dec 27 18:44:02 fuchi-cyber220 kernel: stack backtrace: Dec 27 18:44:02 fuchi-cyber220 kernel: #0 0xffffffff80a7d6f0 at witness_debugger+0x70 Dec 27 18:44:02 fuchi-cyber220 kernel: #1 0xffffffff80a7d5f1 at witness_checkorder+0xe71 Dec 27 18:44:02 fuchi-cyber220 kernel: #2 0xffffffff809fedcb at __lockmgr_args+0xd3b Dec 27 18:44:02 fuchi-cyber220 kernel: #3 0xffffffff80ac55ec at vop_stdlock+0x3c Dec 27 18:44:02 fuchi-cyber220 kernel: #4 0xffffffff80fbb220 at VOP_LOCK1_APV+0x100 Dec 27 18:44:02 fuchi-cyber220 kernel: #5 0xffffffff80ae653a at _vn_lock+0x9a Dec 27 18:44:02 fuchi-cyber220 kernel: #6 0xffffffff8209db13 at gfs_file_create+0x73 Dec 27 18:44:02 fuchi-cyber220 kernel: #7 0xffffffff8209dbbd at gfs_dir_create+0x1d Dec 27 18:44:02 fuchi-cyber220 kernel: #8 0xffffffff821649e7 at zfsctl_mknode_snapdir+0x47 Dec 27 18:44:02 fuchi-cyber220 kernel: #9 0xffffffff8209e135 at gfs_dir_lookup+0x185 Dec 27 18:44:02 fuchi-cyber220 kernel: #10 0xffffffff8209e61d at gfs_vop_lookup+0x1d Dec 27 18:44:02 fuchi-cyber220 kernel: #11 0xffffffff82163a05 at zfsctl_root_lookup+0xf5 Dec 27 18:44:02 fuchi-cyber220 kernel: #12 0xffffffff821648a3 at zfsctl_umount_snapshots+0x83 Dec 27 18:44:02 fuchi-cyber220 kernel: #13 0xffffffff8217d5ab at zfs_umount+0x7b Dec 27 18:44:02 fuchi-cyber220 kernel: #14 0xffffffff80acf0b0 at dounmount+0x530 Dec 27 18:44:02 fuchi-cyber220 kernel: #15 0xffffffff80aceaed at sys_unmount+0x35d Dec 27 18:44:02 fuchi-cyber220 kernel: #16 0xffffffff80e6e13b at amd64_syscall+0x2db Dec 27 18:44:02 fuchi-cyber220 kernel: #17 0xffffffff80e4dd8b at Xfast_syscall+0xfb (From /var/log/messages) First I've only seen this on the console at shutdown or reboot (after sync I think) but later found I can reproduce it by exporting a zpool. While the only pools I can trigger this without a shutdown/reboot are connected via USB(3) I still see it just before poweroff at shutdown when the system's root pool is synced. When I try to export a zpool under heavy load (`make -C /use/src -j 4 buildworld` on a 2 core CPU w/ HT) the system locks up completely. I don't think it's related to memory pressure as I haven't seen swap being used during a buildworld (with 8 gigs of RAM). Has anyone else seen this? Regards, Florian