From owner-freebsd-current@freebsd.org Tue Dec 29 19:54:54 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D710A55BF8 for ; Tue, 29 Dec 2015 19:54:54 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pf0-x22f.google.com (mail-pf0-x22f.google.com [IPv6:2607:f8b0:400e:c00::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 25EBE1B96 for ; Tue, 29 Dec 2015 19:54:54 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pf0-x22f.google.com with SMTP id e65so78812381pfe.1 for ; Tue, 29 Dec 2015 11:54:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=jMowxkml72YpAXIPvB6kvHOy5A5d2lK+5AiFhQ7mRE4=; b=IAOWCoY58kzCCBgRt3rCcKekbp/qlaaBlWSyFsHxNua5D/0VVSZ+Cq/LiINzpEwOSz E4BSQe/uS74PrBqKGpKltqWFmeQ0rfGmudtoGUWq77+PLT7EWkRT88pSwkl6k/3fGln5 XwiQQ1joGR8NSHiErDhATKrqPcAk95dNixR8bvDGSFfdg7KsQD1KZKbbkcruKz2RhS3o Kne49YgGg3mVSomkoBDEC+TictfSYoDRyf4TiL0J92Hq97vS5j6TB3uB+05kiJK2LSdK gzpGddEB3tyImvHwuG/mE4T4RxFHVoNMv8wnTYmhXT1x09hrQNR2xVhFJrDBNeeQcqf2 9tGw== X-Received: by 10.98.89.207 with SMTP id k76mr71911911pfj.36.1451418893732; Tue, 29 Dec 2015 11:54:53 -0800 (PST) Received: from ?IPv6:2601:601:800:126d:fd97:b044:5389:fa9b? ([2601:601:800:126d:fd97:b044:5389:fa9b]) by smtp.gmail.com with ESMTPSA id q27sm41330455pfi.83.2015.12.29.11.54.52 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 29 Dec 2015 11:54:52 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: Stack backtrace on `zpool export` From: NGie Cooper In-Reply-To: <5B1F1E9E-7651-4C1D-807F-A875EDEF8705@fsfe.org> Date: Tue, 29 Dec 2015 11:54:51 -0800 Cc: freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <3B435E9A-F93A-407E-AB54-C7B98291269B@gmail.com> References: <5B1F1E9E-7651-4C1D-807F-A875EDEF8705@fsfe.org> To: Florian Ermisch <0xf10e@fsfe.org> X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Dec 2015 19:54:54 -0000 > On Dec 29, 2015, at 11:48, Florian Ermisch <0xf10e@fsfe.org> wrote: >=20 > Hi *, >=20 > since I've upgraded my laptop from 10.2-RELEASE to 11-CURRENT = (r292536, now r292755) I see this stack backtrace when a zpool is = exported: >=20 > Dec 27 18:44:02 fuchi-cyber220 kernel: lock order reversal: > Dec 27 18:44:02 fuchi-cyber220 kernel: 1st 0xfffff800c7472418 zfs = (zfs) @ /usr/src/sys/kern/vfs_mount.c:1224 > Dec 27 18:44:02 fuchi-cyber220 kernel: 2nd 0xfffff800c73fad50 zfs_gfs = (zfs_gfs) @ = /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/gfs.c:494 > Dec 27 18:44:02 fuchi-cyber220 kernel: stack backtrace: > Dec 27 18:44:02 fuchi-cyber220 kernel: #0 0xffffffff80a7d6f0 at = witness_debugger+0x70 > Dec 27 18:44:02 fuchi-cyber220 kernel: #1 0xffffffff80a7d5f1 at = witness_checkorder+0xe71 > Dec 27 18:44:02 fuchi-cyber220 kernel: #2 0xffffffff809fedcb at = __lockmgr_args+0xd3b > Dec 27 18:44:02 fuchi-cyber220 kernel: #3 0xffffffff80ac55ec at = vop_stdlock+0x3c > Dec 27 18:44:02 fuchi-cyber220 kernel: #4 0xffffffff80fbb220 at = VOP_LOCK1_APV+0x100 > Dec 27 18:44:02 fuchi-cyber220 kernel: #5 0xffffffff80ae653a at = _vn_lock+0x9a > Dec 27 18:44:02 fuchi-cyber220 kernel: #6 0xffffffff8209db13 at = gfs_file_create+0x73 > Dec 27 18:44:02 fuchi-cyber220 kernel: #7 0xffffffff8209dbbd at = gfs_dir_create+0x1d > Dec 27 18:44:02 fuchi-cyber220 kernel: #8 0xffffffff821649e7 at = zfsctl_mknode_snapdir+0x47 > Dec 27 18:44:02 fuchi-cyber220 kernel: #9 0xffffffff8209e135 at = gfs_dir_lookup+0x185 > Dec 27 18:44:02 fuchi-cyber220 kernel: #10 0xffffffff8209e61d at = gfs_vop_lookup+0x1d > Dec 27 18:44:02 fuchi-cyber220 kernel: #11 0xffffffff82163a05 at = zfsctl_root_lookup+0xf5 > Dec 27 18:44:02 fuchi-cyber220 kernel: #12 0xffffffff821648a3 at = zfsctl_umount_snapshots+0x83 > Dec 27 18:44:02 fuchi-cyber220 kernel: #13 0xffffffff8217d5ab at = zfs_umount+0x7b > Dec 27 18:44:02 fuchi-cyber220 kernel: #14 0xffffffff80acf0b0 at = dounmount+0x530 > Dec 27 18:44:02 fuchi-cyber220 kernel: #15 0xffffffff80aceaed at = sys_unmount+0x35d > Dec 27 18:44:02 fuchi-cyber220 kernel: #16 0xffffffff80e6e13b at = amd64_syscall+0x2db > Dec 27 18:44:02 fuchi-cyber220 kernel: #17 0xffffffff80e4dd8b at = Xfast_syscall+0xfb >=20 > (=46rom /var/log/messages) >=20 > First I've only seen this on the console at shutdown or reboot (after = sync I think) but later found I can reproduce it by exporting a zpool. > While the only pools I can trigger this without a shutdown/reboot are = connected via USB(3) I still see it just before poweroff at shutdown = when the system's root pool is synced. >=20 > When I try to export a zpool under heavy load (`make -C /use/src -j 4 = buildworld` on a 2 core CPU w/ HT) the system locks up completely. I = don't think it's related to memory pressure as I haven't seen swap being = used during a buildworld (with 8 gigs of RAM). >=20 > Has anyone else seen this? Please file a bug for this issue and assign it to freebsd-fs. It = doesn=E2=80=99t look familiar (there might be a lock ordering issue with = either zfs or vfs+zfs). Thanks! -NGie=