From owner-freebsd-fs@freebsd.org Mon Mar 11 15:02:07 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46FAE1545984 for ; Mon, 11 Mar 2019 15:02:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D0C736B027 for ; Mon, 11 Mar 2019 15:02:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8F7181545980; Mon, 11 Mar 2019 15:02:06 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F68B154597F for ; Mon, 11 Mar 2019 15:02:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA2386B020 for ; Mon, 11 Mar 2019 15:02:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 2A5F6241F for ; Mon, 11 Mar 2019 15:02:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2BF25YF031727 for ; Mon, 11 Mar 2019 15:02:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2BF25FN031715 for fs@FreeBSD.org; Mon, 11 Mar 2019 15:02:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 236220] ZFS vnode deadlock on zfs_mknode Date: Mon, 11 Mar 2019 15:02:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ncrogers@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2019 15:02:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236220 --- Comment #10 from ncrogers@gmail.com --- I finally got a system to panic with INVARIANTS enabled. panic: solaris assert: dn->dn_type !=3D DMU_OT_NONE, file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c, line: 2540 Below is the full panic and backtrace. Let me know if there's something hel= pful I can provide from kgdb. I also have a second panic but it looks the same. courtland# kgdb /boot/kernel/kernel /var/crash/vmcore.0 GNU gdb (GDB) 8.2.1 [GDB v8.2.1 for FreeBSD] Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd12.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /boot/kernel/kernel...Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug...done. done. Unread portion of the kernel message buffer: panic: solaris assert: dn->dn_type !=3D DMU_OT_NONE, file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c, line: 2540 cpuid =3D 1 time =3D 1552104873 KDB: stack backtrace: #0 0xffffffff80d3bf77 at kdb_backtrace+0x67 #1 0xffffffff80cf4a53 at vpanic+0x1a3 #2 0xffffffff80cf4833 at panic+0x43 #3 0xffffffff803bbcba at assfail+0x1a #4 0xffffffff803f094b at dbuf_create+0x9b #5 0xffffffff803f08a1 at dbuf_create_bonus+0x71 #6 0xffffffff803f8d10 at dmu_bonus_hold+0xb0 #7 0xffffffff804cde29 at zfs_zget+0x69 #8 0xffffffff804bff37 at zfs_get_data+0x67 #9 0xffffffff804d3faf at zil_commit_impl+0x15af #10 0xffffffff804c73bb at zfs_freebsd_fsync+0xcb #11 0xffffffff813861be at VOP_FSYNC_APV+0x9e #12 0xffffffff80dab2d4 at vop_stdfdatasync+0x34 #13 0xffffffff8138a84e at VOP_FDATASYNC_APV+0x9e #14 0xffffffff80dc7bd3 at kern_fsync+0x1b3 #15 0xffffffff811fc532 at amd64_syscall+0x272 #16 0xffffffff811d65ad at fast_syscall_common+0x101 Uptime: 3d10h28m54s Dumping 4867 out of 16241 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..= 91% __curthread () at ./machine/pcpu.h:230 230 ./machine/pcpu.h: No such file or directory. (kgdb) backtrace #0 __curthread () at ./machine/pcpu.h:230 #1 doadump (textdump=3D) at /usr/src/sys/kern/kern_shutdown= .c:366 #2 0xffffffff80cf468b in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:446 #3 0xffffffff80cf4ab3 in vpanic (fmt=3D, ap=3D0xfffffe00b77= c54a0) at /usr/src/sys/kern/kern_shutdown.c:872 #4 0xffffffff80cf4833 in panic (fmt=3D) at /usr/src/sys/kern/kern_shutdown.c:799 #5 0xffffffff803bbcba in assfail (a=3D, f=3D, l=3D) at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81 #6 0xffffffff803f094b in dbuf_create (dn=3D0xfffff8015f77a770, level=3D0 '= \000', blkid=3D18446744073709551615,=20 parent=3D0xfffff80296803c60, blkptr=3D0x0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:2540 #7 0xffffffff803f08a1 in dbuf_create_bonus (dn=3D0xfffff8015f77a770) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:3085 #8 0xffffffff803f8d10 in dmu_bonus_hold (os=3D, object=3D, tag=3D0x0,=20 dbp=3D0xfffffe00b77c5648) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:345 #9 0xffffffff804cde29 in zfs_zget (zfsvfs=3D0xfffffe008ea28000, obj_num=3D= 33262, zpp=3D0xfffffe00b77c56c0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c:= 1137 #10 0xffffffff804bff37 in zfs_get_data (arg=3D0xfffffe008ea28000, lr=3D0xfffffe00a6620300, buf=3D0x0,=20 lwb=3D0xfffff8042cf828c0, zio=3D0xfffff80047eb5000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:= 1297 #11 0xffffffff804d3faf in zil_lwb_commit (zilog=3D, lwb=3D, itx=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1493 #12 zil_process_commit_list (zilog=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2069 #13 zil_commit_writer (zilog=3D, zcw=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2201 #14 zil_commit_impl (zilog=3D, foid=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:2712 #15 0xffffffff804c73bb in zfs_fsync (syncflag=3D0, vp=3D, cr=3D, ct=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:= 2627 #16 zfs_freebsd_fsync (ap=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:= 5079 #17 0xffffffff813861be in VOP_FSYNC_APV (vop=3D0xffffffff81c64f70 , a=3D0xfffffe00b77c58c0) at vnode_if.c:1331 #18 0xffffffff80dab2d4 in VOP_FSYNC (vp=3D, waitfor=3D1, td=3D= ) at ./vnode_if.h:549 #19 vop_stdfdatasync (ap=3D) at /usr/src/sys/kern/vfs_default.c:732 #20 0xffffffff8138a84e in VOP_FDATASYNC_APV (vop=3D0xffffffff81d3b338 , a=3D0xfffffe00b77c5938) at vnode_if.c:4437 #21 0xffffffff80dc7bd3 in VOP_FDATASYNC (vp=3D, td=3D) at ./vnode_if.h:1849 #22 kern_fsync (td=3D0xfffff8000bb96580, fd=3D, fullsync=3Df= alse) at /usr/src/sys/kern/vfs_syscalls.c:3374 #23 0xffffffff811fc532 in syscallenter (td=3D0xfffff8000bb96580) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:135 #24 amd64_syscall (td=3D0xfffff8000bb96580, traced=3D0) at /usr/src/sys/amd64/amd64/trap.c:1076 #25 #26 0x000000080131e8aa in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffd078 (kgdb) --=20 You are receiving this mail because: You are the assignee for the bug.=