From owner-freebsd-bugs@FreeBSD.ORG Fri Jun 28 15:20:00 2013 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BB07D608 for ; Fri, 28 Jun 2013 15:20:00 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 9CC45124E for ; Fri, 28 Jun 2013 15:20:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r5SFK0iB033367 for ; Fri, 28 Jun 2013 15:20:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r5SFK09n033366; Fri, 28 Jun 2013 15:20:00 GMT (envelope-from gnats) Resent-Date: Fri, 28 Jun 2013 15:20:00 GMT Resent-Message-Id: <201306281520.r5SFK09n033366@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, James TD Smith Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A904B4AE for ; Fri, 28 Jun 2013 15:13:58 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121]) by mx1.freebsd.org (Postfix) with ESMTP id 9B4B81206 for ; Fri, 28 Jun 2013 15:13:58 +0000 (UTC) Received: from oldred.freebsd.org ([127.0.1.6]) by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r5SFDuwo066303 for ; Fri, 28 Jun 2013 15:13:56 GMT (envelope-from nobody@oldred.freebsd.org) Received: (from nobody@localhost) by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r5SFDuNV066296; Fri, 28 Jun 2013 15:13:56 GMT (envelope-from nobody) Message-Id: <201306281513.r5SFDuNV066296@oldred.freebsd.org> Date: Fri, 28 Jun 2013 15:13:56 GMT From: James TD Smith To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Subject: kern/180060: ZFS kernel panic, solaris assert on dsl_prop_unregister X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jun 2013 15:20:00 -0000 >Number: 180060 >Category: kern >Synopsis: ZFS kernel panic, solaris assert on dsl_prop_unregister >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Jun 28 15:20:00 UTC 2013 >Closed-Date: >Last-Modified: >Originator: James TD Smith >Release: 9.1 >Organization: >Environment: FreeBSD masada.internal.mohorovi.cc 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #5 r251934: Tue Jun 18 15:41:34 BST 2013 root@masada.internal.mohorovi.cc:/usr/obj/usr/src/sys/MASADA amd64 >Description: I've had two ZFS-related kernel panics over the last few days. Both have happened since I upgraded to 9.1-RELEASE-4 on the 18th. PR 154447 has similar backtraces but was on 8.2, I can't find any other references to a similar problem. It's a mixed UFS/ZFS system, root is on UFS with a single 4-disk RAIDZ1 pool used for backups, a media library, poudriere and a few other jails. I replaced one of the disks in the pool earlier this week which had been failing for a while and have been tuning the ZFS settings based on this guide: http://icesquare.com/wordpress/how-to-improve-zfs-performance/ after I upgraded to 4Gb RAM. Jun 25 08:12:57 masada kernel: panic: solaris assert: dsl_prop_unregister(ds, "atime", atime_changed_cb, zfsvfs) == 0, file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c, line: 1199 Jun 25 08:12:57 masada kernel: cpuid = 0 Jun 25 08:12:57 masada kernel: KDB: stack backtrace: Jun 25 08:12:57 masada kernel: #0 0xffffffff80926c96 at kdb_backtrace+0x66 Jun 25 08:12:57 masada kernel: #1 0xffffffff808f0cae at panic+0x1ce Jun 25 08:12:57 masada kernel: #2 0xffffffff816b5647 at zfs_unregister_callbacks+0x1b7 Jun 25 08:12:57 masada kernel: #3 0xffffffff816b58c5 at zfsvfs_teardown+0x175 Jun 25 08:12:57 masada kernel: #4 0xffffffff816b5a2b at zfs_suspend_fs+0x1b Jun 25 08:12:57 masada kernel: #5 0xffffffff816aa229 at zfs_ioc_rollback+0xf9 Jun 25 08:12:57 masada kernel: #6 0xffffffff816abd46 at zfsdev_ioctl+0xe6 Jun 25 08:12:57 masada kernel: #7 0xffffffff807e077b at devfs_ioctl_f+0x7b Jun 25 08:12:57 masada kernel: #8 0xffffffff80938715 at kern_ioctl+0x115 Jun 25 08:12:57 masada kernel: #9 0xffffffff8093894d at sys_ioctl+0xfd Jun 25 08:12:57 masada kernel: #10 0xffffffff80be71d6 at amd64_syscall+0x546 Jun 28 04:08:02 masada kernel: panic: solaris assert: dsl_prop_unregister(ds, "snapdir", snapdir_changed_cb, zfsvfs) == 0, file: /usr/src/sys/modules/zfs/../.. /cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c, line: 1217 Jun 28 04:08:02 masada kernel: cpuid = 1 Jun 28 04:08:02 masada kernel: KDB: stack backtrace: Jun 28 04:08:02 masada kernel: #0 0xffffffff80926c96 at kdb_backtrace+0x66 Jun 28 04:08:02 masada kernel: #1 0xffffffff808f0cae at panic+0x1ce Jun 28 04:08:02 masada kernel: #2 0xffffffff816b570d at zfs_unregister_callbacks+0x27d Jun 28 04:08:02 masada kernel: #3 0xffffffff816b58c5 at zfsvfs_teardown+0x175 Jun 28 04:08:02 masada kernel: #4 0xffffffff816b5a2b at zfs_suspend_fs+0x1b Jun 28 04:08:02 masada kernel: #5 0xffffffff816aa229 at zfs_ioc_rollback+0xf9 Jun 28 04:08:02 masada kernel: #6 0xffffffff816abd46 at zfsdev_ioctl+0xe6 Jun 28 04:08:02 masada kernel: #7 0xffffffff807e077b at devfs_ioctl_f+0x7b Jun 28 04:08:02 masada kernel: #8 0xffffffff80938715 at kern_ioctl+0x115 Jun 28 04:08:02 masada kernel: #9 0xffffffff8093894d at sys_ioctl+0xfd Jun 28 04:08:02 masada kernel: #10 0xffffffff80be71d6 at amd64_syscall+0x546 >How-To-Repeat: I think it's more likely to happen under heavy IO load. This machine has rsnaphot backups scheduled to run every 4 hours, both of the crashes have occurred shortly after a backup began. It was also building packages with poudriere at the same time in both instances. >Fix: >Release-Note: >Audit-Trail: >Unformatted: