From owner-freebsd-bugs@FreeBSD.ORG Wed Oct 2 11:50:04 2013 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 04D5C62C for ; Wed, 2 Oct 2013 11:50:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D7595224C for ; Wed, 2 Oct 2013 11:50:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r92Bo3xX053892 for ; Wed, 2 Oct 2013 11:50:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r92Bo3dl053891; Wed, 2 Oct 2013 11:50:03 GMT (envelope-from gnats) Resent-Date: Wed, 2 Oct 2013 11:50:03 GMT Resent-Message-Id: <201310021150.r92Bo3dl053891@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Keith White Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 576AA25F for ; Wed, 2 Oct 2013 11:40:42 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 45759212A for ; Wed, 2 Oct 2013 11:40:42 +0000 (UTC) Received: from oldred.freebsd.org ([127.0.1.6]) by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r92BefF8055294 for ; Wed, 2 Oct 2013 11:40:41 GMT (envelope-from nobody@oldred.freebsd.org) Received: (from nobody@localhost) by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r92BefZb055285; Wed, 2 Oct 2013 11:40:41 GMT (envelope-from nobody) Message-Id: <201310021140.r92BefZb055285@oldred.freebsd.org> Date: Wed, 2 Oct 2013 11:40:41 GMT From: Keith White To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Subject: kern/182570: ZFS panic in receive X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Oct 2013 11:50:04 -0000 >Number: 182570 >Category: kern >Synopsis: ZFS panic in receive >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Oct 02 11:50:03 UTC 2013 >Closed-Date: >Last-Modified: >Originator: Keith White >Release: current >Organization: EECS, University of Ottawa >Environment: FreeBSD freebsd10 10.0-ALPHA4 FreeBSD 10.0-ALPHA4 #0 r255937: Sun Sep 29 09:45:21 EDT 2013 kwhite@freebsd10:/usr/obj/usr/src/sys/GENERIC amd64 >Description: [previously sent to freebsd-current] I get the following reproducible panic when doing a zfs send -R | zfs recv of a well churned file system (snapshots of the ports tree before and after libiconv update) running a recent current: panic: solaris assert: dn->dn_maxblkid == 0 && (BP_IS_HOLE(&dn->dn_phys->dn_blkptr[0]) || dnode_block_freed(dn, 0)), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c, line: 598 coredump available. # zfs list -t snapshot -r tank/ports NAME USED AVAIL REFER MOUNTPOINT tank/ports@20130915 72.5M - 3.64G - tank/ports@20130917 0 - 3.65G - tank/ports@20130918 0 - 3.65G - tank/ports@20130921 88.6M - 3.66G - tank/ports@20130928 352K - 3.66G - # zpool list -v NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT m_tank 1.81T 831G 1.00T 44% 1.00x ONLINE - ada4.nop.eli 1.81T 831G 1.00T - tank 928G 831G 96.9G 89% 1.00x ONLINE - ada3.nop.eli 928G 831G 96.9G - # zfs send -vR tank/ports@20130928 | zfs recv -vF m_tank/xports .. panic eventually ... # cd /boot/kernel; kgdb kernel /var/crash/vmcore.last .. (kgdb) up #5 0xffffffff81d09735 in dnode_reallocate (dn=0xfffff8006dde3000, ot=DMU_OT_PLAIN_FILE_CONTENTS, blocksize=1024, bonustype=DMU_OT_SA, bonuslen=168, tx=0xfffff8006d7a2600) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c:596 596 ASSERT(dn->dn_maxblkid == 0 && (kgdb) p dn->dn_maxblkid $1 = 2 So, no question about why the ASSERT failed. Sorry, debugging this is *way* beyond me. Any hints, patches to try? >How-To-Repeat: on a zpool called "tank" with non-existent tank/nobj and tank/xobj: #!/bin/sh zfs create tank/nobj zfs snapshot tank/nobj@0 __MAKECONF=/dev/null SRCCONF=/dev/null MAKEOBJDIRPREFIX=/tank/nobj make -j6 buildkernel zfs snapshot tank/nobj@1 find /tank/nobj -name '.h' -print | xargs rm zfs snapshot tank/nobj@2 rm -rf /tank/nobj/* zfs snapshot tank/nobj@3 __MAKECONF=/dev/null SRCCONF=/dev/null MAKEOBJDIRPREFIX=/tank/nobj make -j6 buildkernel zfs snapshot tank/nobj@4 rm -rf /tank/nobj/* zfs snapshot tank/nobj@5 zfs send -vR tank/nobj@5 | zfs recv -vF tank/xobj >Fix: This will just avoid the panic, the received filesystem is not complete. The "zfs recv" now terminates with "internal error: File too large": Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c (revision 255986) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c (working copy) @@ -592,10 +592,16 @@ rw_enter(&dn->dn_struct_rwlock, RW_WRITER); dnode_setdirty(dn, tx); if (dn->dn_datablksz != blocksize) { +#if 1 + if (dn->dn_maxblkid != 0) { + printf("ZFS should panic here since dn->dn_maxblkid is %lu instead of 0\n...but we'll continue\n", dn->dn_maxblkid); + } +#else /* change blocksize */ ASSERT(dn->dn_maxblkid == 0 && (BP_IS_HOLE(&dn->dn_phys->dn_blkptr[0]) || dnode_block_freed(dn, 0))); +#endif dnode_setdblksz(dn, blocksize); dn->dn_next_blksz[tx->tx_txg&TXG_MASK] = blocksize; } >Release-Note: >Audit-Trail: >Unformatted: