From owner-freebsd-fs@freebsd.org Fri Apr 8 18:05:05 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70BA0B07CE1 for ; Fri, 8 Apr 2016 18:05:05 +0000 (UTC) (envelope-from demis@yandex.ru) Received: from forward11o.cmail.yandex.net (forward11o.cmail.yandex.net [IPv6:2a02:6b8:0:1a72::1e1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B3BF1045; Fri, 8 Apr 2016 18:05:04 +0000 (UTC) (envelope-from demis@yandex.ru) Received: from web20o.yandex.ru (web20o.yandex.ru [IPv6:2a02:6b8:0:1a2d::5:120]) by forward11o.cmail.yandex.net (Yandex) with ESMTP id ADB2621691; Fri, 8 Apr 2016 21:05:00 +0300 (MSK) Received: from web20o.yandex.ru (localhost [127.0.0.1]) by web20o.yandex.ru (Yandex) with ESMTP id ECB1FEE0261; Fri, 8 Apr 2016 21:04:59 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1460138700; bh=iO5JLub+/VNob9r72WEtsuFnexj5ro8Ergl27K6c9tA=; h=From:To:In-Reply-To:References:Subject:Date; b=MSNqIpMJ8j1gNWDROHruHIST4cQoZjf94juEdtX07MFLftTE8qX2ZflcuJdnioF1H c62E/gaBe8qcCQwAjbDpu2G7mqxbL/QoldkzwFvoVko35u/MHiqOpaTZGfpZD1A3FV udxdk1VeTlGNPtzjj4qne3vkO755Shf1YhChE1zs= Received: by web20o.yandex.ru with HTTP; Fri, 08 Apr 2016 21:04:59 +0300 From: DemIS To: Andriy Gapon , Steven Hartland , "freebsd-fs@freebsd.org" In-Reply-To: <5707B4A7.4070203@FreeBSD.org> References: <1061671455578760@web3g.yandex.ru> <56C2655F.9010809@multiplay.co.uk> <1076701455583595@web2g.yandex.ru> <56C2782E.2010404@multiplay.co.uk> <2311371455610101@web11j.yandex.ru> <56C2E7A6.9090004@multiplay.co.uk> <4332271457902960@web8o.yandex.ru> <5707B4A7.4070203@FreeBSD.org> Subject: Re: Kernel panic zio.c, line: 270 FreeBSD 10.2 (or 10.3) MIME-Version: 1.0 Message-Id: <4966921460138699@web20o.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Fri, 08 Apr 2016 21:04:59 +0300 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=koi8-r X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2016 18:05:05 -0000 Andriy, The data you requested: (kgdb) bt full ... ... (kgdb) set print pretty (kgdb) fr 11 #11 0xffffffff8245a777 in dmu_spill_hold_by_dnode (dn=0xfffff805c8be5000, flags=6, tag=0x0, dbp=0xfffff802f48dfa28) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:333 333 err = dbuf_read(db, NULL, flags); (kgdb) p/x *dn $1 = { dn_struct_rwlock = { lock_object = { lo_name = 0xffffffff825bd369, lo_flags = 0x2730000, lo_data = 0x0, lo_witness = 0x0 }, sx_lock = 0x11 }, dn_link = { list_next = 0xfffff805c8be0600, list_prev = 0xfffff805c8be4be0 }, dn_objset = 0xfffff80086ebd400, dn_object = 0x798100, dn_dbuf = 0xfffff805b2fdc2a0, dn_handle = 0xfffff805b2feb008, dn_phys = 0xfffffe001589e000, dn_type = 0x14, dn_bonuslen = 0x130, dn_bonustype = 0x2c, dn_nblkptr = 0x1, dn_checksum = 0x0, dn_compress = 0x0, dn_nlevels = 0x1, dn_indblkshift = 0xe, dn_datablkshift = 0x9, dn_moved = 0x0, dn_datablkszsec = 0x1, dn_datablksz = 0x200, dn_maxblkid = 0x0, dn_next_type = {0x0, 0x0, 0x0, 0x0}, dn_next_nblkptr = {0x0, 0x0, 0x0, 0x0}, dn_next_nlevels = {0x0, 0x0, 0x0, 0x0}, dn_next_indblkshift = {0x0, 0x0, 0x0, 0x0}, dn_next_bonustype = {0x0, 0x0, 0x0, 0x0}, dn_rm_spillblk = {0x0, 0x0, 0x0, 0x0}, dn_next_bonuslen = {0x0, 0x0, 0x0, 0x0}, dn_next_blksz = {0x0, 0x0, 0x0, 0x0}, dn_dbufs_count = 0x3, dn_unlisted_l0_blkid = 0xffffffffffffffff, dn_dirty_link = {{ list_next = 0x0, list_prev = 0x0 }, { list_next = 0x0, list_prev = 0x0 }, { list_next = 0xfffff80086ebd6c0, list_prev = 0xfffff80548419c98 }, { list_next = 0x0, list_prev = 0x0 }}, ---Type to continue, or q to quit--- dn_mtx = { lock_object = { lo_name = 0xffffffff825bd37f, lo_flags = 0x2730000, lo_data = 0x0, lo_witness = 0x0 }, sx_lock = 0x1 }, dn_dirty_records = {{ list_size = 0xd8, list_offset = 0x0, list_head = { list_next = 0xfffff805c8be5128, list_prev = 0xfffff805c8be5128 } }, { list_size = 0xd8, list_offset = 0x0, list_head = { list_next = 0xfffff805c8be5148, list_prev = 0xfffff805c8be5148 } }, { list_size = 0xd8, list_offset = 0x0, list_head = { list_next = 0xfffff803c1d2a000, list_prev = 0xfffff803c1d2a000 } }, { list_size = 0xd8, list_offset = 0x0, list_head = { list_next = 0xfffff805c8be5188, list_prev = 0xfffff805c8be5188 } }}, dn_free_ranges = {0x0, 0x0, 0x0, 0x0}, dn_allocated_txg = 0x0, dn_free_txg = 0x0, dn_assigned_txg = 0x123a3ba, dn_notxholds = { cv_description = 0xffffffff825bd39d, cv_waiters = 0x0 }, dn_dirtyctx = 0x1, dn_dirtyctx_firstset = 0xfffff804e0fefcf0, dn_tx_holds = { rc_count = 0x2 }, dn_holds = { rc_count = 0x6 }, dn_dbufs_mtx = { ---Type to continue, or q to quit--- lock_object = { lo_name = 0xffffffff825bd38b, lo_flags = 0x2730000, lo_data = 0x0, lo_witness = 0x0 }, sx_lock = 0x1 }, dn_dbufs = { avl_root = 0xfffff803b7a7a5f0, avl_compar = 0xffffffff8247b100, avl_offset = 0xb0, avl_numnodes = 0x2, avl_size = 0xe0 }, dn_bonus = 0xfffff80052609620, dn_have_spill = 0x1, dn_zio = 0xfffff801db5bcac8, dn_oldused = 0x0, dn_oldflags = 0x0, dn_olduid = 0x271a, dn_oldgid = 0x2711, dn_newuid = 0x271a, dn_newgid = 0x2711, dn_id_flags = 0x5, dn_zfetch = { zf_rwlock = { lock_object = { lo_name = 0xffffffff825c11ef, lo_flags = 0x2730000, lo_data = 0x0, lo_witness = 0x0 }, sx_lock = 0x1 }, zf_stream = { list_size = 0x70, list_offset = 0x58, list_head = { list_next = 0xfffff805c8be52c8, list_prev = 0xfffff805c8be52c8 } }, zf_dnode = 0xfffff805c8be5000, zf_stream_cnt = 0x0, zf_alloc_fail = 0x0 } } (kgdb) fr 10 #10 0xffffffff8244ef8b in dbuf_read (db=0xfffff803b1e8de00, zio=0xfffff80071def398, flags=6) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:621 621 dbuf_read_impl(db, zio, &flags); (kgdb) p/x *db $2 = { db = { db_object = 0x798100, db_offset = 0x0, db_size = 0x0, db_data = 0x0 }, db_objset = 0xfffff80086ebd400, db_dnode_handle = 0xfffff805b2feb008, db_parent = 0xfffff805b2fdc2a0, db_hash_next = 0x0, db_blkid = 0xfffffffffffffffe, db_blkptr = 0xfffffe001589e180, db_level = 0x0, db_mtx = { lock_object = { lo_name = 0xffffffff825bb470, lo_flags = 0x2730000, lo_data = 0x0, lo_witness = 0x0 }, sx_lock = 0x1 }, db_state = 0x3, db_holds = { rc_count = 0x2 }, db_buf = 0x0, db_changed = { cv_description = 0xffffffff825bb47c, cv_waiters = 0x0 }, db_data_pending = 0x0, db_last_dirty = 0x0, db_link = { avl_child = {0x0, 0x0}, avl_pcb = 0xfffff803b7a7a5f5 }, db_user_ptr = 0x0, db_evict_func = 0x0, db_immediate_evict = 0x0, db_freed_in_flight = 0x0, db_dirtycnt = 0x0 } (kgdb) 08.04.2016, 16:40, "Andriy Gapon" : > On 13/03/2016 23:02, DemIS wrote: >> š#10 0xffffffff8244ef8b in dbuf_read (db=0xfffff803b1e8de00, zio=0xfffff80071def398, flags=6) >> šššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:621 > > ... >> š#11 0xffffffff8245a777 in dmu_spill_hold_by_dnode (dn=0xfffff805c8be5000, flags=6, tag=0x0, dbp=0xfffff802f48dfa28) >> šššššat /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:333 >> šššššššššdb = (dmu_buf_impl_t *) 0xfffff803b1e8de00 >> šššššššššerr = -512 > > If you still have this crash dump, then can you please do the following in kgdb? > > set print pretty > fr 11 > p/x *dn > fr 10 > p/x *db > > I suspect that the following change might be able to prevent future corruption > of your files, but it won't help with already corrupted files: > https://github.com/openzfs/openzfs/pull/81/commits/ee09d80ceafa7ff7930d1e45dddc21ca3d7a87cb > > The following completely untested patch might help to avoid panics with the > corrupted files, but no promises: > > diff --git a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c > b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c > index 4396c01334015..94b353d62e3e6 100644 > --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c > +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c > @@ -329,7 +329,10 @@ dmu_spill_hold_by_dnode > šššššššššššššššššrw_exit(&dn->dn_struct_rwlock); > > šššššššššASSERT(db != NULL); > - err = dbuf_read(db, NULL, flags); > + if (db->db.db_size > 0) > + err = dbuf_read(db, NULL, flags); > + else > + err = ENOENT; > šššššššššif (err == 0) > ššššššššššššššššš*dbp = &db->db; > šššššššššelse > > -- > Andriy Gapon Demis.