From owner-freebsd-amd64@freebsd.org Sun Jan 29 22:57:42 2017 Return-Path: Delivered-To: freebsd-amd64@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66AEFCC7AB2 for ; Sun, 29 Jan 2017 22:57:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5679DB7D for ; Sun, 29 Jan 2017 22:57:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0TMvg3e069067 for ; Sun, 29 Jan 2017 22:57:42 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-amd64@FreeBSD.org Subject: [Bug 216586] zfs panic: sa.sa_magic == 0x2F505A in zfs_space_delta_cb() Date: Sun, 29 Jan 2017 22:57:42 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: johannes@jo-t.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter cc Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2017 22:57:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216586 Bug ID: 216586 Summary: zfs panic: sa.sa_magic =3D=3D 0x2F505A in zfs_space_delta_cb() Product: Base System Version: 10.3-STABLE Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: johannes@jo-t.de CC: freebsd-amd64@FreeBSD.org CC: freebsd-amd64@FreeBSD.org While rsync'ing some files, the system eventually panics with: > panic: solaris assert: sa.sa_magic =3D=3D 0x2F505A (0x51786a88 =3D=3D 0x2= f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/c= ommon/fs/zfs/zfs_vfsops.c, line: 610 The backtrace is: > #0 0xffffffff804f23c0 at kdb_backtrace+0x60 > #1 0xffffffff804b3824 at vpanic+0x124 > #2 0xffffffff804b36f3 at panic+0x43 > #3 0xffffffff8109122f at assfail3+0x2f > #4 0xffffffff80e4d417 at zfs_space_delta_cb+0x107 > #5 0xffffffff80dc156f at dmu_objset_userquota_get_ids+0x36f > #6 0xffffffff80dc952f at dnode_sync+0x9f > #7 0xffffffff80dc0aeb at dmu_objset_sync_dnodes+0x2b > #8 0xffffffff80dc0920 at dmu_objset_sync+0x220 > #9 0xffffffff80dd241a at dsl_dataset_sync+0x1ca > #10 0xffffffff80ddabca at dsl_pool_sync+0x7a > #11 0xffffffff80dfa77d at spa_sync+0x57d > #12 0xffffffff80e03f56 at txg_sync_thread+0x3a6 > #13 0xffffffff8048037a at fork_exit+0x9a > #14 0xffffffff806ccfde at fork_trampoline+0xe This happens on: > FreeBSD fred 10.3-STABLE FreeBSD 10.3-STABLE #1 r308740: Sat Nov 19 21:15= :27 GMT 2016 root@fred:/usr/obj/usr/src/sys/FRED amd64 and as well on a newer version: > FreeBSD fred 10.3-STABLE FreeBSD 10.3-STABLE #0 r312924: Sun Jan 29 01:45= :50 GMT 2017 root@fred:/usr/obj/usr/src/sys/FRED amd64 The machine has ECC RAM, and a zpool scrub did not report any errors. The p= ool is a 2-way-mirror. The actual value of sa.sa_magic changes (not everytime though): e.g. 0x5178= 6a88 or 0x5178691e. The code in question at zfs_vfsops.c, line 610 is: > static int > zfs_space_delta_cb(dmu_object_type_t bonustype, void *data, > uint64_t *userp, uint64_t *groupp) > { > /* > * Is it a valid type of object to track? > */ > if (bonustype !=3D DMU_OT_ZNODE && bonustype !=3D DMU_OT_SA) > return (SET_ERROR(ENOENT)); >=20 > /* > * If we have a NULL data pointer > * then assume the id's aren't changing and > * return EEXIST to the dmu to let it know to > * use the same ids > */ > if (data =3D=3D NULL) > return (SET_ERROR(EEXIST)); >=20 > if (bonustype =3D=3D DMU_OT_ZNODE) { > znode_phys_t *znp =3D data; > *userp =3D znp->zp_uid; > *groupp =3D znp->zp_gid; > } else { > int hdrsize; > sa_hdr_phys_t *sap =3D data; > sa_hdr_phys_t sa =3D *sap; > boolean_t swap =3D B_FALSE; >=20 > ASSERT(bonustype =3D=3D DMU_OT_SA); >=20 > if (sa.sa_magic =3D=3D 0) { > /* > * This should only happen for newly created > * files that haven't had the znode data filled > * in yet. > */ > *userp =3D 0; > *groupp =3D 0; > return (0); > } > if (sa.sa_magic =3D=3D BSWAP_32(SA_MAGIC)) { > sa.sa_magic =3D SA_MAGIC; > sa.sa_layout_info =3D BSWAP_16(sa.sa_layout_info); > swap =3D B_TRUE; > } else { > /*610*/ VERIFY3U(sa.sa_magic, =3D=3D, SA_MAGIC); // FAILS > } >=20 > hdrsize =3D sa_hdrsize(&sa); > VERIFY3U(hdrsize, >=3D, sizeof (sa_hdr_phys_t)); > *userp =3D *((uint64_t *)((uintptr_t)data + hdrsize + > SA_UID_OFFSET)); > *groupp =3D *((uint64_t *)((uintptr_t)data + hdrsize + > SA_GID_OFFSET)); > if (swap) { > *userp =3D BSWAP_64(*userp); > *groupp =3D BSWAP_64(*groupp); > } > } > return (0); > } The stacktrace has dmu_objset_userquota_get_ids listed. And the target file system (into which rsync is writing) has quota enabled: > NAME PROPERTY VALUE = SOURCE > freddata/backup/alexs-imac type filesystem - > freddata/backup/alexs-imac creation Wed Mar 7 18:01 2012 - > freddata/backup/alexs-imac used 521G - > freddata/backup/alexs-imac available 29.2G - > freddata/backup/alexs-imac referenced 213G - > freddata/backup/alexs-imac compressratio 1.11x - > freddata/backup/alexs-imac mounted yes - > freddata/backup/alexs-imac quota 550G = local > freddata/backup/alexs-imac reservation 300G = local > freddata/backup/alexs-imac recordsize 128K = default > freddata/backup/alexs-imac mountpoint /backup/alexs-imac = inherited from freddata/backup > freddata/backup/alexs-imac sharenfs off = default > freddata/backup/alexs-imac checksum fletcher4 = inherited from freddata > freddata/backup/alexs-imac compression lz4 = local > freddata/backup/alexs-imac atime off = inherited from freddata/backup > freddata/backup/alexs-imac devices on = default > freddata/backup/alexs-imac exec on = default > freddata/backup/alexs-imac setuid on = default > freddata/backup/alexs-imac readonly off = default > freddata/backup/alexs-imac jailed off = default > freddata/backup/alexs-imac snapdir hidden = default > freddata/backup/alexs-imac aclmode discard = default > freddata/backup/alexs-imac aclinherit restricted = default > freddata/backup/alexs-imac canmount on = default > freddata/backup/alexs-imac xattr off = temporary > freddata/backup/alexs-imac copies 1 = default > freddata/backup/alexs-imac version 5 - > freddata/backup/alexs-imac utf8only off - > freddata/backup/alexs-imac normalization none - > freddata/backup/alexs-imac casesensitivity sensitive - > freddata/backup/alexs-imac vscan off = default > freddata/backup/alexs-imac nbmand off = default > freddata/backup/alexs-imac sharesmb off = default > freddata/backup/alexs-imac refquota none = default > freddata/backup/alexs-imac refreservation none = default > freddata/backup/alexs-imac primarycache all = default > freddata/backup/alexs-imac secondarycache all = default > freddata/backup/alexs-imac usedbysnapshots 308G - > freddata/backup/alexs-imac usedbydataset 213G - > freddata/backup/alexs-imac usedbychildren 0 - > freddata/backup/alexs-imac usedbyrefreservation 0 - > freddata/backup/alexs-imac logbias latency = default > freddata/backup/alexs-imac dedup off = default > freddata/backup/alexs-imac mlslabel - > freddata/backup/alexs-imac sync standard = default > freddata/backup/alexs-imac refcompressratio 1.06x - > freddata/backup/alexs-imac written 145G - > freddata/backup/alexs-imac logicalused 569G - > freddata/backup/alexs-imac logicalreferenced 221G - > freddata/backup/alexs-imac volmode default = default > freddata/backup/alexs-imac filesystem_limit none = default > freddata/backup/alexs-imac snapshot_limit none = default > freddata/backup/alexs-imac filesystem_count none = default > freddata/backup/alexs-imac snapshot_count none = default > freddata/backup/alexs-imac redundant_metadata all = default --=20 You are receiving this mail because: You are on the CC list for the bug.=