From owner-freebsd-fs@freebsd.org Sun Dec 1 07:19:03 2019 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C37961C1CA5 for ; Sun, 1 Dec 2019 07:19:03 +0000 (UTC) (envelope-from isafonov@me.com) Received: from mr85p00im-hyfv06021301.me.com (mr85p00im-hyfv06021301.me.com [17.58.23.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47QfkF69fMz4XLm for ; Sun, 1 Dec 2019 07:19:01 +0000 (UTC) (envelope-from isafonov@me.com) Received: from [192.168.37.155] (ip-097-206-122-091.static.atnet.ru [91.122.206.97]) by mr85p00im-hyfv06021301.me.com (Postfix) with ESMTPSA id 2086540D5A for ; Sun, 1 Dec 2019 07:18:56 +0000 (UTC) From: Ivan Safonov Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: zfs panic: solaris assert Message-Id: <78272047-A0E6-4EE0-B3B3-1F927D29EC12@me.com> References: To: freebsd-fs@freebsd.org Date: Sun, 1 Dec 2019 10:18:53 +0300 X-Mailer: Apple Mail (2.3445.104.11) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-12-01_01:, , signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1908290000 definitions=main-1912010064 X-Rspamd-Queue-Id: 47QfkF69fMz4XLm X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:17.58.0.0/16:c]; FREEMAIL_FROM(0.00)[me.com]; TO_DN_NONE(0.00)[]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[me.com:+]; DMARC_POLICY_ALLOW(-0.50)[me.com,quarantine]; RCVD_IN_DNSWL_LOW(-0.10)[188.23.58.17.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[me.com]; ASN(0.00)[asn:714, ipnet:17.58.16.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[me.com:s=1a1hai]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[me.com.dwl.dnswl.org : 127.0.5.1]; IP_SCORE(0.00)[ip: (-3.62), ipnet: 17.58.16.0/20(-1.57), asn: 714(-2.31), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Dec 2019 07:19:03 -0000 Hi there I have a small home server with a ZFS pool created in FreeBSD 11.1 and = updated to 12.0 -> 12.1. A few days ago, the server rebooted with panic panic: solaris assert: size <=3D (1ULL << 24) (0x1401000 <=3D = 0x1000000), file: = /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c, line: 296 cpuid =3D 3 time =3D 1574981747 KDB: stack backtrace: #0 0xffffffff80c1d297 at kdb_backtrace+0x67 #1 0xffffffff80bd05cd at vpanic+0x19d #2 0xffffffff80bd0423 at panic+0x43 #3 0xffffffff82a6e22c at assfail3+0x2c #4 0xffffffff8284a8f7 at abd_alloc+0x67 #5 0xffffffff82850319 at arc_hdr_alloc_pabd+0x99 #6 0xffffffff8284d554 at arc_hdr_alloc+0x124 #7 0xffffffff8284ef13 at arc_read+0x243 #8 0xffffffff8287942d at traverse_prefetch_metadata+0xbd #9 0xffffffff828788cc at traverse_visitbp+0x3dc #10 0xffffffff82878930 at traverse_visitbp+0x440 #11 0xffffffff82878930 at traverse_visitbp+0x440 #12 0xffffffff82878930 at traverse_visitbp+0x440 #13 0xffffffff82878930 at traverse_visitbp+0x440 #14 0xffffffff82879513 at traverse_dnode+0xd3 #15 0xffffffff82878c30 at traverse_visitbp+0x740 #16 0xffffffff828780a7 at traverse_impl+0x317 #17 0xffffffff8287837c at traverse_pool+0x14c Uptime: 1m0s I tested the memory and drives and they look OK. Then I installed = FreeBSD 12.1 on a USB drive zpool import pool: zroot id: 5722521002676846505 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, = though some features will not be available without an explicit 'zpool = upgrade'. config: zroot ONLINE raidz1-0 ONLINE gpt/D0-33GTM0VGS ONLINE gpt/D1-Z1E3Q7KP ONLINE gpt/D2-Z4ZARG1V ONLINE I tried to import the pool but got the same panic zpool import -o readonly -f -F -N -R /pool zroot zdb -ue zroot Uberblock: magic =3D 0000000000bab10c version =3D 5000 txg =3D 11751485 guid_sum =3D 14638353410936556308 timestamp =3D 1574776136 UTC =3D Tue Nov 26 16:48:56 2019 checkpoint_txg =3D 0 zdb -l /dev/gpt/D0-33GTM0VGS Code: ------------------------------------ LABEL 0 ------------------------------------ version: 5000 name: 'zroot' state: 0 txg: 11750848 pool_guid: 5722521002676846505 hostid: 1550328424 hostname: '' top_guid: 10071190156355008053 guid: 4016597551985842896 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 10071190156355008053 nparity: 1 metaslab_array: 39 metaslab_shift: 35 ashift: 12 asize: 5997325713408 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 4016597551985842896 path: '/dev/gpt/D0-33GTM0VGS' whole_disk: 1 DTL: 70406 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 18015782253695591313 path: '/dev/gpt/D1-Z1E3Q7KP' whole_disk: 1 DTL: 70405 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 13705750593642370773 path: '/dev/gpt/D2-Z4ZARG1V' whole_disk: 1 DTL: 70323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data ------------------------------------ LABEL 1 ------------------------------------ version: 5000 name: 'zroot' state: 0 txg: 11750848 pool_guid: 5722521002676846505 hostid: 1550328424 hostname: '' top_guid: 10071190156355008053 guid: 4016597551985842896 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 10071190156355008053 nparity: 1 metaslab_array: 39 metaslab_shift: 35 ashift: 12 asize: 5997325713408 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 4016597551985842896 path: '/dev/gpt/D0-33GTM0VGS' whole_disk: 1 DTL: 70406 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 18015782253695591313 path: '/dev/gpt/D1-Z1E3Q7KP' whole_disk: 1 DTL: 70405 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 13705750593642370773 path: '/dev/gpt/D2-Z4ZARG1V' whole_disk: 1 DTL: 70323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data ------------------------------------ LABEL 2 ------------------------------------ version: 5000 name: 'zroot' state: 0 txg: 11750848 pool_guid: 5722521002676846505 hostid: 1550328424 hostname: '' top_guid: 10071190156355008053 guid: 4016597551985842896 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 10071190156355008053 nparity: 1 metaslab_array: 39 metaslab_shift: 35 ashift: 12 asize: 5997325713408 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 4016597551985842896 path: '/dev/gpt/D0-33GTM0VGS' whole_disk: 1 DTL: 70406 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 18015782253695591313 path: '/dev/gpt/D1-Z1E3Q7KP' whole_disk: 1 DTL: 70405 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 13705750593642370773 path: '/dev/gpt/D2-Z4ZARG1V' whole_disk: 1 DTL: 70323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data ------------------------------------ LABEL 3 ------------------------------------ version: 5000 name: 'zroot' state: 0 txg: 11750848 pool_guid: 5722521002676846505 hostid: 1550328424 hostname: '' top_guid: 10071190156355008053 guid: 4016597551985842896 vdev_children: 1 vdev_tree: type: 'raidz' id: 0 guid: 10071190156355008053 nparity: 1 metaslab_array: 39 metaslab_shift: 35 ashift: 12 asize: 5997325713408 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 4016597551985842896 path: '/dev/gpt/D0-33GTM0VGS' whole_disk: 1 DTL: 70406 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 18015782253695591313 path: '/dev/gpt/D1-Z1E3Q7KP' whole_disk: 1 DTL: 70405 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 13705750593642370773 path: '/dev/gpt/D2-Z4ZARG1V' whole_disk: 1 DTL: 70323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data zdb -AAA -F -X zroot ... capacity operations bandwidth ---- = errors ---- description used avail read write read write read = write cksum zroot 1.47T 3.97T 82 0 351K 0 0 = 0 30 raidz1 1.47T 3.97T 82 0 351K 0 0 = 0 121 /dev/gpt/D0-33GTM0VGS 27 0 146K 0 0 = 0 0 /dev/gpt/D1-Z1E3Q7KP 27 0 146K 0 0 = 0 0 /dev/gpt/D2-Z4ZARG1V 27 0 146K 0 0 = 0 0 ... Is it possible to somehow restore the pool or data?=