From owner-freebsd-stable@freebsd.org Sat Sep 30 16:30:34 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 933BDE2A4DD for ; Sat, 30 Sep 2017 16:30:34 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 31E45741E3 for ; Sat, 30 Sep 2017 16:30:33 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v8UGUVpo053249 for ; Sat, 30 Sep 2017 18:30:31 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 680CBBFF; Sat, 30 Sep 2017 18:30:31 +0200 (CEST) Message-ID: <59CFC6A6.6030600@omnilan.de> Date: Sat, 30 Sep 2017 18:30:30 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: panic: Solaris(panic): blkptr invalid CHECKSUM1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 30 Sep 2017 18:30:31 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 16:30:34 -0000 Bad surprise. Most likely I forgot to stop a PCIe-Passthrough NIC before shutting down that (byhve(8)) guest – jhb@ helped my identifying this as the root cause for sever memory corruptions I regularly had (on stable-11). Now this time, corruption affected ZFS's RAM area, obviously. What I haven't expected is the panic. The machine has memory disk as root, so luckily I still can boot (from ZFS, –> mdpreload rootfs) into single user mode, but early rc stage (most likely mounting ZFS datasets) leads to the following panic: Trying to mount root from ufs:/dev/ufs/cetusROOT []... panic: Solaris(panic): blkptr at 0xfffffe0005b6b000 has invalid CHECKSUM 1 cpuid = 1 KDB: stack backtrace: #0 0xffffffff805e3837 at kdb_backtrace+0x67 #1 0xffffffff805a2286 at vpanic+0x186 #2 0xffffffff805a20f3 at panic+0x43 #3 0xffffffff81570192 at vcmn_err+0xc2 #4 0xffffffff812d7dda at zfs_panic_recover+0x5a #5 0xffffffff812ff49b at zfs_blkptr_verify+0x8b #6 0xffffffff812ff72c at zio_read+0x2c #7 0xffffffff812761de at arc_read+0x6de #8 0xffffffff81298b4d at traverse_prefetch_metadata+0xbd #9 0xffffffff812980ed at traverse_visitbp+0x39d #10 0xffffffff81298c27 at traverse_dnode+0xc7 #11 0xffffffff812984a3 at traverse_visitbp+0x753 #12 0xffffffff8129788b at traverse_impl+0x22b #13 0xffffffff81297afc at traverse_pool+0x5c #14 0xffffffff812cce06 at spa_load+0x1c06 #15 0xffffffff812cc302 at spa_load+0x1102 #16 0xffffffff812cac6e at spa_load_best+0x6e #17 0xffffffff812c73a1 at spa_open_common+0x101 Uptime: 37s Dumping 1082 out of 15733 MB:..2%..… Dump complete mps0: Sending StopUnit: path (xpt0:mps0:0:2:ffffffff): handle 12 mps0: Incrementing SSU count … Haven't done any scrub attempts yet – expectation is to get all datasets of the striped mirror pool back... Any hints highly appreciated. -harry