From owner-freebsd-stable@freebsd.org Mon Jun 11 11:29:02 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 588DA1015B0F for ; Mon, 11 Jun 2018 11:29:02 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CDB747F156 for ; Mon, 11 Jun 2018 11:29:01 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 8ECCA1015B0E; Mon, 11 Jun 2018 11:29:01 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 673E91015B0D for ; Mon, 11 Jun 2018 11:29:01 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EF4B57F152; Mon, 11 Jun 2018 11:29:00 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 933C73DDAC; Mon, 11 Jun 2018 13:28:58 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UzkG4BfA6lLD; Mon, 11 Jun 2018 13:28:58 +0200 (CEST) Received: from [192.168.101.70] (vpn.ecoracks.nl [176.74.240.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 1C7E63DDA9; Mon, 11 Jun 2018 13:28:58 +0200 (CEST) Subject: Re: Continuous crashing ZFS server To: Andriy Gapon , "stable@freebsd.org" References: <17446f39-97a1-8603-11a0-32176e8cb833@FreeBSD.org> <100ea6d0-5cf4-1a00-0e3a-dfad6175df6c@FreeBSD.org> From: Willem Jan Withagen Openpgp: preference=signencrypt Message-ID: <17ee24dd-93e5-dede-d7aa-90239c72c287@digiware.nl> Date: Mon, 11 Jun 2018 13:29:00 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <100ea6d0-5cf4-1a00-0e3a-dfad6175df6c@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Language: nl Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jun 2018 11:29:02 -0000 On 11-6-2018 12:53, Andriy Gapon wrote: > On 11/06/2018 13:26, Willem Jan Withagen wrote: >> On 11/06/2018 12:13, Andriy Gapon wrote: >>> On 08/06/2018 13:02, Willem Jan Withagen wrote: >>>> My file server is crashing about every 15 minutes at the moment. >>>> The panic looks like: >>>> >>>> Jun  8 11:48:43 zfs kernel: panic: Solaris(panic): zfs: allocating >>>> allocated segment(offset=12922221670400 size=24576) >>>> Jun  8 11:48:43 zfs kernel: >>>> Jun  8 11:48:43 zfs kernel: cpuid = 1 >>>> Jun  8 11:48:43 zfs kernel: KDB: stack backtrace: >>>> Jun  8 11:48:43 zfs kernel: #0 0xffffffff80aada57 at kdb_backtrace+0x67 >>>> Jun  8 11:48:43 zfs kernel: #1 0xffffffff80a6bb36 at vpanic+0x186 >>>> Jun  8 11:48:43 zfs kernel: #2 0xffffffff80a6b9a3 at panic+0x43 >>>> Jun  8 11:48:43 zfs kernel: #3 0xffffffff82488192 at vcmn_err+0xc2 >>>> Jun  8 11:48:43 zfs kernel: #4 0xffffffff821f73ba at zfs_panic_recover+0x5a >>>> Jun  8 11:48:43 zfs kernel: #5 0xffffffff821dff8f at range_tree_add+0x20f >>>> Jun  8 11:48:43 zfs kernel: #6 0xffffffff821deb06 at metaslab_free_dva+0x276 >>>> Jun  8 11:48:43 zfs kernel: #7 0xffffffff821debc1 at metaslab_free+0x91 >>>> Jun  8 11:48:43 zfs kernel: #8 0xffffffff8222296a at zio_dva_free+0x1a >>>> Jun  8 11:48:43 zfs kernel: #9 0xffffffff8221f6cc at zio_execute+0xac >>>> Jun  8 11:48:43 zfs kernel: #10 0xffffffff80abe827 at >>>> taskqueue_run_locked+0x127 >>>> Jun  8 11:48:43 zfs kernel: #11 0xffffffff80abf9c8 at >>>> taskqueue_thread_loop+0xc8 >>>> Jun  8 11:48:43 zfs kernel: #12 0xffffffff80a2f7d5 at fork_exit+0x85 >>>> Jun  8 11:48:43 zfs kernel: #13 0xffffffff80ec4abe at fork_trampoline+0xe >>>> Jun  8 11:48:43 zfs kernel: Uptime: 9m7s >>>> >>>> Maybe a known bug? >>>> Is there anything I can do about this? >>>> Any debugging needed? >>> >>> Sorry to inform you but your on-disk data got corrupted. >>> The most straightforward thing you can do is try to save data from the pool in >>> readonly mode. >> >> Hi Andriy, >> >> Auch, that is a first in 12 years of using ZFS. "Fortunately" it was of a test >> ZVOL->iSCSI->Win10 disk on which I spool my CAMs. >> >> Removing the ZVOL actually fixed the rebooting, but now the question is: >>     Is the remainder of the zpools on the same disks in danger? > > You can try to check with zdb -b on an idle (better exported) pool. And zpool > scrub. If scrub says things are oke, I can start breathing again? exporting the pool is something for the small hours. Thanx, --WjW