From owner-freebsd-current@freebsd.org Tue Mar 20 08:02:05 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9FAFF5B947 for ; Tue, 20 Mar 2018 08:02:05 +0000 (UTC) (envelope-from fbsd-lists@dudes.ch) Received: from mail.dudes.ch (mail.dudes.ch [193.73.211.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.dudes.ch", Issuer "StartCom Class 3 OV Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D7E885C4D for ; Tue, 20 Mar 2018 08:02:04 +0000 (UTC) (envelope-from fbsd-lists@dudes.ch) Received: from mwoffice.virtualtec.office (pippin.virtualtec.ch [93.189.66.120]) (authenticated bits=0) by mail.dudes.ch (8.15.2/8.15.2) with ESMTPSA id w2K8oP1H039330 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 20 Mar 2018 09:50:25 +0100 (CET) (envelope-from fbsd-lists@dudes.ch) X-Authentication-Warning: mail.dudes.ch: Host pippin.virtualtec.ch [93.189.66.120] claimed to be mwoffice.virtualtec.office Date: Tue, 20 Mar 2018 08:50:28 +0100 From: Markus Wild To: freebsd-current@freebsd.org Subject: Re: ZFS i/o error in recent 12.0 Message-ID: <20180320085028.0b15ff40@mwoffice.virtualtec.office> In-Reply-To: <201803192300.w2JN04fx007127@kx.openedu.org> References: <201803192300.w2JN04fx007127@kx.openedu.org> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; amd64-portbld-freebsd11.1) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 193.73.211.25 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Mar 2018 08:02:05 -0000 Hi there, > I've been encountered suddenly death in ZFS full volume > machine(r330434) about 10 days after installation[1]: > > ZFS: i/o error - all block copies unavailable > ZFS: can't read MOS of pool zroot > gptzfsboot: failed to mount default pool zroot > > 268847104 30978715648 4 freebsd-zfs (14T) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I had faced the exact same issue on a HP Microserver G8 with 8TB disks and a 16TB zpool on FreeBSD 11 about a year ago. My conclusion was, that over time (and updating the kernel), the blocks for that kernel file were reallocated to a later spot on the disks, and that however the loader fetches those blocks, it now failed doing so (perhaps a 2/4TB limit/bug with the BIOS of that server? Unfortunately, there was no UEFI support for it, don't know whether that changed in the meantime). The pool was always importable fine with the USB stick, the problem was only with the boot loader. I worked around the problem stealing space from the swap partitions on two disks to build a "zboot" pool, just containing the /boot directory, having the boot loader load the kernel from there, and then still mount the real root pool to run the system off using loader-variables in loader.conf of the boot pool. It's a hack, but it's working fine since (the server is being used as a backup repository). This is what I have in the "zboot" boot/loader.conf: # zfs boot kludge due to buggy bios vfs.root.mountfrom="zfs:zroot/ROOT/fbsd11" If you're facing the same problem, you might give this a shot? You seem to have plenty of swap to canibalize as well;) Cheers, Markus