From owner-freebsd-current@freebsd.org Wed Mar 21 07:20:44 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3128F4EF93 for ; Wed, 21 Mar 2018 07:20:44 +0000 (UTC) (envelope-from thomas@gibfest.dk) Received: from mail.tyknet.dk (mail.tyknet.dk [IPv6:2a01:4f8:201:2327:144:76:253:226]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B97A80652 for ; Wed, 21 Mar 2018 07:20:44 +0000 (UTC) (envelope-from thomas@gibfest.dk) Received: from [10.137.3.13] (217.71.4.82.static.router4.bolignet.dk [217.71.4.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.tyknet.dk (Postfix) with ESMTPSA id E7770BD7AFD; Wed, 21 Mar 2018 07:20:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.tyknet.dk E7770BD7AFD DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gibfest.dk; s=default; t=1521616843; bh=STSMwvrCYZDLryF7bHWTDjWkYYeXxAB98Fo0eYBykYs=; h=Subject:To:References:From:Date:In-Reply-To; b=q4aBe52bFsKv4f8166xxfwgYfv8HIeJNi7B0OWOH5FiXyEOLDgEwwJL8X+wLS54zU fYiY21EJD98RsI/7JQLyRHBJhyVutRk3USJV44/gY4c5v7AZ8YaOqiRfEytbpovWUr vmhx9z1i1nqNt/nlEa4uspA2l2hjgLYnVTX1xO6BCsHkJzaqJrj8/bjor1SrHmX3N2 eIhPsrL68cLWsuKKUrHqLeIAssfsjqiakUU2ZIM1XlP9872a6IZ9o+HnJKw0Kyo0rY lAAoQO6owVHR07Y/sXsOO0lVYwb9MyY2L/LbMDDURipFjfessLBWUCpOnyYqGjzKed 9kXNtVB4cgp/Q== Subject: Re: ZFS i/o error in recent 12.0 To: Markus Wild , freebsd-current@freebsd.org References: <201803192300.w2JN04fx007127@kx.openedu.org> <20180320085028.0b15ff40@mwoffice.virtualtec.office> From: Thomas Steen Rasmussen Message-ID: Date: Wed, 21 Mar 2018 08:20:42 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180320085028.0b15ff40@mwoffice.virtualtec.office> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2018 07:20:44 -0000 On 03/20/2018 08:50 AM, Markus Wild wrote: > > I had faced the exact same issue on a HP Microserver G8 with 8TB disks and a 16TB zpool on FreeBSD 11 about a year ago. Hello, I will ask you the same question as I asked the OP: Has this pool had new vdevs addded to it since the server was installed? What does a "zpool status" look like when the pool is imported? Explanation: Some controllers only make a small fixed number of devices visible to the bios during boot. Imagine a zpool was booted with, say, 4 disks in a pool, and 4 more was added. If the HBA only shows 4 drives to the bios during boot, you see this error. If you think this might be relevant you need to chase down a setting called "maximum int13 devices for this adapter" or something like that. See page 3-4 in this documentation: https://supermicro.com/manuals/other/LSI_HostRAID_2308.pdf The setting has been set to 4 on a bunch of servers I've bought over the last years. Then you install the server with 4 disks, later add new disks, reboot one day and nothing works until you set it high enough that the bootloader can see the whole pool, and you're good again. /Thomas