From owner-freebsd-stable@FreeBSD.ORG Mon Jan 23 20:43:07 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 663B4106566B for ; Mon, 23 Jan 2012 20:43:07 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 20F2F8FC19 for ; Mon, 23 Jan 2012 20:43:06 +0000 (UTC) Received: by vcbfl17 with SMTP id fl17so3343520vcb.13 for ; Mon, 23 Jan 2012 12:43:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Jz1c8EvADb45FOzguCwKmxXVeiIKe/thqoxjRPAyAE0=; b=BXUB8UiRzxUbJsiVBjeDpjT3IXgCZa66z/KEYIis3Dje9Ld5G2e+M4568H2JI5536O NN5iQPpIV58ZxP+fwQvWeVG56y5L9W3604RW29m93acJet7nawQaPK8JTIS3aShJFHJd MWxGqdqQD33i9pzdMa6xIcEaq+tNa5yUqZmDs= MIME-Version: 1.0 Received: by 10.220.157.83 with SMTP id a19mr6028690vcx.54.1327351386298; Mon, 23 Jan 2012 12:43:06 -0800 (PST) Received: by 10.220.117.11 with HTTP; Mon, 23 Jan 2012 12:43:06 -0800 (PST) In-Reply-To: References: <520B9285BCC0498286196195933D67E9@multiplay.co.uk> Date: Mon, 23 Jan 2012 12:43:06 -0800 Message-ID: From: Freddie Cash To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Cc: freebsd-stable@freebsd.org Subject: Re: ZFS: i/o error - all block copies unavailable on large disk number machines X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 20:43:07 -0000 On Mon, Jan 23, 2012 at 11:21 AM, Steven Hartland wrote: > Not something I've seem made clear, but quite possibly. Even with > 9 disks you could easily get this if the BIOS doesn't see all of > said disks, be that initially or due to disks added to the machine. > > For reference the original install was done on a zpool with 6 disks > in a raidz2 config but then 6 additional disks where added to expand > capacity. > > It was only when the new kernel was installed that data required > to boot was then written to disks in the seconds raidz2 which is > inaccessible to the boot code even though in perfect working order > on a booted system. > > So something to document, watch out for and potentially safe > guard against? > > It maybe something specific to machines with legacy BIOS hence not > an issue with Sun kit? >From what I've gathered on the zfs-discuss mailing list, Solaris only supports rpool's (bootable pool) to use mirror vdevs, and only a single vdev in the rpool. FreeBSD is (AFAIK) the only ZFS implementation that supports booting from a raidz vdev, and from a pool with multiple raidz vdevs. IME, separating the bootable disks from the storage disks will always save you time, effort, and grief in the long run. :) Whether that means using a separate UFS / filesystem, or a mirrored set of disks for /, or a separate ZFS pool with a single mirror vdev is up to the admin. But boot/OS should be separate from bulk storage. :) -- Freddie Cash fjwcash@gmail.com