From owner-freebsd-fs@FreeBSD.ORG Thu Sep 17 19:43:36 2009 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42FD6106566C for ; Thu, 17 Sep 2009 19:43:36 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-yw0-f191.google.com (mail-yw0-f191.google.com [209.85.211.191]) by mx1.freebsd.org (Postfix) with ESMTP id EDFA58FC20 for ; Thu, 17 Sep 2009 19:43:35 +0000 (UTC) Received: by ywh29 with SMTP id 29so595699ywh.7 for ; Thu, 17 Sep 2009 12:43:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to :content-type:content-transfer-encoding; bh=n1CR7y6FJuc/sYX2p1BS9ahzylOijC0JgKoYRjPn/U8=; b=W2LpMyKlFW4Q0ZWrQBjNMCK1pwlK8muLLAwdvn1JkKb8VJxSaTJ9P+1P2wmd1REUR3 X0K6igjrFgycjfg9qVID1C4Gi74iErXt0DZ4q4RCpew5Ie4V5C5BAN1Zac6REt7tNsxP BaL4x8ttaCblmJC1c6Gbb0IVDq+XNR3zNunfw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; b=j3bXuwMGMqX3XndHxC/9oEVMpQ43D5aV7Amo6qJhn47TtTeEMhBH4E1oNef1lKBVwV dSEsYR00YYIqX/w05ABFlZ9HoqjRa8mAmbZhSdaNcArAzX5NW8/DWycCBOqdAcCgI61F v72oKeyHVrQqFs+6iZtPHAtXmqnsemilapfzs= MIME-Version: 1.0 Sender: artemb@gmail.com Received: by 10.91.141.6 with SMTP id t6mr566023agn.49.1253214988196; Thu, 17 Sep 2009 12:16:28 -0700 (PDT) In-Reply-To: References: Date: Thu, 17 Sep 2009 12:16:28 -0700 X-Google-Sender-Auth: 355b581d86a78d88 Message-ID: From: Artem Belevich To: Matt Reimer , fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Can FreeBSD boot from a pool comprised of multiple vdevs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Sep 2009 19:43:36 -0000 Hi, I've just spent a bit of time investigating the same issue. There was a thread on -current in June that discussed ZFS booting issue and this email suggested that zfsboot does not support gang blocks. http://lists.freebsd.org/pipermail/freebsd-current/2009-June/008589.html If I understand it correctly, gang blocks are used when ZFS can't (or does not want to) allocate contiguous block of required size, so the block is split in sub-blocks that in turn may be gang blocks. I.e. we're looking at a tree of blocks. If that's indeed the case, it implies that even on a single-disk system we may eventually run into a situation when newly installed kernel will be written using gang block and thus would not be handled correctly by zfsboot. That said, I've tried many possible combinations on a VirtualBox running 8.0BETA4 and most of them didn't work. Here's the summary: Zpool configurations I was able to boot from: * 1 disk * 1 mirror of 2 disks * 2 mirrors of 2 disks each * 2-disk raidz (kind of pointless, but..) Configurations that didn't work: * RAIDZ2 with 8 disks * RAIDZ2 with 4 disks * RAIDZ2 with 3 disks * RAIDZ with 8 disks * RAIDZ with 3 disks * 4 mirrors of 2 disks each * 3 mirrors of 3 disks each. Usually gptzfsboot complains that it can't read MOS. Striped multi-disk setups resulted in "can't find dsl_dir". I did some digging in case of RAIDZ2 booting failure and it does look that the failure happens because calculated checksum does not match expected one. Due to lack of my ZFS knowlegde I didn't get deeper than that. --Artem On Thu, Sep 17, 2009 at 10:41 AM, Matt Reimer wrote= : > Is FreeBSD able to boot from a pool comprised of multiple vdevs? If > not, what would it take to get it working? > > I got 8.0-BETA4 booting from a pool comprised of a single raidz vdev, > but when I added another raidz vdev it fails to boot with this error: > > ZFS: i/o error - all block copies unavailable > ZFS: can't read object set for dataset lld > Can't find root filesystem - giving up > ZFS: unexpected object set type lld > ZFS: unexpected object set type lld > > FreeBSD/i386 boot > Default: glamdring:/boot/kernel/kernel > boot: > ZFS: unexpected object set type lld > > (Apparently loader's printf() doesn't understand %lld.) > > Boots successfully with gptzfsboot: > > # zpool status > =A0pool: glamdring > =A0state: ONLINE > =A0scrub: none requested > config: > > =A0 =A0 =A0 =A0NAME =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 STATE =A0 =A0= READ WRITE CKSUM > =A0 =A0 =A0 =A0glamdring =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ONLINE =A0 =A0 = =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0raidz1 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ONLINE =A0 =A0 = =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-0.0 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-0.1 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-0.2 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > > errors: No known data errors > > Does not boot: > > # zpool status > =A0pool: glamdring > =A0state: ONLINE > =A0scrub: none requested > config: > > =A0 =A0 =A0 =A0NAME =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 STATE =A0 =A0= READ WRITE CKSUM > =A0 =A0 =A0 =A0glamdring =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ONLINE =A0 =A0 = =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0raidz1 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ONLINE =A0 =A0 = =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-0.0 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-0.1 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-0.2 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0raidz1 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ONLINE =A0 =A0 = =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-1.0 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-1.1 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0label/glamdring-1.2 =A0ONLINE =A0 =A0 =A0 0 =A0 = =A0 0 =A0 =A0 0 > > errors: No known data errors > > Thanks in advance. > > Matt > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >