From owner-freebsd-current@freebsd.org Sun May 7 05:45:14 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4327BD62166 for ; Sun, 7 May 2017 05:45:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE76669E for ; Sun, 7 May 2017 05:45:13 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x236.google.com with SMTP id k91so34910505ioi.1 for ; Sat, 06 May 2017 22:45:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=AakS+ajkWL9kNu06sdmCKkFCv1z33X1diH9xpX4eE/0=; b=CRxc5l3KbEECmmR5ZBpSR+A/C80pSdol996GGAjVFsaCSW5oAckKNv1NE8tM4GM1wR yfHlHt4Luu0jBcBOXf1gwASRL1ObZYPa0us+tnSPJxzihpIh0onCdqcBWFFnq9FykOb5 a1Hzsy88VgU7H0RBsdIozQEvJoqFbwaNhbdfIMlrk8wQBNCFxjK2iwcikSaZ+/aSfiIo XRZzulJm8ZznZYg8gt7N8ONnQTJzxZGaJKou7VVBjpLWE7TDuYlg/3xmwWWRh2g08jzK MR82ht6nBeFm004ljAtnUPvc5q7ZlmT5BvEPGwCfbrF9ykjYQDs0jtD2uNQXKSIz/YA+ F4zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=AakS+ajkWL9kNu06sdmCKkFCv1z33X1diH9xpX4eE/0=; b=fSATvvzdSWMDWl6hIiyJdBxkjiXSCMVCKwP2NCFDYHsT+7ESxwRmLzQX6A4Vn5PIqC VWL95eCqP1X8gEhx1ka2EF3At2y9zl0GBNOMPWXz7wlV3yx1Yqcz3tI7P/d/+2J2wwae 3RX2qCJepTv3wfuUP8m1s9FQuz+aG+Ts596pLLiZnHt0b7i1q12lB99FAYVknKppwPCR Sm77jugGAKs6wRq3KKo4P3NpHAjmAFMPOp924vqfVE0p1niePQsTuuLzMPxjIr1CTSrI iVpCVqzXZxTm9cW0uolSd1W1Ow9lS4oQfYTGjrqc7v1eji6KF5O4iuIW6ejBPgI2dtZn IjqQ== X-Gm-Message-State: AN3rC/5nQs3XpY0ZZnHBTGnt+QQ7ij2R8fpAL4C4W2g3whXeErwBzb8U wWxPYshfMgaiN4JHH7KhjbxLEMchzg== X-Received: by 10.107.7.18 with SMTP id 18mr53952969ioh.218.1494135913035; Sat, 06 May 2017 22:45:13 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.126.6 with HTTP; Sat, 6 May 2017 22:45:12 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:b442:a902:202f:8fc6] In-Reply-To: <972d2a0b-862c-2510-090d-7e8f5d1fce4d@freebsd.org> References: <963c5c97-2f92-9983-cf90-ec9d59d87bba@freebsd.org> <053354DF-651F-423C-8057-494496DA3B91@me.com> <972d2a0b-862c-2510-090d-7e8f5d1fce4d@freebsd.org> From: Warner Losh Date: Sat, 6 May 2017 23:45:12 -0600 X-Google-Sender-Auth: 5fpeFUko5iXt6HOpiciS8a0l3oM Message-ID: Subject: Re: bootcode capable of booting both UFS and ZFS? (Amazon/ec2) To: Julian Elischer Cc: Toomas Soome , freebsd-current , Toomas Soome , Andriy Gapon , Colin Percival Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 May 2017 05:45:14 -0000 On Sat, May 6, 2017 at 10:03 PM, Julian Elischer wrote= : > On 6/5/17 4:01 am, Toomas Soome wrote: >> >> >>> On 5. mai 2017, at 22:07, Julian Elischer >> > wrote: >>> >>> Subject says it all really, is this an option at this time? >>> >>> we'd like to try boot the main zfs root partition and then fall back to= a >>> small UFS based recovery partition.. is that possible? >>> >>> I know we could use grub but I'd prefer keep it in the family. >>> >>> >>> >> >> >> it is, sure. but there is an compromise to be made for it. >> >> Lets start with what I have done in illumos port, as the idea there is >> exactly about having as =E2=80=9Cuniversal=E2=80=9D binaries as possible= (just the binaries >> are listed below to get the size): >> >> -r-xr-xr-x 1 root sys 171008 apr 30 19:55 bootia32.efi >> -r-xr-xr-x 1 root sys 148992 apr 30 19:55 bootx64.efi >> -r--r--r-- 1 root sys 1255 okt 25 2015 cdboot >> -r--r--r-- 1 root sys 154112 apr 30 19:55 gptzfsboot >> -r-xr-xr-x 1 root sys 482293 mai 2 21:10 loader32.efi >> -r-xr-xr-x 1 root sys 499218 mai 2 21:10 loader64.efi >> -r--r--r-- 1 root sys 512 okt 15 2015 pmbr >> -r--r--r-- 1 root sys 377344 mai 2 21:10 pxeboot >> -r--r--r-- 1 root sys 376832 mai 2 21:10 zfsloader >> >> the loader (bios/efi) is built with full complement - zfs, ufs, dosfs, >> cd9660, nfs, tftp + gzipfs. The cdboot is starting zfsloader (thats triv= ial >> string change). >> >> The gptzfsboot in illumos case is only built with zfs, dosfs and ufs - a= s >> it has to support only disk based media to read out the loader. Also I a= m >> building gptzfsboot with libstand and libi386 to get as much shared code= as >> possible - which has both good and bad sides, as usual;) >> >> The gptzfsboot size means that with ufs the dedicated boot partition is >> needed (freebsd-boot), with zfs the illumos port is always using the 3.5= MB >> boot area after first 2 labels (as there is no geli, the illumos does no= t >> need dedicated boot partition with zfs). >> >> As the freebsd-boot is currently created 512k, the size is not an issue. >> Also using common code does allow the generic partition code to be used,= so >> GPT/MBR/BSD (VTOC in illumos case) labels are not problem. >> >> >> So, even just with cd boot (iso), starting zfsloader (which in fbsd has >> built in ufs, zfs etc), you already can get rescue capability. >> >> Now, even with just adding ufs reader to gptzfsboot, we can use gpt + >> freebsd-boot and ufs root but loading zfsloader on usb image, so it can = be >> used for both live/install and rescue, because zfsloader itself has supp= ort >> for all file systems + partition types. >> >> I have kept myself a bit off from freebsd gptzfsboot because of simple >> reason - the older setups have smaller size for freebsd boot, and not >> everyone is necessarily happy about size changes:D also in freebsd case >> there is another factor called geli - it most certainly does contribute = some >> bits, but also needs to be properly addressed on IO call stack (as we ha= ve >> seen with zfsbootcfg bits). But then again, here also the shared code ca= n >> help to reduce the complexity. >> >> Yea, the zfsloader/loader*.efi in that listing above is actually built >> with framebuffer code and compiled in 8x16 default font (lz4 compressed >> ascii+boxdrawing basically - because zfs has lz4, the decompressor is al= ways >> there), and ficl 4.1, so thats a bit of difference from fbsd loader. >> >> Also note that we can still build the smaller dedicated blocks like boot= 2, >> just that we can not use those blocks for more universal cases and >> eventually those special cases will diminish. > > > thanks for that.. > > so, here's my exact problem I need to solve. > FreeBSD 10 (or newer) on Amazon EC2. > We need to have a plan for recovering the scenario where somethign goes > wrong (e.g. during an upgrade) and we are left with a system where the > default zpool rootfs points to a dataset that doesn't boot. It is possibl= e > that mabe the entire pool is unbootable into multi-user.. Maybe somehow = it > filled up? who knows. It's hard to predict future problems. > There is no console access at all so there is no possibility of human > intervention. So all recovery paths that start "enter single user mode > and...." are unusable. > > The customers who own the amazon account are not crazy about giving us th= e > keys to the kingdom as far as all their EC2 instances, so taking a root > drive off a 'sick' VM and grafting it onto a freebsd instance to 'repair'= it > becomes a task we don't want to really have to ask them to do. They may n= ot > have the in-house expertise to do it. confidently. > > This leaves us with automatic recovery, or at least automatic methods of > getting access to that drive from the network. > Since the regular root is zfs, my gut feeling is that to deduce the chanc= es > of confusion during recovery, I'd like the (recovery) system itself to be > running off a UFS partition, and potentially, with a memory root filesyst= em. > As long as it can be reached over the network we can then take over. > > we'd also like to have the boot environment support in the bootcode. > so, what would be the minimum set we'd need? > > Ufs support, zfs support, BE support, and support for selecting a complet= ely > different boot procedure after some number of boot attempts without getti= ng > all the way to multi-user. > > How does that come out size-wise? And what do I need to configure to ge= t > that? > > The current EC2 Instances have a 64kB boot partition , but I have a windo= w > to convince management to expand that if I have a good enough argument. > (since we a re doing a repartition on the next upgrade, which is "special= " > (it's out upgrade to 10.3 from 8.0). > Being able to self heal or at least 'get at' a sick instance might be a g= ood > enough argument and would make the EC2 instances the same as all the othe= r > versions of the product.. You should convince them to move to 512k post-haste. I doubt 64k will suffice, and 512k is enough to get all the features you desire. Warner > /me has thought.. I wonder if the ec2 instance bios has enough network > support to allow PXE-like behaviour? or at least being able to receive > packets..? > >> >> rgds, >> toomas >> > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org= "