From owner-freebsd-cloud@freebsd.org  Wed Jan  9 09:31:03 2019
Return-Path: <owner-freebsd-cloud@freebsd.org>
Delivered-To: freebsd-cloud@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57E9F149CB6B
 for <freebsd-cloud@mailman.ysv.freebsd.org>;
 Wed,  9 Jan 2019 09:31:03 +0000 (UTC) (envelope-from
 0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com)
Received: from a8-237.smtp-out.amazonses.com (a8-237.smtp-out.amazonses.com
 [54.240.8.237])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 5E187752E7
 for <freebsd-cloud@freebsd.org>; Wed,  9 Jan 2019 09:31:02 +0000 (UTC)
 (envelope-from
 0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
 s=dqtolf56kk3wpt62c3jnwboqvr7iedax; d=tarsnap.com; t=1547026027;
 h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding;
 bh=cORvlez6PwnYDnQ+lyJVp20/IK4HZibdIldXXP9pMtU=;
 b=Y3YxESELAL+ZCNdhQipp67B0KSTUSwWzcPwgS4JFQvbN6H7DaSPddEckey1GHTDw
 eEPkY9AqXO2lb34q/Wy2jaqBGaHOvMFGyQTZlhgXnsvnpaTejVxxjVFRr3cyuQnlMFU
 niGca732M2aDJCwbtKFOiQdptQL3vUYuXNDly91I=
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
 s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug; d=amazonses.com; t=1547026027;
 h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID;
 bh=cORvlez6PwnYDnQ+lyJVp20/IK4HZibdIldXXP9pMtU=;
 b=F7eHjfULDxxzdY12mO5YD85UZA0nKVFIwnNtOXdrJK8g4qhAIMxO834tPp2qQf/t
 NxMl5z1JC6XoOOjAPnmpSq8+cTrM1u4/9iip41XZ3Kk50HgX6WtqjG3qPpSZ+6cvWCR
 cvSrW+P7SOtQWLGH1KGT7+T7bfb2irVKVQDZmro4=
Subject: Re: Any plans for EC2 AMIs with RootOnZFS?
To: Ben Woods <woodsb02@gmail.com>
Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>,
 freebsd-cloud@freebsd.org
References: <CAOc73CBQ+282amu-d+cxUehLBuhPKSrWTnft1_eSvY5FNKynwg@mail.gmail.com>
 <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com>
 <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com>
 <CAOc73CBQBOYT6XTP35HuiP8VxXyoqXSxNkEt-TZ=Y-0u1aBT8A@mail.gmail.com>
From: Colin Percival <cperciva@tarsnap.com>
Openpgp: preference=signencrypt
Autocrypt: addr=cperciva@tarsnap.com; prefer-encrypt=mutual; keydata=
 mQGhBElrAAcRBACDfDys4ZtK+ErCJ1HAzYeteKpm3OEsvT/49AjUTLihkF79HhIKrCQU+1KC
 zv7BwHCMLb6hq30As9L7iFKG7n5QFLFC4Te/VcITUnWHMG/c3ViLOfJGvi+9/nOEHaM1dVJY
 D6tEp5yM1nHmVQpo9932j4KGuGFR0LhOK5IHXOSfGwCgxSFDPdgxe2OEjWxjGgY+oV3EafcD
 +JROXCTjlcQiG/OguQH4Vks3mhHfFnEppLxTkDuYgHZQiUtpcT9ssH5khgqoTyMar05OUdAj
 ZIhNbWDh4LgTj+7ZmvLhXT5Zxw8LX9d7T36aTB8XDQSenDqEtinMWOb0TCBBLbsB8EFG1WTT
 ESbZci9jJS5yhtktuZoY/eM8uXMD/3k4FWFO80VRRkELSp+XSy/VlSQjyi/rhl2nQq/oOA9F
 oJbDaB0yq9VNhxP+uFBzBWSqeIX0t1ZWLtNfVFr4TRP5hihI5ICrg/0OpqgisKsU2NFe9xyO
 hyJLYmfD8ebpDJ/9k30C7Iju9pVrwLm1QgS4S2fqJRcR+U4WbjvP7CgStCVDb2xpbiBQZXJj
 aXZhbCA8Y3BlcmNpdmFAdGFyc25hcC5jb20+iGEEExECACEFAklrALYCGwMHCwkIBwMCAQQV
 AggDBBYCAwECHgECF4AACgkQOM7KaQxqam6/igCgn+z2k3V5ggNppmWrZstt1U2lugsAoL7L
 wS9V9yLtil3oWmHtwpUqYruEuQINBElrAAcQCAD3ZLMIsP4CIDoJORg+YY0lqLVBgcnF7pFb
 4Uy2+KvdWofN+DKH61rZLjgXXkNE9M4EQC1B4lGttBP8IY2gs41y3AUogGdyFbidq99rCBz7
 LTsgARHwFxZoaHmXyiZLEU1QZuMqwPZV1mCviRhN5E3rRqYNXVcrnXAAuhBpvNyj/ntHvcDN
 2/m+ochiuBYueU4kX3lHya7sOj+mTsndcWmQ9soOUyr8O0r/BG088bMn4qqtUw4dl5/pglXk
 jbl7uOOPinKf0WVd2r6M0wLPJCD4NPHrCWRLLLAjwfjrtoSRvXxDbXhCdgGBa72+K8eYLzVs
 hgq7tJOoBWzjVK6XRxR7AAMGB/9Mo3iJ2DxqDecd02KCB5BsFDICbJGhPltU7FwrtbC7djSb
 XUrwsEVLHi4st4cbdGNCWCrp0BRezXZKohKnNAPFOTK++ZfgeKxrV2sJod+Q9RILF86tQ4XF
 7A7Yme5hy92t/WgiU4vc/fWbgP8gV/19f8nunaT2E9NSa70mZFjZNu4iuwThoUUO5CV3Wo0Y
 UISsnRK8XD1+LR3A2qVyLiFRwh/miC1hgLFCTGCQ3GLxZeZzIpYSlGdQJ0L5lixW5ZQD9r1I
 8i/8zhE6qRFAM0upUMI3Gt1Oq2w03DiXrZU0Fu/R8Rm8rlnkQKA+95mRTUq1xL5P5NZIi4gJ
 Z569OPMFiEkEGBECAAkFAklrAAcCGwwACgkQOM7KaQxqam41igCfbaldnFTu5uAdrnrghESv
 EI3CAo8AoLkNMks1pThl2BJNRm4CtTK9xZeH
Message-ID: <0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@email.amazonses.com>
Date: Wed, 9 Jan 2019 09:27:07 +0000
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101
 Thunderbird/60.0
MIME-Version: 1.0
In-Reply-To: <CAOc73CBQBOYT6XTP35HuiP8VxXyoqXSxNkEt-TZ=Y-0u1aBT8A@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-SES-Outgoing: 2019.01.09-54.240.8.237
Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES
X-Rspamd-Queue-Id: 5E187752E7
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
 dkim=pass header.d=tarsnap.com header.s=dqtolf56kk3wpt62c3jnwboqvr7iedax
 header.b=Y3YxESEL; 
 dkim=pass header.d=amazonses.com header.s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug
 header.b=F7eHjfUL; 
 spf=pass (mx1.freebsd.org: domain of
 0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com
 designates 54.240.8.237 as permitted sender)
 smtp.mailfrom=0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com
X-Spamd-Result: default: False [-1.73 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[];
 TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18];
 DKIM_TRACE(0.00)[tarsnap.com:+,amazonses.com:+];
 MX_GOOD(-0.01)[feedback-smtp.us-east-1.amazonses.com,feedback-smtp.us-east-1.amazonses.com,feedback-smtp.us-east-1.amazonses.com];
 NEURAL_HAM_SHORT(-0.92)[-0.924,0];
 FORGED_SENDER(0.30)[cperciva@tarsnap.com,0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com];
 FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0];
 IP_SCORE(-2.10)[ip: (-2.50), ipnet: 54.240.8.0/21(-4.59), asn: 14618(-3.34),
 country: US(-0.08)]; MIME_TRACE(0.00)[0:+];
 ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US];
 FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[];
 FROM_NEQ_ENVFROM(0.00)[cperciva@tarsnap.com,0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com];
 ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
 R_DKIM_ALLOW(-0.20)[tarsnap.com:s=dqtolf56kk3wpt62c3jnwboqvr7iedax,amazonses.com:s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug];
 FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3];
 NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
 DMARC_NA(0.00)[tarsnap.com]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 RCVD_IN_DNSWL_NONE(0.00)[237.8.240.54.list.dnswl.org : 127.0.15.0];
 SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[]
X-BeenThere: freebsd-cloud@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure,
 etc.\)" <freebsd-cloud.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-cloud>,
 <mailto:freebsd-cloud-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-cloud/>
List-Post: <mailto:freebsd-cloud@freebsd.org>
List-Help: <mailto:freebsd-cloud-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-cloud>,
 <mailto:freebsd-cloud-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Jan 2019 09:31:03 -0000

Hi Ben,

On 1/5/19 4:39 AM, Ben Woods wrote:
> On Fri, 28 Dec 2018 at 12:43, Colin Percival <cperciva@tarsnap.com> wrote:
>> I've created an experimental FreeBSD 12.0 ZFS AMI in the us-east-1 region:
>> ami-0786f5b55d5aa573f
> 
> Thanks a lot for doing this - much appreciated.
> 
> I have just given it a try, and have the following feedback for you:
> 
> 1. ZFS DATASETS AND PROPERTIES SHOULD MATCH INSTALLER DEFAULT

Oops.  I thought I was doing this, but I guess the wiki page I was following
(https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot) didn't quite match...

> Looking at how you created the zpool and zfs datasets (using the "zpool
> history zroot" command), the following changes would achieve this outcome:
> [snip]

Thanks!

> 2. LOADER.CONF SETTINGS
> The following settings should be included in /boot/loader.conf:
> kern.geom.label.disk_ident.enable="0"
> kern.geom.label.gptid.enable="0"
> 
> 3. SYSCTL.CONF SETTINGS
> The following settings should be included in /etc/sysctl.conf:
> vfs.zfs.min_auto_ashift=12

Got it.

> 4. THE GPT TABLE IS INITIALLY CORRUPT
> I suspect this is because I used a 50Gb disk, and it hasn't been expanded
> correctly.
> The following error was in DMESG and I suspect is related:
> GEOM: ada0: the secondary GPT header is not in the last LBA.

Yes.  This message appears whenever you launch an instance with a
larger-than-default root disk; with UFS, the AMI promptly fixes the GPT and
resizes the
UFS filesystem.

We can't avoid having the message (well, not without fiddling with the kernel
bits to silence the warning) but we should fix the autoresizing to work with
ZFS as well as UFS.

> I was able to fix this with the following commands on firstboot:
> # gpart recover ada0
> # gpart resize -i 2 ada0
> # zpool online -e zroot ada0p2

Can you look at /etc/rc.d/growfs (which currently only works for UFS, using
the 'growfs' utility) and see if you can teach it to handle ZFS filesystems?

Alternatively, maybe someone can teach growfs(8) to say "hey, this looks like
a disk with ZFS... let's run 'zpool online -e' instead"?

Either way, I don't think this is something I can fix for a FreeBSD 12.0 ZFS
AMI, but if the growfs rc.d script can be made to do the right thing with ZFS
then at least future releases will work.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid