From owner-freebsd-cloud@freebsd.org Wed Jan 9 09:31:03 2019 Return-Path: <owner-freebsd-cloud@freebsd.org> Delivered-To: freebsd-cloud@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57E9F149CB6B for <freebsd-cloud@mailman.ysv.freebsd.org>; Wed, 9 Jan 2019 09:31:03 +0000 (UTC) (envelope-from 0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com) Received: from a8-237.smtp-out.amazonses.com (a8-237.smtp-out.amazonses.com [54.240.8.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E187752E7 for <freebsd-cloud@freebsd.org>; Wed, 9 Jan 2019 09:31:02 +0000 (UTC) (envelope-from 0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=dqtolf56kk3wpt62c3jnwboqvr7iedax; d=tarsnap.com; t=1547026027; h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=cORvlez6PwnYDnQ+lyJVp20/IK4HZibdIldXXP9pMtU=; b=Y3YxESELAL+ZCNdhQipp67B0KSTUSwWzcPwgS4JFQvbN6H7DaSPddEckey1GHTDw eEPkY9AqXO2lb34q/Wy2jaqBGaHOvMFGyQTZlhgXnsvnpaTejVxxjVFRr3cyuQnlMFU niGca732M2aDJCwbtKFOiQdptQL3vUYuXNDly91I= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug; d=amazonses.com; t=1547026027; h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=cORvlez6PwnYDnQ+lyJVp20/IK4HZibdIldXXP9pMtU=; b=F7eHjfULDxxzdY12mO5YD85UZA0nKVFIwnNtOXdrJK8g4qhAIMxO834tPp2qQf/t NxMl5z1JC6XoOOjAPnmpSq8+cTrM1u4/9iip41XZ3Kk50HgX6WtqjG3qPpSZ+6cvWCR cvSrW+P7SOtQWLGH1KGT7+T7bfb2irVKVQDZmro4= Subject: Re: Any plans for EC2 AMIs with RootOnZFS? To: Ben Woods <woodsb02@gmail.com> Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>, freebsd-cloud@freebsd.org References: <CAOc73CBQ+282amu-d+cxUehLBuhPKSrWTnft1_eSvY5FNKynwg@mail.gmail.com> <01000167dd7b16cd-58ee733f-c662-42d5-9c75-2301fbc8377a-000000@email.amazonses.com> <01000167f27b4c13-41c15731-ae4c-4658-820d-80c4d4fbb01c-000000@email.amazonses.com> <CAOc73CBQBOYT6XTP35HuiP8VxXyoqXSxNkEt-TZ=Y-0u1aBT8A@mail.gmail.com> From: Colin Percival <cperciva@tarsnap.com> Openpgp: preference=signencrypt Autocrypt: addr=cperciva@tarsnap.com; prefer-encrypt=mutual; keydata= mQGhBElrAAcRBACDfDys4ZtK+ErCJ1HAzYeteKpm3OEsvT/49AjUTLihkF79HhIKrCQU+1KC zv7BwHCMLb6hq30As9L7iFKG7n5QFLFC4Te/VcITUnWHMG/c3ViLOfJGvi+9/nOEHaM1dVJY D6tEp5yM1nHmVQpo9932j4KGuGFR0LhOK5IHXOSfGwCgxSFDPdgxe2OEjWxjGgY+oV3EafcD +JROXCTjlcQiG/OguQH4Vks3mhHfFnEppLxTkDuYgHZQiUtpcT9ssH5khgqoTyMar05OUdAj ZIhNbWDh4LgTj+7ZmvLhXT5Zxw8LX9d7T36aTB8XDQSenDqEtinMWOb0TCBBLbsB8EFG1WTT ESbZci9jJS5yhtktuZoY/eM8uXMD/3k4FWFO80VRRkELSp+XSy/VlSQjyi/rhl2nQq/oOA9F oJbDaB0yq9VNhxP+uFBzBWSqeIX0t1ZWLtNfVFr4TRP5hihI5ICrg/0OpqgisKsU2NFe9xyO hyJLYmfD8ebpDJ/9k30C7Iju9pVrwLm1QgS4S2fqJRcR+U4WbjvP7CgStCVDb2xpbiBQZXJj aXZhbCA8Y3BlcmNpdmFAdGFyc25hcC5jb20+iGEEExECACEFAklrALYCGwMHCwkIBwMCAQQV AggDBBYCAwECHgECF4AACgkQOM7KaQxqam6/igCgn+z2k3V5ggNppmWrZstt1U2lugsAoL7L wS9V9yLtil3oWmHtwpUqYruEuQINBElrAAcQCAD3ZLMIsP4CIDoJORg+YY0lqLVBgcnF7pFb 4Uy2+KvdWofN+DKH61rZLjgXXkNE9M4EQC1B4lGttBP8IY2gs41y3AUogGdyFbidq99rCBz7 LTsgARHwFxZoaHmXyiZLEU1QZuMqwPZV1mCviRhN5E3rRqYNXVcrnXAAuhBpvNyj/ntHvcDN 2/m+ochiuBYueU4kX3lHya7sOj+mTsndcWmQ9soOUyr8O0r/BG088bMn4qqtUw4dl5/pglXk jbl7uOOPinKf0WVd2r6M0wLPJCD4NPHrCWRLLLAjwfjrtoSRvXxDbXhCdgGBa72+K8eYLzVs hgq7tJOoBWzjVK6XRxR7AAMGB/9Mo3iJ2DxqDecd02KCB5BsFDICbJGhPltU7FwrtbC7djSb XUrwsEVLHi4st4cbdGNCWCrp0BRezXZKohKnNAPFOTK++ZfgeKxrV2sJod+Q9RILF86tQ4XF 7A7Yme5hy92t/WgiU4vc/fWbgP8gV/19f8nunaT2E9NSa70mZFjZNu4iuwThoUUO5CV3Wo0Y UISsnRK8XD1+LR3A2qVyLiFRwh/miC1hgLFCTGCQ3GLxZeZzIpYSlGdQJ0L5lixW5ZQD9r1I 8i/8zhE6qRFAM0upUMI3Gt1Oq2w03DiXrZU0Fu/R8Rm8rlnkQKA+95mRTUq1xL5P5NZIi4gJ Z569OPMFiEkEGBECAAkFAklrAAcCGwwACgkQOM7KaQxqam41igCfbaldnFTu5uAdrnrghESv EI3CAo8AoLkNMks1pThl2BJNRm4CtTK9xZeH Message-ID: <0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@email.amazonses.com> Date: Wed, 9 Jan 2019 09:27:07 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: <CAOc73CBQBOYT6XTP35HuiP8VxXyoqXSxNkEt-TZ=Y-0u1aBT8A@mail.gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2019.01.09-54.240.8.237 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-Rspamd-Queue-Id: 5E187752E7 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tarsnap.com header.s=dqtolf56kk3wpt62c3jnwboqvr7iedax header.b=Y3YxESEL; dkim=pass header.d=amazonses.com header.s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug header.b=F7eHjfUL; spf=pass (mx1.freebsd.org: domain of 0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com designates 54.240.8.237 as permitted sender) smtp.mailfrom=0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com X-Spamd-Result: default: False [-1.73 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18]; DKIM_TRACE(0.00)[tarsnap.com:+,amazonses.com:+]; MX_GOOD(-0.01)[feedback-smtp.us-east-1.amazonses.com,feedback-smtp.us-east-1.amazonses.com,feedback-smtp.us-east-1.amazonses.com]; NEURAL_HAM_SHORT(-0.92)[-0.924,0]; FORGED_SENDER(0.30)[cperciva@tarsnap.com,0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; IP_SCORE(-2.10)[ip: (-2.50), ipnet: 54.240.8.0/21(-4.59), asn: 14618(-3.34), country: US(-0.08)]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; FROM_NEQ_ENVFROM(0.00)[cperciva@tarsnap.com,0100016831efd1f5-6da3fa16-a86b-46ea-a1f4-8d9bb073153c-000000@amazonses.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[tarsnap.com:s=dqtolf56kk3wpt62c3jnwboqvr7iedax,amazonses.com:s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[tarsnap.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[237.8.240.54.list.dnswl.org : 127.0.15.0]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" <freebsd-cloud.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-cloud>, <mailto:freebsd-cloud-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-cloud/> List-Post: <mailto:freebsd-cloud@freebsd.org> List-Help: <mailto:freebsd-cloud-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-cloud>, <mailto:freebsd-cloud-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 09 Jan 2019 09:31:03 -0000 Hi Ben, On 1/5/19 4:39 AM, Ben Woods wrote: > On Fri, 28 Dec 2018 at 12:43, Colin Percival <cperciva@tarsnap.com> wrote: >> I've created an experimental FreeBSD 12.0 ZFS AMI in the us-east-1 region: >> ami-0786f5b55d5aa573f > > Thanks a lot for doing this - much appreciated. > > I have just given it a try, and have the following feedback for you: > > 1. ZFS DATASETS AND PROPERTIES SHOULD MATCH INSTALLER DEFAULT Oops. I thought I was doing this, but I guess the wiki page I was following (https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot) didn't quite match... > Looking at how you created the zpool and zfs datasets (using the "zpool > history zroot" command), the following changes would achieve this outcome: > [snip] Thanks! > 2. LOADER.CONF SETTINGS > The following settings should be included in /boot/loader.conf: > kern.geom.label.disk_ident.enable="0" > kern.geom.label.gptid.enable="0" > > 3. SYSCTL.CONF SETTINGS > The following settings should be included in /etc/sysctl.conf: > vfs.zfs.min_auto_ashift=12 Got it. > 4. THE GPT TABLE IS INITIALLY CORRUPT > I suspect this is because I used a 50Gb disk, and it hasn't been expanded > correctly. > The following error was in DMESG and I suspect is related: > GEOM: ada0: the secondary GPT header is not in the last LBA. Yes. This message appears whenever you launch an instance with a larger-than-default root disk; with UFS, the AMI promptly fixes the GPT and resizes the UFS filesystem. We can't avoid having the message (well, not without fiddling with the kernel bits to silence the warning) but we should fix the autoresizing to work with ZFS as well as UFS. > I was able to fix this with the following commands on firstboot: > # gpart recover ada0 > # gpart resize -i 2 ada0 > # zpool online -e zroot ada0p2 Can you look at /etc/rc.d/growfs (which currently only works for UFS, using the 'growfs' utility) and see if you can teach it to handle ZFS filesystems? Alternatively, maybe someone can teach growfs(8) to say "hey, this looks like a disk with ZFS... let's run 'zpool online -e' instead"? Either way, I don't think this is something I can fix for a FreeBSD 12.0 ZFS AMI, but if the growfs rc.d script can be made to do the right thing with ZFS then at least future releases will work. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid