From owner-freebsd-cloud@freebsd.org Mon Sep 14 15:36:09 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B85493D9F58 for ; Mon, 14 Sep 2020 15:36:09 +0000 (UTC) (envelope-from ric.lombardi@gmail.com) Received: from mail-oi1-x22c.google.com (mail-oi1-x22c.google.com [IPv6:2607:f8b0:4864:20::22c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bqr6w31JJz3cDl for ; Mon, 14 Sep 2020 15:36:08 +0000 (UTC) (envelope-from ric.lombardi@gmail.com) Received: by mail-oi1-x22c.google.com with SMTP id x69so296588oia.8 for ; Mon, 14 Sep 2020 08:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=BsC1iGlOz6LlxOHJp1Fa620D27ebcoh5U1w0boGsE7o=; b=bezOcaMtNJ/twQTszB0IqTKY68Wjr8XuTyr+I3Gx/9IkSEkKnVhBqcSvR1n1Qz3C2l ZF9HQkKhx2orw0uJF6cuslGfadLJbATPdoRLNfsJNi+x0cWd4pYjlUty4m+/qX1gNzQI taFB/z9O5ewHg/AJRdp+8u/Ox78A5tDF2lRy0dqGzJtRkjEK/BNbkZ9JZlvd0qi0nRip srIUCaHDBguXW5xJZZrTI91fjTYWoBrhX0S2gGCQ37Ebhrgqi7K8o1Y99oJLDz3Jzo84 107zCQEatMh0Qt24R0ZUtGqW+xX2dWBNhlvnwjDziBAOpOFKak6QheXt7GhO4Vo3hPTQ sIqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=BsC1iGlOz6LlxOHJp1Fa620D27ebcoh5U1w0boGsE7o=; b=ZhMN3b6LPBNqzmuwQbOud5IdbGlnFGdJEwzm1VN4TKUCtijl8sACBRCf+8bMXGyKv8 DVX3+Vw6sf0da9LJrTReGWhwlBWJMCnEfFyA9TmBZvj0iA+WMA1L30eATEjO1LfuG7PV P4KwmwuIz9n59aqkaproBAyvtG1jdjYVObZU16VWx38XXZ+pU3W+4UCFceHc49JRrXjW juOXVywIlpy5u9vrmgCZ0GAIr9IJDUm4VGoAJ4fonm/pSBNcxcianL/hxG+VLPHn1IfB MhOWPgmXCb+sLMZutcVPqjz4jiXtMmhrPRiqCaXQhaTmhJdkTweScWGQc1KCKWlSC0z7 BwOg== X-Gm-Message-State: AOAM533b0Nkxo3rz6CmjpQb6f0Q8I5xXz0ZxgGI9pZ2nf8rBjRfkapWn MUvHcaNSamrI/bwG5RTcYyDWFctgQGqRr75Amejoy+CE5+4hRw== X-Google-Smtp-Source: ABdhPJxlGDz9p43XGLLZMO/R/TZrXznIxeqYE1KUbD1H7RYq6KtA6+wqAKI5VOBqxK2gkIao8SzHwSmb60LiorHHMqo= X-Received: by 2002:aca:f302:: with SMTP id r2mr8598369oih.151.1600097766960; Mon, 14 Sep 2020 08:36:06 -0700 (PDT) MIME-Version: 1.0 From: Riccardo Lombardi Date: Mon, 14 Sep 2020 17:35:31 +0200 Message-ID: Subject: EC2 AMI size of Freebsd 12 (and maybe a feature request) To: freebsd-cloud@freebsd.org X-Rspamd-Queue-Id: 4Bqr6w31JJz3cDl X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=bezOcaMt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of riclombardi@gmail.com designates 2607:f8b0:4864:20::22c as permitted sender) smtp.mailfrom=riclombardi@gmail.com X-Spamd-Result: default: False [-3.46 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.01)[-1.014]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22c:from]; NEURAL_HAM_SHORT(-0.44)[-0.445]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-cloud]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Sep 2020 15:36:09 -0000 Hi all and hi Colin, First of all, thank you for all your efforts for making FreeBSD usable in EC2 environments, and sorry for my english. I'm mainly a hobby web developer, with limited (end-user) experience in Unix systems. I recently switched to FreeBSD for my web server and noticed that the "out-of-the-box" FreeBSD 12 AMI in AWS uses a lot of disk space (3.8Gb using the df -h command) in comparison with other Linux distributions, which usually start with 1-1.5Gb disk space. I also wanted to point out that the size of the FreeBSD 11 AMI is strangely quite smaller (2.4G). The largest directories I could find in FreeBSD 12 are: /usr/lib/debug: 1.5G /var/db/freebsd-update: 508M My aim (and I suppose this could be useful for many developers and EC2 users) was originally to shrink the AMI size by eliminating things that are not necessary in a production environment (maybe the kernel-debug tools). Unfortunately I don't think i have enough knowledge to make my own custom light AMI. So, if this could be useful for the community, and I think so, it would be very nice to see some of the following things / features: - First of all check it there's some problem with the disk size of the FreeBSD 12 (compared with the 11, maybe there are old unused libraries from the update), and eventually publish a new, "clean", AMI - OR/AND maybe implement a "lightweight" version of the FreeBSD AMI, without some non essential components. - Reduce the minimum EBS volume size from the current 10G to 8 or maybe 5 (accordingly with the final size of the new AMI) Thanks and best regards. Riccardo Lombardi From owner-freebsd-cloud@freebsd.org Mon Sep 14 22:04:41 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 803F63E37B9 for ; Mon, 14 Sep 2020 22:04:41 +0000 (UTC) (envelope-from 010001748ea604b7-67a61380-0272-4a14-94e3-e2d80f02c763-000000@amazonses.com) Received: from a8-237.smtp-out.amazonses.com (a8-237.smtp-out.amazonses.com [54.240.8.237]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Br0lD4Nfjz4P59 for ; Mon, 14 Sep 2020 22:04:40 +0000 (UTC) (envelope-from 010001748ea604b7-67a61380-0272-4a14-94e3-e2d80f02c763-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn; d=tarsnap.com; t=1600121079; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=KoO2+qlBqyIGm0bBuGxkVW8HTcj2bLOc7rDBPOx5MOc=; b=mH691y0hsvA74Gn05JOxSQLP8jZSOn6nMCUhfYpbhgzuxgjU8ogY4f1N+5Imxybb i+a78q89QzB15hzsumopZLx1hXWazuOJgr856WIIXJBdOlMz4e6JsnlyLsIkAm4Dz0G R8cjB551gAMpWTmseM1nir0CEeBbkCjveJaayB0A= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1600121079; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=KoO2+qlBqyIGm0bBuGxkVW8HTcj2bLOc7rDBPOx5MOc=; b=LqQ9KVkzsQ6wd1nxm0WjqNnpkrjRdvG4L/b+Y4PT6HxZdtXG3/3pDEW4L8AplhYh vGcXnLfgUot2ysiButs8O5640T+XeLWWg23GCf6C0b9gyD0UrAHu/8HCYsMnWqD9Xzg 7w1A2if4IjtK70axzmrceC00CStNGA7nnUFB8MCA= Subject: Re: EC2 AMI size of Freebsd 12 (and maybe a feature request) To: Riccardo Lombardi , freebsd-cloud@freebsd.org References: From: Colin Percival Autocrypt: addr=cperciva@tarsnap.com; prefer-encrypt=mutual; keydata= mQGhBElrAAcRBACDfDys4ZtK+ErCJ1HAzYeteKpm3OEsvT/49AjUTLihkF79HhIKrCQU+1KC zv7BwHCMLb6hq30As9L7iFKG7n5QFLFC4Te/VcITUnWHMG/c3ViLOfJGvi+9/nOEHaM1dVJY D6tEp5yM1nHmVQpo9932j4KGuGFR0LhOK5IHXOSfGwCgxSFDPdgxe2OEjWxjGgY+oV3EafcD +JROXCTjlcQiG/OguQH4Vks3mhHfFnEppLxTkDuYgHZQiUtpcT9ssH5khgqoTyMar05OUdAj ZIhNbWDh4LgTj+7ZmvLhXT5Zxw8LX9d7T36aTB8XDQSenDqEtinMWOb0TCBBLbsB8EFG1WTT ESbZci9jJS5yhtktuZoY/eM8uXMD/3k4FWFO80VRRkELSp+XSy/VlSQjyi/rhl2nQq/oOA9F oJbDaB0yq9VNhxP+uFBzBWSqeIX0t1ZWLtNfVFr4TRP5hihI5ICrg/0OpqgisKsU2NFe9xyO hyJLYmfD8ebpDJ/9k30C7Iju9pVrwLm1QgS4S2fqJRcR+U4WbjvP7CgStCVDb2xpbiBQZXJj aXZhbCA8Y3BlcmNpdmFAdGFyc25hcC5jb20+iGEEExECACEFAklrALYCGwMHCwkIBwMCAQQV AggDBBYCAwECHgECF4AACgkQOM7KaQxqam6/igCgn+z2k3V5ggNppmWrZstt1U2lugsAoL7L wS9V9yLtil3oWmHtwpUqYruEuQINBElrAAcQCAD3ZLMIsP4CIDoJORg+YY0lqLVBgcnF7pFb 4Uy2+KvdWofN+DKH61rZLjgXXkNE9M4EQC1B4lGttBP8IY2gs41y3AUogGdyFbidq99rCBz7 LTsgARHwFxZoaHmXyiZLEU1QZuMqwPZV1mCviRhN5E3rRqYNXVcrnXAAuhBpvNyj/ntHvcDN 2/m+ochiuBYueU4kX3lHya7sOj+mTsndcWmQ9soOUyr8O0r/BG088bMn4qqtUw4dl5/pglXk jbl7uOOPinKf0WVd2r6M0wLPJCD4NPHrCWRLLLAjwfjrtoSRvXxDbXhCdgGBa72+K8eYLzVs hgq7tJOoBWzjVK6XRxR7AAMGB/9Mo3iJ2DxqDecd02KCB5BsFDICbJGhPltU7FwrtbC7djSb XUrwsEVLHi4st4cbdGNCWCrp0BRezXZKohKnNAPFOTK++ZfgeKxrV2sJod+Q9RILF86tQ4XF 7A7Yme5hy92t/WgiU4vc/fWbgP8gV/19f8nunaT2E9NSa70mZFjZNu4iuwThoUUO5CV3Wo0Y UISsnRK8XD1+LR3A2qVyLiFRwh/miC1hgLFCTGCQ3GLxZeZzIpYSlGdQJ0L5lixW5ZQD9r1I 8i/8zhE6qRFAM0upUMI3Gt1Oq2w03DiXrZU0Fu/R8Rm8rlnkQKA+95mRTUq1xL5P5NZIi4gJ Z569OPMFiEkEGBECAAkFAklrAAcCGwwACgkQOM7KaQxqam41igCfbaldnFTu5uAdrnrghESv EI3CAo8AoLkNMks1pThl2BJNRm4CtTK9xZeH Message-ID: <010001748ea604b7-67a61380-0272-4a14-94e3-e2d80f02c763-000000@email.amazonses.com> Date: Mon, 14 Sep 2020 22:04:39 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2020.09.14-54.240.8.237 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-Rspamd-Queue-Id: 4Br0lD4Nfjz4P59 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tarsnap.com header.s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn header.b=mH691y0h; dkim=pass header.d=amazonses.com header.s=224i4yxa5dv7c2xz3womw6peuasteono header.b=LqQ9KVkz; dmarc=pass (policy=none) header.from=tarsnap.com; spf=pass (mx1.freebsd.org: domain of 010001748ea604b7-67a61380-0272-4a14-94e3-e2d80f02c763-000000@amazonses.com designates 54.240.8.237 as permitted sender) smtp.mailfrom=010001748ea604b7-67a61380-0272-4a14-94e3-e2d80f02c763-000000@amazonses.com X-Spamd-Result: default: False [-0.54 / 15.00]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; FORGED_SENDER(0.30)[cperciva@tarsnap.com,010001748ea604b7-67a61380-0272-4a14-94e3-e2d80f02c763-000000@amazonses.com]; R_DKIM_ALLOW(-0.20)[tarsnap.com:s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn,amazonses.com:s=224i4yxa5dv7c2xz3womw6peuasteono]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18:c]; NEURAL_HAM_LONG(-1.00)[-0.999]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_MEDIUM(-0.98)[-0.977]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[tarsnap.com:+,amazonses.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.240.8.237:from]; NEURAL_HAM_SHORT(-0.36)[-0.359]; DMARC_POLICY_ALLOW(-0.50)[tarsnap.com,none]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; RCVD_COUNT_ZERO(0.00)[0]; RWL_MAILSPIKE_POSSIBLE(0.00)[54.240.8.237:from]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[cperciva@tarsnap.com,010001748ea604b7-67a61380-0272-4a14-94e3-e2d80f02c763-000000@amazonses.com]; MAILMAN_DEST(0.00)[freebsd-cloud] X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Sep 2020 22:04:41 -0000 On 2020-09-14 08:35, Riccardo Lombardi wrote: > First of all, thank you for all your efforts for making FreeBSD usable in > EC2 environments, and sorry for my english. Your English seems fine to me! > I'm mainly a hobby web developer, with limited (end-user) experience in > Unix systems. > I recently switched to FreeBSD for my web server and noticed that the > "out-of-the-box" FreeBSD 12 AMI in AWS uses a lot of disk space (3.8Gb > using the df -h command) in comparison with other Linux distributions, > which usually start with 1-1.5Gb disk space. > I also wanted to point out that the size of the FreeBSD 11 AMI is strangely > quite smaller (2.4G). > The largest directories I could find in FreeBSD 12 are: > /usr/lib/debug: 1.5G > /var/db/freebsd-update: 508M You're seeing two things here: 1. FreeBSD 12 has far more space used for debug symbols than FreeBSD 11 (and FreeBSD 13 has even more -- over 2 GB). 2. When you launched FreeBSD 12 it downloaded updates -- including kernel updates and the associated updated debug symbols. Assuming the FreeBSD 11 you launched was 11.4, it didn't have nearly as many updates to download. > My aim (and I suppose this could be useful for many developers and EC2 > users) was originally to shrink the AMI size by eliminating things that are > not necessary in a production environment (maybe the kernel-debug tools). > Unfortunately I don't think i have enough knowledge to make my own custom > light AMI. > > So, if this could be useful for the community, and I think so, it would be > very nice to see some of the following things / features: > - First of all check it there's some problem with the disk size of the > FreeBSD 12 (compared with the 11, maybe there are old unused libraries from > the update), and eventually publish a new, "clean", AMI > - OR/AND maybe implement a "lightweight" version of the FreeBSD AMI, > without some non essential components. I'm seriously considering providing "minimal" AMIs without debug symbols (and maybe other stuff; I'm not sure if there's anything else worth excluding). Right now this and other "flavoured FreeBSDs" is blocked by Amazon not enabling the SSM Parameter Store for FreeBSD to register AMIs. > - Reduce the minimum EBS volume size from the current 10G to 8 or maybe 5 > (accordingly with the final size of the new AMI) That is already done -- while we have 10 GB as the *default* size, you can launch FreeBSD instances with any size as long as it's large enough for the disk image (between 3-5 GB depending on version). ... with the exception of the AWS Marketplace, because they couldn't figure out how to copy our disk snapshots properly, that is. Nothing I can do to fix them. :-/ -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-cloud@freebsd.org Tue Sep 15 21:31:37 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0EA313E085E for ; Tue, 15 Sep 2020 21:31:37 +0000 (UTC) (envelope-from erobison@gmail.com) Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Brbyb6W87z3S6m for ; Tue, 15 Sep 2020 21:31:35 +0000 (UTC) (envelope-from erobison@gmail.com) Received: by mail-wr1-x436.google.com with SMTP id w5so4774841wrp.8 for ; Tue, 15 Sep 2020 14:31:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=lWkzxdSUkUjYFZNUHHDx+8cNSrQYp+3AP22qqItK1Og=; b=dWTATV4n07IsZQg2C1TR1ASMseSQInn82YDrzj1xNtwGELZz/Ew4sGNa3or6ifJjFn 42yNZtY9Tg8vK9+4cKn2fwd/tTtsZcJrGJ3UCcAEvDOv7nxsP7tN5Lqrz4g0zZCX6uPl ruvrSu6Tki66achf0UZPSuHfUJ9LM26EZvOecVZKAjd1NCyLJJGdgM+1Zzkp7FQR3Nlz rtG633IulOf4RIy4YJqd48QjUUECf1zf+YcbvevzOGJrc7/8dKPvG0mFJNOFXQiblGX+ V3vXjV6m7EUiX40ClYpKFSUlvKuwxWEZnK9iUlHxj0/MyvF30URTKARj6t7jeX298RL/ faXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=lWkzxdSUkUjYFZNUHHDx+8cNSrQYp+3AP22qqItK1Og=; b=nI0ScTyQwTLhwfJ+Njz2UFSVwg767AepYQiEw0Qu+4v7IFehOJBzJA1H0Hcy1hkjCx S8rqZtv8k0/vXo664/XeDvYxIbLWgxyBcnIaZWR96zfMlmc4AjVHe01/aXSB2AUFq7bO OqxcskZMYWP22uKI9NYW2ruEbY/2HJ6QxxRMc2IIOxSr2ytHdHskf0lIhD4RSFuprucU GraX6pq1CcqteQEcn0h8wg6DX7t11V8hzmxcr9M2yuOCx+I6fZ44he+21YmNb3NZBr/Z eFAhbef8ib7wNBXIqRq7RJ10MjotObR+1gsG8E3DEBA/tjPIB8ECbUS4qDCMnvRpcPJA j24Q== X-Gm-Message-State: AOAM5318MIFztZjEnWtnhIP6R9AvM71zuUx/i/JmDObxoR6ud7OqzRZ7 ifyudWH1T3IZRWRZaXAE0cpP04XuZ2+ihN2A0DGk+wuocQU= X-Google-Smtp-Source: ABdhPJyeehrnGx5QjBOTnHitY08W3CPsF54sxV6x4ggv4ilQk99GyOQpcoMwc1IDKx/e0XUgDga7BFmC9o/WO/h9bCE= X-Received: by 2002:a5d:6547:: with SMTP id z7mr23176372wrv.322.1600205493910; Tue, 15 Sep 2020 14:31:33 -0700 (PDT) MIME-Version: 1.0 From: ericr Date: Tue, 15 Sep 2020 15:30:57 -0600 Message-ID: Subject: filesystem checksum problems on AWS EC2 instances To: freebsd-cloud@freebsd.org X-Rspamd-Queue-Id: 4Brbyb6W87z3S6m X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=dWTATV4n; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of erobison@gmail.com designates 2a00:1450:4864:20::436 as permitted sender) smtp.mailfrom=erobison@gmail.com X-Spamd-Result: default: False [-2.64 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.87)[-0.865]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.98)[-0.984]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::436:from]; NEURAL_SPAM_SHORT(0.21)[0.206]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-cloud]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Sep 2020 21:31:37 -0000 Hi, I'm having disk/filesystem problems creating EC2 instances using 12.1-RELEASE (AMI ami-034de978f5d9c9b1b in us-east-1 and AMI ami-06d9854b88cba8200 in us-east-2, both from the Marketplace). I'm using t2.large instances. As a part of the instance creation, the OS gets updated from 12.1-RELEASE to 12.1-RELEASE-P9. THe AMIs use UFS. I created instances with a 200GB disk, in both us-east-1 and us-east-2, to rule out flakey hardware. Once the OS starts at first boot, I get cylinder checksum errors: Sep 1 20:50:15 freebsd kernel: xbd0: attaching as ada0 Sep 1 20:50:15 freebsd kernel: GEOM: ada0: the secondary GPT header is not in the last LBA. Sep 1 20:50:15 freebsd kernel: Trying to mount root from ufs:/dev/gpt/rootfs [rw]... Sep 1 20:50:15 freebsd kernel: random: unblocking device. Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) cylinder checksum failed: cg 0, cgp: 0x9c14700e != bp: 0x27bfa3d0 ... Sep 1 20:50:15 freebsd kernel: device_attach: intsmb0 attach returned 6 Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) cylinder checksum failed: cg 0, cgp: 0x9c14700e != bp: 0x27bfa3d0 Sep 1 20:50:15 freebsd syslogd: last message repeated 1 times Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) cylinder checksum failed: cg 7, cgp: 0x43ed3fa1 != bp: 0xe9b0182e and from there on, I get cylinder checksum errors pretty often. Forcing fsck to run at the next boot produces messages about the cylinder groups: ** Phase 5 - Check Cyl groups CG 0: BAD CHECK-HASH 0x9c14700e vs 0x27bfa3d0 FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? no SUMMARY INFORMATION BAD SALVAGE? no BLK(S) MISSING IN BIT MAPS SALVAGE? no CG 7: BAD CHECK-HASH 0x43ed3fa1 vs 0xe9b0182e 43494 files, 1012718 used, 49736354 free (8570 frags, 6215973 blocks, 0.0% fragmentation) Running fsck multiple times at boot doesn't seem to fix the problems. (I edited /etc/rc.d/fsck so that anytime fsck runs, it uses "fsck -f -d -T ffs:-R -T ufs:-R" ) I've also tried creating instances with 100GB disks, I get similar issues. tunefs didn't show any obvious issues: tunefs: POSIX.1e ACLs: (-a) disabled tunefs: NFSv4 ACLs: (-N) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) enabled tunefs: soft update journaling: (-j) disabled tunefs: gjournal: (-J) disabled tunefs: trim: (-t) disabled tunefs: maximum blocks per file in a cylinder group: (-e) 4096 tunefs: average file size: (-f) 16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: space to hold for metadata blocks: (-k) 6408 tunefs: optimization preference: (-o) time tunefs: volume label: (-L) rootfs I can replicate the problem pretty easily, I've created instances half a dozen times now, they all have the same issue. So I'm stumped. I looked in the bug database and the list archives and didn't find anything like this. I can't figure out if it's a problem with the way AWS creates the disk, or if it's an installer issue with FreeBSD. (or if I'm just screwing up the creation myself) I know the default disk size is 10GB, and growfs doesn't seem to complain when it increases the size to 100 or 200GB. Is there some size limit on EBS disks? Also, does anyone have docs on how to create a custom AMI that'll create a ZFS root filesystem and install the OS to it? I can't seem to find AMIs for 12.1-RELEASE that use it. Thanks! From owner-freebsd-cloud@freebsd.org Wed Sep 16 00:24:53 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7AB4C3E8427 for ; Wed, 16 Sep 2020 00:24:53 +0000 (UTC) (envelope-from 01000174944cbdce-21cd3935-fb0f-4c8f-9cba-4783f346a23d-000000@amazonses.com) Received: from a8-52.smtp-out.amazonses.com (a8-52.smtp-out.amazonses.com [54.240.8.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BrgpX4V2Qz3yX3 for ; Wed, 16 Sep 2020 00:24:52 +0000 (UTC) (envelope-from 01000174944cbdce-21cd3935-fb0f-4c8f-9cba-4783f346a23d-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn; d=tarsnap.com; t=1600215891; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=zf+t83AdIAAcM+iC0D2U8mBpH+rpIKsWvWZSDKk604E=; b=okzz70uibRjQ5N56yOnT9B1EbfvEs2dRUFyeBcnMSqrWBkgNB2zvAQyIC9wLw75K kRH/2J0zC5kaI/ebT1a9btkc2bpnJxDE4oG7PBeMDsxcRi+yq4d2s8l1b2ALbOO6jls LNM8tdGOTpIDo5lCHaBIEuAP6te+9T7M2H/H6Y6E= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1600215891; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=zf+t83AdIAAcM+iC0D2U8mBpH+rpIKsWvWZSDKk604E=; b=RS8AeNeE15/OFZWQctzE8LKQEEZIqH5B8T2DJItW96LIRC8zs+i9VdohV/QErd1c dJUTjqShnXPp7bb+25jtnsLrSD4XcxCF6n4JK7FjXbsPw0n80IKn4Ko/4FIuk47QTDc Xj74BLqz6QhbKU1eLz/2YeFj5HASM/0B7x1gohaA= Subject: Re: filesystem checksum problems on AWS EC2 instances To: ericr , freebsd-cloud@freebsd.org References: From: Colin Percival Autocrypt: addr=cperciva@tarsnap.com; prefer-encrypt=mutual; keydata= mQGhBElrAAcRBACDfDys4ZtK+ErCJ1HAzYeteKpm3OEsvT/49AjUTLihkF79HhIKrCQU+1KC zv7BwHCMLb6hq30As9L7iFKG7n5QFLFC4Te/VcITUnWHMG/c3ViLOfJGvi+9/nOEHaM1dVJY D6tEp5yM1nHmVQpo9932j4KGuGFR0LhOK5IHXOSfGwCgxSFDPdgxe2OEjWxjGgY+oV3EafcD +JROXCTjlcQiG/OguQH4Vks3mhHfFnEppLxTkDuYgHZQiUtpcT9ssH5khgqoTyMar05OUdAj ZIhNbWDh4LgTj+7ZmvLhXT5Zxw8LX9d7T36aTB8XDQSenDqEtinMWOb0TCBBLbsB8EFG1WTT ESbZci9jJS5yhtktuZoY/eM8uXMD/3k4FWFO80VRRkELSp+XSy/VlSQjyi/rhl2nQq/oOA9F oJbDaB0yq9VNhxP+uFBzBWSqeIX0t1ZWLtNfVFr4TRP5hihI5ICrg/0OpqgisKsU2NFe9xyO hyJLYmfD8ebpDJ/9k30C7Iju9pVrwLm1QgS4S2fqJRcR+U4WbjvP7CgStCVDb2xpbiBQZXJj aXZhbCA8Y3BlcmNpdmFAdGFyc25hcC5jb20+iGEEExECACEFAklrALYCGwMHCwkIBwMCAQQV AggDBBYCAwECHgECF4AACgkQOM7KaQxqam6/igCgn+z2k3V5ggNppmWrZstt1U2lugsAoL7L wS9V9yLtil3oWmHtwpUqYruEuQINBElrAAcQCAD3ZLMIsP4CIDoJORg+YY0lqLVBgcnF7pFb 4Uy2+KvdWofN+DKH61rZLjgXXkNE9M4EQC1B4lGttBP8IY2gs41y3AUogGdyFbidq99rCBz7 LTsgARHwFxZoaHmXyiZLEU1QZuMqwPZV1mCviRhN5E3rRqYNXVcrnXAAuhBpvNyj/ntHvcDN 2/m+ochiuBYueU4kX3lHya7sOj+mTsndcWmQ9soOUyr8O0r/BG088bMn4qqtUw4dl5/pglXk jbl7uOOPinKf0WVd2r6M0wLPJCD4NPHrCWRLLLAjwfjrtoSRvXxDbXhCdgGBa72+K8eYLzVs hgq7tJOoBWzjVK6XRxR7AAMGB/9Mo3iJ2DxqDecd02KCB5BsFDICbJGhPltU7FwrtbC7djSb XUrwsEVLHi4st4cbdGNCWCrp0BRezXZKohKnNAPFOTK++ZfgeKxrV2sJod+Q9RILF86tQ4XF 7A7Yme5hy92t/WgiU4vc/fWbgP8gV/19f8nunaT2E9NSa70mZFjZNu4iuwThoUUO5CV3Wo0Y UISsnRK8XD1+LR3A2qVyLiFRwh/miC1hgLFCTGCQ3GLxZeZzIpYSlGdQJ0L5lixW5ZQD9r1I 8i/8zhE6qRFAM0upUMI3Gt1Oq2w03DiXrZU0Fu/R8Rm8rlnkQKA+95mRTUq1xL5P5NZIi4gJ Z569OPMFiEkEGBECAAkFAklrAAcCGwwACgkQOM7KaQxqam41igCfbaldnFTu5uAdrnrghESv EI3CAo8AoLkNMks1pThl2BJNRm4CtTK9xZeH Message-ID: <01000174944cbdce-21cd3935-fb0f-4c8f-9cba-4783f346a23d-000000@email.amazonses.com> Date: Wed, 16 Sep 2020 00:24:51 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2020.09.16-54.240.8.52 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-Rspamd-Queue-Id: 4BrgpX4V2Qz3yX3 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tarsnap.com header.s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn header.b=okzz70ui; dkim=pass header.d=amazonses.com header.s=224i4yxa5dv7c2xz3womw6peuasteono header.b=RS8AeNeE; dmarc=pass (policy=none) header.from=tarsnap.com; spf=pass (mx1.freebsd.org: domain of 01000174944cbdce-21cd3935-fb0f-4c8f-9cba-4783f346a23d-000000@amazonses.com designates 54.240.8.52 as permitted sender) smtp.mailfrom=01000174944cbdce-21cd3935-fb0f-4c8f-9cba-4783f346a23d-000000@amazonses.com X-Spamd-Result: default: False [-1.01 / 15.00]; ARC_NA(0.00)[]; FORGED_SENDER(0.30)[cperciva@tarsnap.com,01000174944cbdce-21cd3935-fb0f-4c8f-9cba-4783f346a23d-000000@amazonses.com]; R_DKIM_ALLOW(-0.20)[tarsnap.com:s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn,amazonses.com:s=224i4yxa5dv7c2xz3womw6peuasteono]; NEURAL_HAM_MEDIUM(-0.98)[-0.976]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18]; NEURAL_HAM_LONG(-1.01)[-1.010]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[tarsnap.com:+,amazonses.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.240.8.52:from]; NEURAL_HAM_SHORT(-0.82)[-0.822]; DMARC_POLICY_ALLOW(-0.50)[tarsnap.com,none]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; RCVD_COUNT_ZERO(0.00)[0]; RWL_MAILSPIKE_POSSIBLE(0.00)[54.240.8.52:from]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; FROM_NEQ_ENVFROM(0.00)[cperciva@tarsnap.com,01000174944cbdce-21cd3935-fb0f-4c8f-9cba-4783f346a23d-000000@amazonses.com]; MAILMAN_DEST(0.00)[freebsd-cloud] X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Sep 2020 00:24:53 -0000 On 2020-09-15 14:30, ericr wrote: > I'm having disk/filesystem problems creating EC2 instances using > 12.1-RELEASE (AMI ami-034de978f5d9c9b1b in us-east-1 and AMI > ami-06d9854b88cba8200 in us-east-2, both from the Marketplace). I'm using > t2.large instances. As a part of the instance creation, the OS gets > updated from 12.1-RELEASE to 12.1-RELEASE-P9. THe AMIs use UFS. > > I created instances with a 200GB disk, in both us-east-1 and us-east-2, to > rule out flakey hardware. Once the OS starts at first boot, I get cylinder > checksum errors: > [...] > Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) > cylinder checksum failed: cg 0, cgp: 0x9c14700e != bp: 0x27bfa3d0 > Sep 1 20:50:15 freebsd syslogd: last message repeated 1 times > Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) > cylinder checksum failed: cg 7, cgp: 0x43ed3fa1 != bp: 0xe9b0182e > > and from there on, I get cylinder checksum errors pretty often. Do you get this if you launch from the non-Marketplace AMIs listed in the release announcement? https://www.freebsd.org/releases/12.1R/announce.html Theoretically they should be bit-for-bit identical, but I wouldn't put it past the Marketplace people to have somehow mangled it when they created their copy of the AMI disk image. > Also, does anyone have docs on how to create a custom AMI that'll create a > ZFS root filesystem and install the OS to it? I can't seem to find AMIs > for 12.1-RELEASE that use it. I created some ZFS images for FreeBSD 12.0 last year -- you could launch one of those and upgrade: https://lists.freebsd.org/pipermail/freebsd-cloud/2019-February/000200.html I've been meaning to create ZFS images for 12.1 but haven't had time (this isn't my day job, although I do get a few dollars from a Patreon). At some point I'll add code to the src tree so this and other "additional flavours" get built as part of the standard release engineering process. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-cloud@freebsd.org Wed Sep 16 22:15:44 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C91963F3321 for ; Wed, 16 Sep 2020 22:15:44 +0000 (UTC) (envelope-from erobison@gmail.com) Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BsDv31V42z4MWp for ; Wed, 16 Sep 2020 22:15:42 +0000 (UTC) (envelope-from erobison@gmail.com) Received: by mail-wm1-x330.google.com with SMTP id k18so86523wmj.5 for ; Wed, 16 Sep 2020 15:15:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=iPVvouXPmZ3dTG1p5qHrWOLqq+2zeCGPQuobqae97Os=; b=eiiDa1AgVfoP7sVFw27zKjSo7K0dnSTUDEfXBClkKiLCdZqXnJCO4hliLbNULr6e/W 2ZNqLC2VFs3Ri3QK9w8aTzpFAhEbrRzBPTj0n5fwA4eVCJ8uowMfwrhcwLTcUGT5eCDl cvy/WV6ZFLNFMcAVv98fUxPQFLhWSVfVzCu4VT7kJsqiRKzuXwOLoak7sVvmfNq/fqfZ OXJVkMhGWzxEIHaIuLGc33TOyM9LnAuq0O4tF2aTJkaBhVmmovXzcLyQ9bVlNL/vWht2 JEad2PoAldgs3S6rjeijC3UUNLemODMSiLFPchIfTCPljYVWR61N8++16kpzyZfDyurK R/Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=iPVvouXPmZ3dTG1p5qHrWOLqq+2zeCGPQuobqae97Os=; b=CIWG/FD71/poFU+v96vRRM7+YEyP/LSp/Znn5QEKLpJs3InAUPvqrQ+lfD9AvxKsjX docNDjMAFxgDO2TvxC/XUTASF+AQ31ABZisAEgc4xIfSHT7nghkDTReZcSiCE95t1PuG ZWg/slGMh61uJY5L0GoGYWPgFCPzFVetqjFWg1JMbkLCF/mS0gkvrgMJ1lP2xgTon2Iv 64nVkaT0owSBeoUtjFAmIzrM9apFJ5Y3jyWWIIP9Ds/gBtBDbfmN7W8sPRl09r/JkGlM S/cna8BcdayfeKExh+KzGjaC1FfZJc6juFnyKr2fugo2zZWc5AviVULqDGppatdQCMCx soVA== X-Gm-Message-State: AOAM5322/SrA9C2c92ER8wbGw/AE34loGrV0GsP0PFCecjj7yd+5KQIZ js+f00vuFEWH3vReF1ZEgdQp7O3oloDFLcSGxwOY+LY5Uco= X-Google-Smtp-Source: ABdhPJw5wpkaVv2mPb88UgtNz7XUTuX//TzSMf4TwXfJ8WptuieQRGYhjLamsB3JlFsU0941sdrcd0b4OtEP9Lyt2MI= X-Received: by 2002:a1c:5685:: with SMTP id k127mr7090455wmb.135.1600294541283; Wed, 16 Sep 2020 15:15:41 -0700 (PDT) MIME-Version: 1.0 References: <01000174944cbe04-bc3ea16f-b855-4aac-9bcf-1b9c4e30a241-000000@email.amazonses.com> In-Reply-To: <01000174944cbe04-bc3ea16f-b855-4aac-9bcf-1b9c4e30a241-000000@email.amazonses.com> From: ericr Date: Wed, 16 Sep 2020 16:15:02 -0600 Message-ID: Subject: Re: filesystem checksum problems on AWS EC2 instances To: freebsd-cloud@freebsd.org X-Rspamd-Queue-Id: 4BsDv31V42z4MWp X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=eiiDa1Ag; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of erobison@gmail.com designates 2a00:1450:4864:20::330 as permitted sender) smtp.mailfrom=erobison@gmail.com X-Spamd-Result: default: False [-2.67 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-0.95)[-0.947]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.98)[-0.983]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::330:from]; NEURAL_SPAM_SHORT(0.26)[0.259]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-cloud]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Sep 2020 22:15:44 -0000 On Tue, Sep 15, 2020 at 6:24 PM Colin Percival wrote: > On 2020-09-15 14:30, ericr wrote: > ... > > Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) > > cylinder checksum failed: cg 0, cgp: 0x9c14700e != bp: 0x27bfa3d0 > > Sep 1 20:50:15 freebsd syslogd: last message repeated 1 > times > > Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) > > cylinder checksum failed: cg 7, cgp: 0x43ed3fa1 != bp: 0xe9b0182e > > > > and from there on, I get cylinder checksum errors pretty often. > > Do you get this if you launch from the non-Marketplace AMIs listed in the > release announcement? > https://www.freebsd.org/releases/12.1R/announce.html Yes. I just tried both of these AMI's from the release notes: us-east-1 region: ami-0de268ac2498ba33d us-east-2 region: ami-0a44f10b2c6deb365 I got the same errors. I haven't tried the 12.2-BETA images yet, I suppose I should. Theoretically they should be bit-for-bit identical, but I wouldn't put it > past the Marketplace people to have somehow mangled it when they created > their copy of the AMI disk image. > Seems odd that it happens with both the AMIs from Release Engineering and the Marketplace. I think the only change I'm making that would affect the instance is that I change the disk size to 200GB. It seems pretty idiot proof, but maybe there's something I'm missing. > > Also, does anyone have docs on how to create a custom AMI that'll create > a > > ZFS root filesystem and install the OS to it? I can't seem to find AMIs > > for 12.1-RELEASE that use it. > > I created some ZFS images for FreeBSD 12.0 last year -- you could launch > one > of those and upgrade: > > https://lists.freebsd.org/pipermail/freebsd-cloud/2019-February/000200.html Thanks for the pointer! -- ericr From owner-freebsd-cloud@freebsd.org Thu Sep 17 22:22:44 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7C1613EF46D for ; Thu, 17 Sep 2020 22:22:44 +0000 (UTC) (envelope-from ric.lombardi@gmail.com) Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bss0g4Sm9z3Ssv for ; Thu, 17 Sep 2020 22:22:43 +0000 (UTC) (envelope-from ric.lombardi@gmail.com) Received: by mail-oi1-x22a.google.com with SMTP id n2so4445791oij.1 for ; Thu, 17 Sep 2020 15:22:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=doQOGoVdY6E44jiFxNgz30mw7Xzv19XweALWB99zOTY=; b=TtzRB9wSLsPgFVwV+Edu65fb04yB/E4R0lQ9L0gZbl7mY/38SlxYkBy7mle+pomsmc 0WeKF8v5u1Ka2/hT+u8Dh0PjTIRyBgIkVBnreHE68ultBoO/nCMd/RQj6W7uOKboTxBO mmHh3FW+TUGIkvXf6eHAfXA7zqHI5kjot/XnBMDj6ufCWLu9lU9DQMfvDPlPzp2hU40y kTntg1mMsWiCzwhvKBW3eoToz+cTdpPfqgatpkCq69e3ryeIHpYs4hm1bVX+4tjVZ8FF LuPzJidNfchMLdg9xn6UGzr4rvMVwo63xWGL4ZNmZKk8nP95F37prgtQPS3SWlPAIQ7B 7CfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=doQOGoVdY6E44jiFxNgz30mw7Xzv19XweALWB99zOTY=; b=JYgIuf0OFKfFqvERZSwmd7WeQG+jOfl8EMMgtVhehAPJdFaPiEcjlM1yeFDsaLFptW aODn0pemDcOCw4WZeFli7Ve+UZ4MFXgm+tWDczJIBIh5B0Se91+yOKH6gaSdHv1fmtHD h7IsUccUunOO4NjM4pdV8W1eehcQI0P7Bj6CU2xzui+kcHAsLfdYQTBCx8wNPpCR+ftI ZuEQ8VaPWMlBt6IGtKJCS4CK4BTIUj4UXgCvtEY5tbMmajQcHhTu+zR8UkCajoHhKrKU Gi02eEexRZQOkkFCEhJ3DFP3s910psbBSbuTe6iWTdx2hub8/Jc14E3NVohWovoUvn4H /qGA== X-Gm-Message-State: AOAM533OnPDV3iRK0zBUbdDbgcOX0Nrc0IqMSWQ5a74qYA06xQf8x387 aZTE9se0iCuDrsUgKWRWBl1+emwwlKuiP/JEj4NAzFDfoykhzg== X-Google-Smtp-Source: ABdhPJxv0kX/Su1BdIsnbdbaLuDt/cERppGXaTOHToGlzVqwAaZbmDzuNj5UknrNXigQONcmFU4vuA88Lhu+xph9fgc= X-Received: by 2002:aca:4e03:: with SMTP id c3mr7155030oib.169.1600381362329; Thu, 17 Sep 2020 15:22:42 -0700 (PDT) MIME-Version: 1.0 References: <010001748ea604b7-3edaf425-d677-4a88-90fd-791d869c9a20-000000@email.amazonses.com> In-Reply-To: <010001748ea604b7-3edaf425-d677-4a88-90fd-791d869c9a20-000000@email.amazonses.com> From: Riccardo Lombardi Date: Fri, 18 Sep 2020 00:22:06 +0200 Message-ID: Subject: Re: EC2 AMI size of Freebsd 12 (and maybe a feature request) To: Colin Percival Cc: freebsd-cloud@freebsd.org X-Rspamd-Queue-Id: 4Bss0g4Sm9z3Ssv X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=TtzRB9wS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of riclombardi@gmail.com designates 2607:f8b0:4864:20::22a as permitted sender) smtp.mailfrom=riclombardi@gmail.com X-Spamd-Result: default: False [-3.31 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.02)[-1.024]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; NEURAL_HAM_LONG(-1.04)[-1.039]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22a:from]; NEURAL_HAM_SHORT(-0.25)[-0.247]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-cloud]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Sep 2020 22:22:44 -0000 El mar., 15 sept. 2020 a las 0:04, Colin Percival () escribi=C3=B3: > On 2020-09-14 08:35, Riccardo Lombardi wrote: > > First of all, thank you for all your efforts for making FreeBSD usable = in > > EC2 environments, and sorry for my english. > > Your English seems fine to me! > Thanks! > > > I'm mainly a hobby web developer, with limited (end-user) experience in > > Unix systems. > > I recently switched to FreeBSD for my web server and noticed that the > > "out-of-the-box" FreeBSD 12 AMI in AWS uses a lot of disk space (3.8Gb > > using the df -h command) in comparison with other Linux distributions, > > which usually start with 1-1.5Gb disk space. > > I also wanted to point out that the size of the FreeBSD 11 AMI is > strangely > > quite smaller (2.4G). > > The largest directories I could find in FreeBSD 12 are: > > /usr/lib/debug: 1.5G > > /var/db/freebsd-update: 508M > > You're seeing two things here: > 1. FreeBSD 12 has far more space used for debug symbols than FreeBSD 11 > (and > FreeBSD 13 has even more -- over 2 GB). > 2. When you launched FreeBSD 12 it downloaded updates -- including kernel > updates and the associated updated debug symbols. Assuming the FreeBSD 1= 1 > you launched was 11.4, it didn't have nearly as many updates to download. > Yes, this is probably what happened, since I tried it a few weeks ago, these should be the last versions. > > > My aim (and I suppose this could be useful for many developers and EC2 > > users) was originally to shrink the AMI size by eliminating things that > are > > not necessary in a production environment (maybe the kernel-debug tools= ). > > Unfortunately I don't think i have enough knowledge to make my own cust= om > > light AMI. > > > > So, if this could be useful for the community, and I think so, it would > be > > very nice to see some of the following things / features: > > - First of all check it there's some problem with the disk size of the > > FreeBSD 12 (compared with the 11, maybe there are old unused libraries > from > > the update), and eventually publish a new, "clean", AMI > > - OR/AND maybe implement a "lightweight" version of the FreeBSD AMI, > > without some non essential components. > > I'm seriously considering providing "minimal" AMIs without debug symbols > (and > maybe other stuff; I'm not sure if there's anything else worth excluding)= . > Right now this and other "flavoured FreeBSDs" is blocked by Amazon not > enabling the SSM Parameter Store for FreeBSD to register AMIs. > > Thank you, this is really great news. May I suggest, if Amazon's reaction time is still too slow or tending to infinity, the "CentOS" way. I don't know the actual reason, but their AMIs are not listed on the official Marketplace, they just publish the AMI-IDs on their website, just like any community AMI. Thanks again and best regards. > > > -- > Colin Percival > Security Officer Emeritus, FreeBSD | The power to serve > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoi= d > From owner-freebsd-cloud@freebsd.org Fri Sep 18 20:04:49 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1201A3EDBF2 for ; Fri, 18 Sep 2020 20:04:49 +0000 (UTC) (envelope-from 01000174a2d1ae7d-67d1be08-42fe-48a1-aa45-c707d264f01b-000000@amazonses.com) Received: from a8-52.smtp-out.amazonses.com (a8-52.smtp-out.amazonses.com [54.240.8.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BtPv36V19z437n for ; Fri, 18 Sep 2020 20:04:47 +0000 (UTC) (envelope-from 01000174a2d1ae7d-67d1be08-42fe-48a1-aa45-c707d264f01b-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn; d=tarsnap.com; t=1600459485; h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=xMXE+VwI5zsTTftuq0sL1gUr8ciMJ21rxZkc37xfqy4=; b=ble/N5vyryiS29Dt3MktUJrMQmgNER9R6+yUWpDE9yMt9C3ezmDArSI9HmqS2KUY RZYWlru/+c3wZjCGANvx5Fo58lrXUrcTzHnI0b5ZYIbxQ11wTZyhIn9VHdeUXjeN3Et r19CKSh6THVMwCCHms5YAgrw9RovIhiJ38TibRjA= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1600459485; h=Subject:To:Cc:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=xMXE+VwI5zsTTftuq0sL1gUr8ciMJ21rxZkc37xfqy4=; b=e3c+rW2oFjBVOW4CgvdQsU8dmO24fPoXUPrWA5e9ve/dEWKQn3gmRaMLPyGX27uJ KDwd6tKDZMeiAg080HCY0S4lL8LnwggFWNUdGIqohN7GzEhOs43UZZhFJ6fOuNw0w6y znap3mlegqHXMhdu6UtGgsX7SVymFBrDEAqmxczI= Subject: Re: EC2 AMI size of Freebsd 12 (and maybe a feature request) To: Riccardo Lombardi Cc: freebsd-cloud@freebsd.org References: <010001748ea604b7-3edaf425-d677-4a88-90fd-791d869c9a20-000000@email.amazonses.com> From: Colin Percival Autocrypt: addr=cperciva@tarsnap.com; prefer-encrypt=mutual; keydata= mQGhBElrAAcRBACDfDys4ZtK+ErCJ1HAzYeteKpm3OEsvT/49AjUTLihkF79HhIKrCQU+1KC zv7BwHCMLb6hq30As9L7iFKG7n5QFLFC4Te/VcITUnWHMG/c3ViLOfJGvi+9/nOEHaM1dVJY D6tEp5yM1nHmVQpo9932j4KGuGFR0LhOK5IHXOSfGwCgxSFDPdgxe2OEjWxjGgY+oV3EafcD +JROXCTjlcQiG/OguQH4Vks3mhHfFnEppLxTkDuYgHZQiUtpcT9ssH5khgqoTyMar05OUdAj ZIhNbWDh4LgTj+7ZmvLhXT5Zxw8LX9d7T36aTB8XDQSenDqEtinMWOb0TCBBLbsB8EFG1WTT ESbZci9jJS5yhtktuZoY/eM8uXMD/3k4FWFO80VRRkELSp+XSy/VlSQjyi/rhl2nQq/oOA9F oJbDaB0yq9VNhxP+uFBzBWSqeIX0t1ZWLtNfVFr4TRP5hihI5ICrg/0OpqgisKsU2NFe9xyO hyJLYmfD8ebpDJ/9k30C7Iju9pVrwLm1QgS4S2fqJRcR+U4WbjvP7CgStCVDb2xpbiBQZXJj aXZhbCA8Y3BlcmNpdmFAdGFyc25hcC5jb20+iGEEExECACEFAklrALYCGwMHCwkIBwMCAQQV AggDBBYCAwECHgECF4AACgkQOM7KaQxqam6/igCgn+z2k3V5ggNppmWrZstt1U2lugsAoL7L wS9V9yLtil3oWmHtwpUqYruEuQINBElrAAcQCAD3ZLMIsP4CIDoJORg+YY0lqLVBgcnF7pFb 4Uy2+KvdWofN+DKH61rZLjgXXkNE9M4EQC1B4lGttBP8IY2gs41y3AUogGdyFbidq99rCBz7 LTsgARHwFxZoaHmXyiZLEU1QZuMqwPZV1mCviRhN5E3rRqYNXVcrnXAAuhBpvNyj/ntHvcDN 2/m+ochiuBYueU4kX3lHya7sOj+mTsndcWmQ9soOUyr8O0r/BG088bMn4qqtUw4dl5/pglXk jbl7uOOPinKf0WVd2r6M0wLPJCD4NPHrCWRLLLAjwfjrtoSRvXxDbXhCdgGBa72+K8eYLzVs hgq7tJOoBWzjVK6XRxR7AAMGB/9Mo3iJ2DxqDecd02KCB5BsFDICbJGhPltU7FwrtbC7djSb XUrwsEVLHi4st4cbdGNCWCrp0BRezXZKohKnNAPFOTK++ZfgeKxrV2sJod+Q9RILF86tQ4XF 7A7Yme5hy92t/WgiU4vc/fWbgP8gV/19f8nunaT2E9NSa70mZFjZNu4iuwThoUUO5CV3Wo0Y UISsnRK8XD1+LR3A2qVyLiFRwh/miC1hgLFCTGCQ3GLxZeZzIpYSlGdQJ0L5lixW5ZQD9r1I 8i/8zhE6qRFAM0upUMI3Gt1Oq2w03DiXrZU0Fu/R8Rm8rlnkQKA+95mRTUq1xL5P5NZIi4gJ Z569OPMFiEkEGBECAAkFAklrAAcCGwwACgkQOM7KaQxqam41igCfbaldnFTu5uAdrnrghESv EI3CAo8AoLkNMks1pThl2BJNRm4CtTK9xZeH Message-ID: <01000174a2d1ae7d-67d1be08-42fe-48a1-aa45-c707d264f01b-000000@email.amazonses.com> Date: Fri, 18 Sep 2020 20:04:44 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-SES-Outgoing: 2020.09.18-54.240.8.52 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-Rspamd-Queue-Id: 4BtPv36V19z437n X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tarsnap.com header.s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn header.b=ble/N5vy; dkim=pass header.d=amazonses.com header.s=224i4yxa5dv7c2xz3womw6peuasteono header.b=e3c+rW2o; dmarc=pass (policy=none) header.from=tarsnap.com; spf=pass (mx1.freebsd.org: domain of 01000174a2d1ae7d-67d1be08-42fe-48a1-aa45-c707d264f01b-000000@amazonses.com designates 54.240.8.52 as permitted sender) smtp.mailfrom=01000174a2d1ae7d-67d1be08-42fe-48a1-aa45-c707d264f01b-000000@amazonses.com X-Spamd-Result: default: False [-0.78 / 15.00]; ARC_NA(0.00)[]; FORGED_SENDER(0.30)[cperciva@tarsnap.com,01000174a2d1ae7d-67d1be08-42fe-48a1-aa45-c707d264f01b-000000@amazonses.com]; R_DKIM_ALLOW(-0.20)[tarsnap.com:s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn,amazonses.com:s=224i4yxa5dv7c2xz3womw6peuasteono]; NEURAL_HAM_MEDIUM(-0.99)[-0.990]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18:c]; NEURAL_HAM_LONG(-1.00)[-1.001]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[tarsnap.com:+,amazonses.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.240.8.52:from]; NEURAL_HAM_SHORT(-0.59)[-0.586]; DMARC_POLICY_ALLOW(-0.50)[tarsnap.com,none]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; RWL_MAILSPIKE_POSSIBLE(0.00)[54.240.8.52:from]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; FROM_NEQ_ENVFROM(0.00)[cperciva@tarsnap.com,01000174a2d1ae7d-67d1be08-42fe-48a1-aa45-c707d264f01b-000000@amazonses.com]; MAILMAN_DEST(0.00)[freebsd-cloud] X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Sep 2020 20:04:49 -0000 On 2020-09-17 15:22, Riccardo Lombardi wrote: > El mar., 15 sept. 2020 a las 0:04, Colin Percival ( >) escribió: > I'm seriously considering providing "minimal" AMIs without debug symbols (and > maybe other stuff; I'm not sure if there's anything else worth excluding). > Right now this and other "flavoured FreeBSDs" is blocked by Amazon not > enabling the SSM Parameter Store for FreeBSD to register AMIs. > > Thank you, this is really great news.  > May I suggest, if Amazon's reaction time is still too slow or tending to > infinity, the "CentOS" way. > I don't know the actual reason, but their AMIs are not listed on the official > Marketplace, they just publish the AMI-IDs on their website, just like any > community AMI. We do publish AMI IDs on the FreeBSD website, in the release announcements. But it's unwieldy enough including 2 architectures x 17 regions worth of AMI IDs in the announcement emails already; it would be impractical to include 2 architectures x 2 filesystems (UFS vs ZFS) x 4 flavours (minimal, base, cloudinit, AMI Builder) x 17 regions in each announcement. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From owner-freebsd-cloud@freebsd.org Sat Sep 19 03:00:52 2020 Return-Path: Delivered-To: freebsd-cloud@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D67C43F8559 for ; Sat, 19 Sep 2020 03:00:52 +0000 (UTC) (envelope-from 01000174a44ea02e-d511a094-ae7b-4c0c-b017-df25edaddd42-000000@amazonses.com) Received: from a8-60.smtp-out.amazonses.com (a8-60.smtp-out.amazonses.com [54.240.8.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Btb775hGrz4ThD for ; Sat, 19 Sep 2020 03:00:51 +0000 (UTC) (envelope-from 01000174a44ea02e-d511a094-ae7b-4c0c-b017-df25edaddd42-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn; d=tarsnap.com; t=1600484450; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=CnMrB6dydKrpJx8WYJCgOoBfx/zBMqF/tTvBZr3nhhY=; b=b+cJdcBA9V+0EZoYco2D+PMHDksP4d+YR6HYX9oBxry5F9FbsOPWXmj4yv/7rEa8 tc0O3YV3K+8Pp9S71WN1pCwW5rPpoEIs2R/TnC2v2NEXHKDLcfj+TNmhStlxb7B/fOV jrNaLP9IyIm/qz2Ks3QJObmgjleq0Obe9KpYWB7k= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1600484450; h=Subject:To:References:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=CnMrB6dydKrpJx8WYJCgOoBfx/zBMqF/tTvBZr3nhhY=; b=WFsjMq0W8hdO0AnOlaJwKhqUSlVmYLkt19vq7QFe13tFbouzPZkfOSwo3i/Rq+R3 EvJ9sRDqVpQezCtstzq1SsO/IXJRhcqV9e9AvijhRa5ZbzFNcMh2EYB2BaXjOAtvjWr etwscCKju2Z4oPEkXx3lKiON8d507ZWlXnY/C2UA= Subject: Re: filesystem checksum problems on AWS EC2 instances To: ericr , freebsd-cloud@freebsd.org, Kirk McKusick References: <01000174944cbe04-bc3ea16f-b855-4aac-9bcf-1b9c4e30a241-000000@email.amazonses.com> From: Colin Percival Autocrypt: addr=cperciva@tarsnap.com; prefer-encrypt=mutual; keydata= mQGhBElrAAcRBACDfDys4ZtK+ErCJ1HAzYeteKpm3OEsvT/49AjUTLihkF79HhIKrCQU+1KC zv7BwHCMLb6hq30As9L7iFKG7n5QFLFC4Te/VcITUnWHMG/c3ViLOfJGvi+9/nOEHaM1dVJY D6tEp5yM1nHmVQpo9932j4KGuGFR0LhOK5IHXOSfGwCgxSFDPdgxe2OEjWxjGgY+oV3EafcD +JROXCTjlcQiG/OguQH4Vks3mhHfFnEppLxTkDuYgHZQiUtpcT9ssH5khgqoTyMar05OUdAj ZIhNbWDh4LgTj+7ZmvLhXT5Zxw8LX9d7T36aTB8XDQSenDqEtinMWOb0TCBBLbsB8EFG1WTT ESbZci9jJS5yhtktuZoY/eM8uXMD/3k4FWFO80VRRkELSp+XSy/VlSQjyi/rhl2nQq/oOA9F oJbDaB0yq9VNhxP+uFBzBWSqeIX0t1ZWLtNfVFr4TRP5hihI5ICrg/0OpqgisKsU2NFe9xyO hyJLYmfD8ebpDJ/9k30C7Iju9pVrwLm1QgS4S2fqJRcR+U4WbjvP7CgStCVDb2xpbiBQZXJj aXZhbCA8Y3BlcmNpdmFAdGFyc25hcC5jb20+iGEEExECACEFAklrALYCGwMHCwkIBwMCAQQV AggDBBYCAwECHgECF4AACgkQOM7KaQxqam6/igCgn+z2k3V5ggNppmWrZstt1U2lugsAoL7L wS9V9yLtil3oWmHtwpUqYruEuQINBElrAAcQCAD3ZLMIsP4CIDoJORg+YY0lqLVBgcnF7pFb 4Uy2+KvdWofN+DKH61rZLjgXXkNE9M4EQC1B4lGttBP8IY2gs41y3AUogGdyFbidq99rCBz7 LTsgARHwFxZoaHmXyiZLEU1QZuMqwPZV1mCviRhN5E3rRqYNXVcrnXAAuhBpvNyj/ntHvcDN 2/m+ochiuBYueU4kX3lHya7sOj+mTsndcWmQ9soOUyr8O0r/BG088bMn4qqtUw4dl5/pglXk jbl7uOOPinKf0WVd2r6M0wLPJCD4NPHrCWRLLLAjwfjrtoSRvXxDbXhCdgGBa72+K8eYLzVs hgq7tJOoBWzjVK6XRxR7AAMGB/9Mo3iJ2DxqDecd02KCB5BsFDICbJGhPltU7FwrtbC7djSb XUrwsEVLHi4st4cbdGNCWCrp0BRezXZKohKnNAPFOTK++ZfgeKxrV2sJod+Q9RILF86tQ4XF 7A7Yme5hy92t/WgiU4vc/fWbgP8gV/19f8nunaT2E9NSa70mZFjZNu4iuwThoUUO5CV3Wo0Y UISsnRK8XD1+LR3A2qVyLiFRwh/miC1hgLFCTGCQ3GLxZeZzIpYSlGdQJ0L5lixW5ZQD9r1I 8i/8zhE6qRFAM0upUMI3Gt1Oq2w03DiXrZU0Fu/R8Rm8rlnkQKA+95mRTUq1xL5P5NZIi4gJ Z569OPMFiEkEGBECAAkFAklrAAcCGwwACgkQOM7KaQxqam41igCfbaldnFTu5uAdrnrghESv EI3CAo8AoLkNMks1pThl2BJNRm4CtTK9xZeH Message-ID: <01000174a44ea02e-d511a094-ae7b-4c0c-b017-df25edaddd42-000000@email.amazonses.com> Date: Sat, 19 Sep 2020 03:00:50 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-SES-Outgoing: 2020.09.19-54.240.8.60 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-Rspamd-Queue-Id: 4Btb775hGrz4ThD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tarsnap.com header.s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn header.b=b+cJdcBA; dkim=pass header.d=amazonses.com header.s=224i4yxa5dv7c2xz3womw6peuasteono header.b=WFsjMq0W; dmarc=pass (policy=none) header.from=tarsnap.com; spf=pass (mx1.freebsd.org: domain of 01000174a44ea02e-d511a094-ae7b-4c0c-b017-df25edaddd42-000000@amazonses.com designates 54.240.8.60 as permitted sender) smtp.mailfrom=01000174a44ea02e-d511a094-ae7b-4c0c-b017-df25edaddd42-000000@amazonses.com X-Spamd-Result: default: False [-1.19 / 15.00]; ARC_NA(0.00)[]; FORGED_SENDER(0.30)[cperciva@tarsnap.com,01000174a44ea02e-d511a094-ae7b-4c0c-b017-df25edaddd42-000000@amazonses.com]; R_DKIM_ALLOW(-0.20)[tarsnap.com:s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn,amazonses.com:s=224i4yxa5dv7c2xz3womw6peuasteono]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:54.240.0.0/18]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.01)[-1.007]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[tarsnap.com:+,amazonses.com:+]; DMARC_POLICY_ALLOW(-0.50)[tarsnap.com,none]; RCVD_IN_DNSWL_NONE(0.00)[54.240.8.60:from]; NEURAL_HAM_SHORT(-0.98)[-0.980]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org,FreeBSD.org]; RCVD_COUNT_ZERO(0.00)[0]; RWL_MAILSPIKE_POSSIBLE(0.00)[54.240.8.60:from]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:54.240.8.0/21, country:US]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; FROM_NEQ_ENVFROM(0.00)[cperciva@tarsnap.com,01000174a44ea02e-d511a094-ae7b-4c0c-b017-df25edaddd42-000000@amazonses.com]; MAILMAN_DEST(0.00)[freebsd-cloud] X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Sep 2020 03:00:52 -0000 [Adding Kirk since this seems like a UFS issue...] On 2020-09-16 15:15, ericr wrote: > On Tue, Sep 15, 2020 at 6:24 PM Colin Percival wrote: >> On 2020-09-15 14:30, ericr wrote: >>> Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) >>> cylinder checksum failed: cg 0, cgp: 0x9c14700e != bp: 0x27bfa3d0 >>> Sep 1 20:50:15 freebsd syslogd: last message repeated 1 >> times >>> Sep 1 20:50:15 freebsd kernel: UFS /dev/gpt/rootfs (/) >>> cylinder checksum failed: cg 7, cgp: 0x43ed3fa1 != bp: 0xe9b0182e >>> >>> and from there on, I get cylinder checksum errors pretty often. >> >> Do you get this if you launch from the non-Marketplace AMIs listed in the >> release announcement? >> https://www.freebsd.org/releases/12.1R/announce.html > > > Yes. I just tried both of these AMI's from the release notes: > us-east-1 region: ami-0de268ac2498ba33d > us-east-2 region: ami-0a44f10b2c6deb365 > > I got the same errors. I've managed to reproduce this, with a filesystem which I've verified is clean (at least, which passes fsck) before resizing up to ~ 200 GB: > root@freebsd:/usr/home/ec2-user # fsck_ufs /dev/nvd1p2 > ** /dev/nvd1p2 > ** Last Mounted on /releng/12-amd64-GENERIC-release/usr/obj/usr/src/amd64.amd64/release/cw-ec2/new > ** Phase 1 - Check Blocks and Sizes > ** Phase 2 - Check Pathnames > ** Phase 3 - Check Connectivity > ** Phase 4 - Check Reference Counts > ** Phase 5 - Check Cyl groups > 25701 files, 758977 used, 229774 free (9654 frags, 27515 blocks, 1.0% fragmentation) > > ***** FILE SYSTEM IS CLEAN ***** > root@freebsd:/usr/home/ec2-user # gpart recover /dev/nvd1 > nvd1 recovered > root@freebsd:/usr/home/ec2-user # gpart resize -i 2 /dev/nvd1 > nvd1p2 resized > root@freebsd:/usr/home/ec2-user # growfs -y /dev/nvd1p2 > super-block backups (for fsck_ffs -b #) at: > [snip] > root@freebsd:/usr/home/ec2-user # fsck_ufs /dev/nvd1p2 > ** /dev/nvd1p2 > ** Last Mounted on > ** Phase 1 - Check Blocks and Sizes > ** Phase 2 - Check Pathnames > ** Phase 3 - Check Connectivity > ** Phase 4 - Check Reference Counts > ** Phase 5 - Check Cyl groups > CG 0: BAD CHECK-HASH 0x9c14700e vs 0xc9441f74 > SUMMARY INFORMATION BAD > SALVAGE? [yn] n > > CG 7: BAD CHECK-HASH 0xad168305 vs 0x74ba48a > 25701 files, 758977 used, 50019285 free (9661 frags, 6251203 blocks, 0.0% fragmentation) > > ***** FILE SYSTEM MARKED DIRTY ***** > > ***** PLEASE RERUN FSCK ***** This seems like a bug in UFS and/or growfs, but I'm not familiar enough with either to say any more. Kirk, are you aware of any issues on FreeBSD 12.1-RELEASE which can cause cylinder checksum errors after growfs? (On amd64 if it matters.) If it would help I can provide you with SSH access to an affected EC2 instance. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid