From nobody Sun May 23 13:49:49 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2332A9E39E8 for ; Sun, 23 May 2021 13:50:01 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fp1td0Rxlz4ngk for ; Sun, 23 May 2021 13:50:01 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from mail-yb1-f179.google.com (mail-yb1-f179.google.com [209.85.219.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: lwhsu/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id E534A2BC30 for ; Sun, 23 May 2021 13:50:00 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: by mail-yb1-f179.google.com with SMTP id i4so34330745ybe.2 for ; Sun, 23 May 2021 06:50:00 -0700 (PDT) X-Gm-Message-State: AOAM532Ob0sopWe0/f32x4QeORlEzzUlayMLtRCKb+cOSJhV7tQl9zHx CNEsgrybItH3w+fr9ytCLc8boCbiF3LHha7Xquo= X-Google-Smtp-Source: ABdhPJw3PXdhX//0dUk58HutgODteuFjnXCVjucZIE0OmmAA0W+675mLkh04QMQtDUbFccMnKakme4WOQhdigItaLBQ= X-Received: by 2002:a25:18d5:: with SMTP id 204mr27921868yby.497.1621777800177; Sun, 23 May 2021 06:50:00 -0700 (PDT) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 From: Li-Wen Hsu Date: Sun, 23 May 2021 21:49:49 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: sysutils/azure-agent has been updated to 2.2.54.2 To: "freebsd-cloud@FreeBSD.org" Content-Type: text/plain; charset="UTF-8" sysutils/azure-agent has been updated to 2.2.54.2, along with several fixes: https://cgit.freebsd.org/ports/commit/?id=b185a2b866b8642e2500dd691267694ac388875b Please help test if you're using FreeBSD on Azure. I will include this in the (belated) 13.0 Azure marketplace image, sorry that it was delayed because of several technical and non-technical issues. Best, Li-Wen From nobody Sun Aug 29 04:05:32 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 47F3F179C971 for ; Sun, 29 Aug 2021 04:05:40 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from mail.tarsnap.com (mail.tarsnap.com [54.86.246.204]) by mx1.freebsd.org (Postfix) with SMTP id 4Gy0H73QPfz3FMf for ; Sun, 29 Aug 2021 04:05:39 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: (qmail 23576 invoked from network); 29 Aug 2021 04:05:33 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by mail.tarsnap.com with SMTP; 29 Aug 2021 04:05:33 -0000 Received: (qmail 80845 invoked from network); 29 Aug 2021 04:05:32 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by localhost with SMTP; 29 Aug 2021 04:05:32 -0000 To: "freebsd-cloud@freebsd.org" From: Colin Percival Subject: RFC: Switching FreeBSD/EC2 images to UEFI boot Message-ID: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> Date: Sat, 28 Aug 2021 21:05:32 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Gy0H73QPfz3FMf X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:14618, ipnet:54.86.0.0/16, country:US] X-ThisMailContainsUnwantedMimeParts: N Hi Cloudy people, We have a decision to make and I'm looking for input from the community: Should FreeBSD's x86 EC2 AMIs be marked as booting using UEFI instead of the current (default) BIOS boot mode? The argument in favour of sticking with BIOS booting is that the older EC2 instance types -- up to M4/C4/R4/T2 -- don't support UEFI. There's a full list of which instance types support UEFI at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ami-boot.html but roughly speaking it's a case of "Nitro instances support UEFI while Xen based instances require BIOS". (Note that whatever decision we make, it will still be possible to build AMIs which support BIOS booting -- the question is just which mode the official project-published images are marked to use.) The argument in favour of switching to UEFI booting is that it's faster -- around 4.7 seconds faster in my testing. While this may not seem like a big difference, it's the difference between being on par with most Linuxes or lagging far behind; and there's a definite quality-of-life benefit to being able to spin up an EC2 instance and SSH in to start using it quickly. (Also benefits for people using autoscaling, since they can respond to increased load faster.) One obvious question to ask here is "how many people are using the older instance types which only support BIOS", and I have some data from the AWS Marketplace about that: * Instances originally launched on FreeBSD 9.x and 10.x are all using older instance types. * About 60% of instances originally launched on FreeBSD 11.x are using older instance types. * About 40% of instances originally launched on FreeBSD 12.x and 13.0 are using older instance types. Almost all of the "older instance type" usage on FreeBSD 12.x and 13.0 is T2 family instances, which I suspect is for two reasons: 1. For a long time I had t2.micro listed as the default instance type on the AWS Marketplace listings, and 2. FreeBSD 13.0 has problems on T3 instances due to the TSC timecounter not working reliably. This problem has now been corrected in HEAD and I'm hoping the fix will be MFCed in time for FreeBSD 13.1. I don't have any data on instances launched outside of the AWS Marketplace. So, as I started this email saying: We have a decision to make and I'm looking for input from the community. Faster booting, or support for older EC2 instance types in our published AMIs? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From nobody Sun Aug 29 11:30:23 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C634C17A4916 for ; Sun, 29 Aug 2021 11:30:34 +0000 (UTC) (envelope-from raf+NU=eb39f044@rafal.net) Received: from smtp-out-4.mxes.net (smtp-out-4.mxes.net [198.205.123.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GyB8T5q2Xz4l50 for ; Sun, 29 Aug 2021 11:30:33 +0000 (UTC) (envelope-from raf+NU=eb39f044@rafal.net) Received: from Customer-MUA (mua.mxes.net [10.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 4GyB8K4y9Fz3c8d for ; Sun, 29 Aug 2021 07:30:25 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mxes.net; s=mta; t=1630236626; bh=zjAz7mDsHU8x0iFp4wcMa+/+wIoUCA3qE7CUxtDpvto=; h=From:Content-Type:Mime-Version:Subject:Date:References:To: In-Reply-To:Message-Id; b=i/9ANCl/7Sq/qJW7WnqRXNT0+f1jSHYKIgHUBf2yvQRzPVtxiUWsYiaJQ+cjeUZf6 fXMpvqiOOw6VwQksA3CQxVk16lXu1+5ffJO6qmvfJTQqP3cqIzTURyZW/f729auYBo qWLOfA8Ga0taYh5LlUiahckyEZqHJBDXxoH0jOIg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rafal.net; s=tm; t=1630236625; bh=zjAz7mDsHU8x0iFp4wcMa+/+wIoUCA3qE7CUxtDpvto=; l=986; h=From:Subject:Date:References:To:In-Reply-To:From; b=bMORh9SXxm6yzc4bgR1fSV/ph/G4NYPFV53OZqhA490eiTeqnd4cXepGhUJnwrsep /UCzYFj3+8kIVYdCnwoTuKYmmGhR0ijUqBM6AkWkMIrH6HbjSwKCXO2VwoqgKLYeRx 3xhLw9XvNFbEb8hZ2exOmFS1FwvyjQigGq6bVbng= Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: RFC: Switching FreeBSD/EC2 images to UEFI boot Date: Sun, 29 Aug 2021 12:30:23 +0100 References: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> To: "freebsd-cloud@freebsd.org" In-Reply-To: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> Message-Id: <7F854796-C481-4BA7-B729-29E84BAF1856@rafal.net> X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Sent-To: X-Rspamd-Queue-Id: 4GyB8T5q2Xz4l50 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=mxes.net header.s=mta header.b="i/9ANCl/"; dkim=pass header.d=rafal.net header.s=tm header.b=bMORh9SX; dmarc=pass (policy=reject) header.from=rafal.net; spf=pass (mx1.freebsd.org: domain of raf@rafal.net designates 198.205.123.69 as permitted sender) smtp.mailfrom=raf@rafal.net X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[mxes.net:s=mta,rafal.net:s=tm]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[mxes.net:dkim]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_SPF_ALLOW(-0.20)[+ip4:198.205.123.0/25]; DKIM_TRACE(0.00)[mxes.net:+,rafal.net:+]; DMARC_POLICY_ALLOW(-0.50)[rafal.net,reject]; NEURAL_HAM_SHORT(-1.00)[-0.997]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:19844, ipnet:198.205.122.0/23, country:US]; TAGGED_FROM(0.00)[NU=eb39f044]; RCVD_IN_DNSWL_LOW(-0.10)[198.205.123.69:from] Reply-To: raf@rafal.net From: Rafal Lukawiecki via freebsd-cloud X-Original-From: Rafal Lukawiecki X-ThisMailContainsUnwantedMimeParts: N > On 29 Aug 2021, at 05:05, Colin Percival wrote: >=20 > We have a decision to make and I'm > looking for input from the community. Faster booting, or support for = older > EC2 instance types in our published AMIs? >=20 > --=20 > Colin Percival Thanks for asking, Colin. =46rom my perspective, any improvement to boot = times is highly welcome. I am not affected by the older/legacy instance = types. I suspect that in the world of fast-changing cloud provisioning, = and very much unlike in the world of real hardware, there would be less = of a need to maintain support for older platforms. While it is important = for FreeBSD to support older physical hardware, laptops, cards etc, I do = not think that is as important for cloud platforms at all. However, I = fully respect that others may have different perspectives, so please do = not base your decision merely on mine. -- Rafal Lukawiecki Data Scientist=20 Project Botticelli Ltd= From nobody Sun Aug 29 16:59:53 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1AAD217A6966 for ; Sun, 29 Aug 2021 16:59:38 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [66.165.241.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.nomadlogic.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GyKS95KgVz3pRg; Sun, 29 Aug 2021 16:59:37 +0000 (UTC) (envelope-from pete@nomadlogic.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomadlogic.org; s=04242021; t=1630256369; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aHLKkLEV2+GYgBQ2OfOrCI/gFy0anc7hoUEf75e7vH4=; b=d/gzBNOp+vxf6G31tdoldj3g1D7Q88T2B6WbKMxUlZ2yYTyU57yTI/D1m6VdaYmRVNiGbM JPxu0rOUvoDV1cXrZ1Rg+F2n1kOP0XBikU72Lh4xKGU9rgqk/oaQP5Gai3gtMq4tH360m/ NMsEf1J67AaBlzMubqU9EQphfkrQKBE= Received: from [192.168.1.223] (cpe-24-24-163-126.socal.res.rr.com [24.24.163.126]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id 1755139d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sun, 29 Aug 2021 16:59:28 +0000 (UTC) Subject: Re: RFC: Switching FreeBSD/EC2 images to UEFI boot To: Colin Percival , "freebsd-cloud@freebsd.org" References: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> Message-ID: <1e1829ab-bf3d-91f4-d21d-424398a899f7@nomadlogic.org> Date: Sun, 29 Aug 2021 09:59:53 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 In-Reply-To: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4GyKS95KgVz3pRg X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Reply-To: pete@nomadlogic.org From: Pete Wright via freebsd-cloud X-Original-From: Pete Wright X-ThisMailContainsUnwantedMimeParts: N On 8/28/21 9:05 PM, Colin Percival wrote: > Hi Cloudy people, > > We have a decision to make and I'm looking for input from the community: > Should FreeBSD's x86 EC2 AMIs be marked as booting using UEFI instead of > the current (default) BIOS boot mode? I am in favor of making UEFI the default.  faster boot times and more parity with linux would both be wins for me.  having said that I've standardized on Nitro instance types a year or so ago, and use AMD instance types for my BSD systems so there would be no downside for me in particular. i did see on twitter that at least one person noted that free credits only apply to T2 instance class systems, so their may be some downside for people investigating moving to EC2 if they want to leverage the free tier.  this make we wonder, do linux vendors provide support for both BIOS and UEFI enabled systems? Cheers, -pete -- Pete Wright pete@nomadlogic.org @nomadlogicLA From nobody Mon Aug 30 13:49:25 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id EF43817AF8B0 for ; Mon, 30 Aug 2021 13:49:30 +0000 (UTC) (envelope-from james.wright@digital-chaos.com) Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.13]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GysBK2m3gz4bTc for ; Mon, 30 Aug 2021 13:49:29 +0000 (UTC) (envelope-from james.wright@digital-chaos.com) Received: from [192.168.0.12] ([82.29.153.121]) by mrelayeu.kundenserver.de (mreue107 [212.227.15.179]) with ESMTPSA (Nemesis) id 1MpUpW-1mm3jB44qq-00pwA0 for ; Mon, 30 Aug 2021 15:49:22 +0200 Subject: Re: RFC: Switching FreeBSD/EC2 images to UEFI boot To: freebsd-cloud@freebsd.org References: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> From: James Wright Message-ID: <276e84a5-b1b0-a35c-46dd-964c84c7d6e1@digital-chaos.com> Date: Mon, 30 Aug 2021 14:49:25 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 In-Reply-To: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Provags-ID: V03:K1:YeyHawVJ324MTMzb/DxtpIqw0rNX35/M9JXYlbtx6UaEwaW5JoC N0JxcQ2BD7E0+EMlY0ddUcOAqP3MxJZvmMXrKvbJorzRhtOBC4PB73OiY/DPXIiNp6rCwW3 fnNYwBU2dBVE1NFzpfmW9mSGQKeMX/8x9GwbGGCBF2H0VwBKW/qA6jQ192SgvfEteZyoFBz z1JYaA140Eqj9WSW8LiuQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:apREYxlZZZY=:tDS+lJgl4ifkSWfRG7MgoS H/eTFsAZAeGu7W2ctXHMRTgg+doR3QCYGBbVzx0teOtUDF5ZOqOSTHICsDBo4JWRT2RCMHOkh aH2G/wU58dJRaNX9L3BtnWLPzfDHuoTNGCzviN3WD3yPxZUk/0fQHYD57meI+raVk0rhePxSu z7r464/VDiw/h868g5oNr2yFf6vwYeZERtalkR9sPRC/6zPEx5GZWG1BDn+dNNAEqItg9Zy9s h6utD+X+FmRt27fs2Au+OiiA1RqUPDFgN8OOkiBugJKHLSduna/dAI14s6GyBlucpO50ROy3q un56leBke8qM9AatIfiSuykJ+sa/IKNGo9EcY6CjIBgYOctWUI9f6Grqu9hZvkHiHhJASLiVw yXN5wHS+DIa+8n/mwWhOH8OvAMSIo1jRHo9hZmtVItpqys9OLPf633p1+hC5ZGz+UqJP/B37F tAvKF3RkC9+b8R+CwykqILXpyalxiwuvmZE7/9LNdCAFOZJvagdinkSpBJ1Oy+ckOivt8lqGI B5iP9WqsVLF6QWy+dmSP/YOwUAi3MGF/qoA3UrHiUUttLFIUunDgzUiqsAZnc/slHX8UAEThh 4T/2LIhMBGAvnNVeIIQE4LfmgLAH3GLptqC5NjOtKGz/P3O13BzyNyh20J+Zyou4sHaRQz2j/ 7itIakUvBO3pW7NwG4Mr0sGYvhBsTh3K5CGn8Lcop40eLm3X88QirDXLK9ame/qCmH2/IyfxQ M68v25CS3ZZhUKvs0qR9fugh5nB8viCkQrueCzoFU4PrIdcSygyb2Yer4U4= X-Rspamd-Queue-Id: 4GysBK2m3gz4bTc X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of james.wright@digital-chaos.com has no SPF policy when checking 212.227.17.13) smtp.mailfrom=james.wright@digital-chaos.com X-Spamd-Result: default: False [-1.74 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[212.227.17.13:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.993]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[digital-chaos.com]; NEURAL_HAM_SHORT(-0.87)[-0.874]; NEURAL_HAM_MEDIUM(-0.77)[-0.770]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[82.29.153.121:received] X-ThisMailContainsUnwantedMimeParts: N if On 29/08/2021 05:05, Colin Percival wrote: > I don't have any data on instances launched outside of the AWS Marketplace. > > So, as I started this email saying: We have a decision to make and I'm > looking for input from the community. Faster booting, or support for older > EC2 instance types in our published AMIs? Hi Colin, Here is what I currently run 24/7 on AWS with FreeBSD (non Marketplace AMI); 4 x t3a.nano (FreeBSD-12.2) 1 x t3a.micro (FreeBSD-12.2) I try move to newer instance types where possible (and stable) as they generally offer the same or higher performance for less cost. I haven't moved to FreeBSD-13 yet as I'm allowing time for any issues to be ironed out, hopefully in 13.1 (such as that TSC timecounter fix). Next move might be onto the t4g (Graviton/ARM) instances but haven't properly tested them yet. I personally would prefer the faster booting with UEFI rather than supporting older instance types which will eventually become obsolete anyway. Thanks, James From nobody Mon Aug 30 16:42:49 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C21FE17AF4F9 for ; Mon, 30 Aug 2021 16:43:05 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from wnew2-smtp.messagingengine.com (wnew2-smtp.messagingengine.com [64.147.123.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gyx2c5CRNz3QTd for ; Mon, 30 Aug 2021 16:43:04 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailnew.west.internal (Postfix) with ESMTP id F149D2B00947 for ; Mon, 30 Aug 2021 12:42:58 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Mon, 30 Aug 2021 12:42:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h= date:from:to:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=HPeuo03niQdgFN8Do/VIsoqh4X/ P0RqzJpMPne9azOc=; b=YO3W3fRqoA2ICB93vXqpv0myarTFQVFHxmMitmjxjrr MXKUXQ2Z105ubOr71jxJW4qkUFZbOZOsCgcsfwLu0ldnlBaxstXvqLzO6d6UVuPy o1Kk/Hy9Z14A5YimNSmN1cFn0w9IwF994L5GakSripGm8ykihNOTF8h0iAnmvkMt EgjWCwyObR6qVmdxlhHuyAgGfTRM1WIxbcrlaqT4hpWD/Op/JSR4yg6EApOakYjf yLkqAV+XLvie8SZEHUEyOmpF7j95y6YsiKydhMQgn/3DF7svfe/KDtym5+78OWow 32qQnKH0kRgba1wrMzH+67wtlB+Ei+7jFHtd7m+3VYQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=HPeuo0 3niQdgFN8Do/VIsoqh4X/P0RqzJpMPne9azOc=; b=mVsfOq4fVZy5gwLV1/BMgp xfi2+Fkkcwte6ZVNECSNq0mV3xHZQNKDM/xwnwhd94YWwofw0k/bdrn3T/UQbxD4 U81R7RE/B/PjMToCRHhRKLGst/dd1/JCdJ4fQzNiwOraSFMoSAar8Ms111UFPY+L ud6KnJIaD3QTu9Pa+HnsnhHORfVjbdP6jOAsnbVKqK+HWCxYLDTUXlk02WdJVt2d MfsXInfOLP8yLfr605Tlll95Dn79Bz5MTZ4WeleVC2REVubLkdkI5h6C2iirdyTr 3+0xiDO7swAUfYZQtvNy47GwOSgN2JjLUmpQT/IwMnDewMY2oFxfS7emz18zQuaA == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudduledguddtgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehgtd erredttddvnecuhfhrohhmpehtvggthhdqlhhishhtshcuoehtvggthhdqlhhishhtshes iiihgihsthdrnhgvtheqnecuggftrfgrthhtvghrnheptedttdduuefggeeghfekkeetke ejleefffelheejfffgffdtfeeftdejgeeuieffnecuffhomhgrihhnpehfrhgvvggsshgu rdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epthgvtghhqdhlihhsthhsseiihiigshhtrdhnvght X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Mon, 30 Aug 2021 12:42:51 -0400 (EDT) Date: Mon, 30 Aug 2021 17:42:49 +0100 From: tech-lists To: freebsd-cloud@freebsd.org Subject: Re: sysutils/azure-agent has been updated to 2.2.54.2 Message-ID: References: List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QRT899oSdDQNBGY+" Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4Gyx2c5CRNz3QTd X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zyxst.net header.s=fm3 header.b=YO3W3fRq; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=mVsfOq4f; dmarc=none; spf=none (mx1.freebsd.org: domain of tech-lists@zyxst.net has no SPF policy when checking 64.147.123.27) smtp.mailfrom=tech-lists@zyxst.net X-Spamd-Result: default: False [-6.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm3,messagingengine.com:s=fm3]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; RCVD_COUNT_THREE(0.00)[4]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_NA(0.00)[zyxst.net]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.27:from] X-ThisMailContainsUnwantedMimeParts: N --QRT899oSdDQNBGY+ Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, On Sun, May 23, 2021 at 09:49:49PM +0800, Li-Wen Hsu wrote: >sysutils/azure-agent has been updated to 2.2.54.2, along with several fixe= s: > >https://cgit.freebsd.org/ports/commit/?id=3Db185a2b866b8642e2500dd69126769= 4ac388875b > >Please help test if you're using FreeBSD on Azure. > >I will include this in the (belated) 13.0 Azure marketplace image, >sorry that it was delayed because of several technical and >non-technical issues. Is it safe to upgrade from 12.2-p9 to 13.0-p4 via freebsd-upgrade, on an az= ure instance? thanks, --=20 J. --QRT899oSdDQNBGY+ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAmEtCoEACgkQs8o7QhFz NAWKZQ/+K9ZV3al2UN1cOprrQroZe+1YF6qHmPqIKS2RGDc+lQ5g+qGrpMsA1W78 GEQBHq8cQ+IokxGzJwrLRjZseIQQ6rEM7qHnhIR+Od7nq372gWQKFCe1gfdDQWEQ zGNHtDH//qruWAzZUst3CDCDbKr0jjOxyqDKGQoVGrA2+XZkr8MP3HDzgmR/Vi+S gc42N5vWI4qCjyD655uryVwKWtDmFC5H7jvFZ8bMK40kY09bEQ0ZL2f8m1QnjjLy FP3/GvnjZul/TRGoM/jKzP0W7t2Bpwe2QIS7wzOtHiyyN8djXS4bE8g50ody8NVw tHxoug3d0JtFZoPcs57JhXmWCL7Zep4P6MGziSjzAZ7ouyNPQn0ML5oQY61UzHZh adQznKO9f6dqO4WZQZzCoh+YhqSj29e5j4znhSLef9IhLe1+0v6WXyq3jGOfGuTP vH8mQF6/HSsWl+Ios5NwQKfN3iclkkZ0J7Oxag+i0cOCP9Kxrw2QiyTtE3vzUFvs y3gjCKF2NqjE4vPQ6FCzh5J428QlLkDh6prTnadX/q0z+DBw+N/LZgWFSI8JvZyO aPjw54WJSAbCCnCZajHfUQv4lU2JKTHiy5+Y1l1mTFx8C3a1OtGTeeRt+GslFDu0 D2qnwwBtUOa+VqDttlvto0SLONwFpy7cCiy6NqJZK0o9AY66iY0= =Vy5v -----END PGP SIGNATURE----- --QRT899oSdDQNBGY+-- From nobody Mon Aug 30 17:45:18 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 93961179C041 for ; Mon, 30 Aug 2021 17:45:30 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GyyQf3hqNz4RsG for ; Mon, 30 Aug 2021 17:45:30 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from mail-vs1-f49.google.com (mail-vs1-f49.google.com [209.85.217.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: lwhsu/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 62B602450 for ; Mon, 30 Aug 2021 17:45:30 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: by mail-vs1-f49.google.com with SMTP id b5so11186104vsq.2 for ; Mon, 30 Aug 2021 10:45:30 -0700 (PDT) X-Gm-Message-State: AOAM532GpyKfted+Y4cqf7nOUc6QTQs21pXNxfj0Vh1+MrUXh8/9ej07 1hbQ12EpWmP/kW4Sdn14IaURfqq8WcttJ0Y7lYo= X-Google-Smtp-Source: ABdhPJzT0vzvqS4SkdmO38oy2B7JvYcxGIaQGuLrv2ATWHYKAd47KKm+4Liaxh3tUzkXt0GwTZvO25zU70hqMYCPggQ= X-Received: by 2002:a67:ef51:: with SMTP id k17mr8501067vsr.2.1630345530047; Mon, 30 Aug 2021 10:45:30 -0700 (PDT) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Li-Wen Hsu Date: Tue, 31 Aug 2021 01:45:18 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: sysutils/azure-agent has been updated to 2.2.54.2 To: tech-lists Cc: "freebsd-cloud@FreeBSD.org" Content-Type: text/plain; charset="UTF-8" X-ThisMailContainsUnwantedMimeParts: N On Tue, Aug 31, 2021 at 12:43 AM tech-lists wrote: > > Hello, > > On Sun, May 23, 2021 at 09:49:49PM +0800, Li-Wen Hsu wrote: > >sysutils/azure-agent has been updated to 2.2.54.2, along with several fixes: > > > >https://cgit.freebsd.org/ports/commit/?id=b185a2b866b8642e2500dd691267694ac388875b > > > >Please help test if you're using FreeBSD on Azure. > > > >I will include this in the (belated) 13.0 Azure marketplace image, > >sorry that it was delayed because of several technical and > >non-technical issues. > > Is it safe to upgrade from 12.2-p9 to 13.0-p4 via freebsd-upgrade, on an azure instance? I haven't tested this, but I believe this should be safe, I use buildworld and buildkernel to upgrade the image builder VM from 12.2 to 13.0, no code is modified. The builder is running 13.0-p4 now. BTW, some updates for the official 13.0 image: I'm trying to provide support for the gen2 VM with 13.0, but certain gen2 VM sizes have a boot issue so I may directly go with the gen1 VM for now, then continue the debugging of the gen2 VM. Best, Li-Wen From nobody Mon Aug 30 18:03:19 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 68DA917A2289 for ; Mon, 30 Aug 2021 18:03:38 +0000 (UTC) (envelope-from jedberg@gmail.com) Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GyyqY4tMCz4WbW; Mon, 30 Aug 2021 18:03:37 +0000 (UTC) (envelope-from jedberg@gmail.com) Received: by mail-pj1-x1033.google.com with SMTP id u13-20020a17090abb0db0290177e1d9b3f7so41865pjr.1; Mon, 30 Aug 2021 11:03:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=O4+sKsqVbEMiqDhd6ZfeBDO7s1q5+KAepy1Rs2FGaVQ=; b=L5DCivyOzlfl0wZcPqTN01dxvMCOXccdgYKQOtB1IVhc4Js9xbnImfExdCdIYoLPlm msWes0V7yEswFFJPtxTI4y7bTHsFmYLYGkU2W5wS+fzsypJuv36IIpYkPbS+sPora2SG fbcjAcTD4oYQWR1LAlVlk7RwOyMaIRf0U4mbIxhULy51Q4hSRGsgqgiLmyhiIIjFHpn1 YzMecM6eYTmJ1f5CKwQxSk9hEpQixTIW/6NN7+rCVLlk9lYIZpQCjYNAb2vgqRjOGeey ZuI/TAaP2KfBseq41xkNlo7MOpAc0wju1+rEkkBK1Gl5p5Yn7XqknTQ79JJrRi+/ws7q UJcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=O4+sKsqVbEMiqDhd6ZfeBDO7s1q5+KAepy1Rs2FGaVQ=; b=h9CMr8Rj/zvR3zuegKHJezhRncnB7jNiXU+ixTVfDD/mxSkbtqD7EnOa+OTOhuHagh l44GdQpdwI9M1mLkLu7siiLOoSdTnT49APdh1ZmuG3RHHXrr38iCmti8RPLEnC5LXJNv lIObhet1Bx4ZYMJRBWGO1O8EECIsRT4WVYDB17q3pGNopaarRZ0UWBxwRvGPyrhtLdQl Sh8Iqwqw4XS0//icYDUSM0xzGNHNq1xnE4hJkf5dWS7HqkaBqJgXJUuDkcC3ym3La9g9 gcn9cKvVF0K9QcaAOqMIxccEgK6Vh2kpartmfY5O2uJxi8rq1dFFj9IAep26MdpRKajV XDrQ== X-Gm-Message-State: AOAM532EKWPa8u3I8DPvh61IvRxmHmjeD95zSBcadzwYpCv9StfM0HiJ ItGx9w6kclGn3fTqoScFxlFKJ+HLWzxg9mo/rxGqBCDHTdY= X-Google-Smtp-Source: ABdhPJzXpNIyNitmQiiBhFHbPqSdyx6PY7lASgKkpvht7jORSfFi9nbP+4JXShRVDVepxoQPvTeVMWMtKwBqbcCFOWg= X-Received: by 2002:a17:90a:fe8e:: with SMTP id co14mr354279pjb.200.1630346611046; Mon, 30 Aug 2021 11:03:31 -0700 (PDT) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 References: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> In-Reply-To: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> From: Jeremy Edberg Date: Mon, 30 Aug 2021 11:03:19 -0700 Message-ID: Subject: Re: RFC: Switching FreeBSD/EC2 images to UEFI boot To: Colin Percival Cc: "freebsd-cloud@freebsd.org" Content-Type: multipart/alternative; boundary="0000000000009cbd7205cacaa639" X-Rspamd-Queue-Id: 4GyyqY4tMCz4WbW X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=L5DCivyO; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jedberg@gmail.com designates 2607:f8b0:4864:20::1033 as permitted sender) smtp.mailfrom=jedberg@gmail.com X-Spamd-Result: default: False [-2.19 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.81)[0.807]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1033:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: Y --0000000000009cbd7205cacaa639 Content-Type: text/plain; charset="UTF-8" I'm always a fan of optimal defaults. If UFEI boot is faster, it should probably be the default, especially if you're still able to generate BIOS boot instances. Unless there is a large group of people who for legal/compliance reasons *must* use the official AMIs and also can't switch to the newer instance types. j On Sat, Aug 28, 2021 at 9:05 PM Colin Percival wrote: > Hi Cloudy people, > > We have a decision to make and I'm looking for input from the community: > Should FreeBSD's x86 EC2 AMIs be marked as booting using UEFI instead of > the current (default) BIOS boot mode? > > The argument in favour of sticking with BIOS booting is that the older EC2 > instance types -- up to M4/C4/R4/T2 -- don't support UEFI. There's a full > list of which instance types support UEFI at > > https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ami-boot.html > > but roughly speaking it's a case of "Nitro instances support UEFI while Xen > based instances require BIOS". (Note that whatever decision we make, it > will > still be possible to build AMIs which support BIOS booting -- the question > is > just which mode the official project-published images are marked to use.) > > The argument in favour of switching to UEFI booting is that it's faster -- > around 4.7 seconds faster in my testing. While this may not seem like a > big > difference, it's the difference between being on par with most Linuxes or > lagging far behind; and there's a definite quality-of-life benefit to being > able to spin up an EC2 instance and SSH in to start using it quickly. > (Also > benefits for people using autoscaling, since they can respond to increased > load faster.) > > One obvious question to ask here is "how many people are using the older > instance types which only support BIOS", and I have some data from the AWS > Marketplace about that: > * Instances originally launched on FreeBSD 9.x and 10.x are all using older > instance types. > * About 60% of instances originally launched on FreeBSD 11.x are using > older > instance types. > * About 40% of instances originally launched on FreeBSD 12.x and 13.0 are > using older instance types. > > Almost all of the "older instance type" usage on FreeBSD 12.x and 13.0 is > T2 > family instances, which I suspect is for two reasons: > 1. For a long time I had t2.micro listed as the default instance type on > the AWS Marketplace listings, and > 2. FreeBSD 13.0 has problems on T3 instances due to the TSC timecounter not > working reliably. This problem has now been corrected in HEAD and I'm > hoping > the fix will be MFCed in time for FreeBSD 13.1. > > I don't have any data on instances launched outside of the AWS Marketplace. > > So, as I started this email saying: We have a decision to make and I'm > looking for input from the community. Faster booting, or support for older > EC2 instance types in our published AMIs? > > -- > Colin Percival > Security Officer Emeritus, FreeBSD | The power to serve > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid > > --0000000000009cbd7205cacaa639-- From nobody Mon Aug 30 18:10:22 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C488317A5FC6 for ; Mon, 30 Aug 2021 18:10:51 +0000 (UTC) (envelope-from dan@langille.org) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gyyzt4kl1z4ZDJ for ; Mon, 30 Aug 2021 18:10:50 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 3CEFB320092B for ; Mon, 30 Aug 2021 14:10:44 -0400 (EDT) Received: from imap42 ([10.202.2.92]) by compute6.internal (MEProxy); Mon, 30 Aug 2021 14:10:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=OtysnV+N8k6JBTevdQ7VYhlIa/zURjT eyainvyIUDys=; b=r//XeuhUCTbzJ9K6M/RoWfWmVfpaxarWcoFS2GSOwbZsjVI /AovCCcCQO9wWMS4g40sNMioVrMBfpDhtMEa193Ej5hoNNUSp844uHhBqWXg3h21 DRDFVcAD2BjEJyJt5vaK2UULH0BlHUnIvi/Wll5yDMf2fqwsnhpLCLjlxGceuGOV 7S06vScX2Jg1pybEFXEqHyhXY44rlMWQntmzdUoi0NcK3UqDcwfNUw/VaQccKb0y aFEMjEafb5Iyb4wScJLfW3rBvd2wZ9uIOvcV6D2srd7zxPbgwlwNGprxQO8MyX5A sc4LY9yXfnjAjfT60Pyt5OAxErXGjz78JqBxl6w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=OtysnV +N8k6JBTevdQ7VYhlIa/zURjTeyainvyIUDys=; b=YsnE3WOj07mGt7jbrTZkbi 3oKcyf9wES/5uq+eAcxsIIRD8MxjpIu/HaDCudXu7O2WP3uK2r7q6MlzRlEGmoUt wxDgqUS0OAGWu10MBCXtF2EFuvL+3D/POlp6C9CE5SSlaFmuRXlVMTzi8oIxJVpv egmCPcckAnGxGTXTnmx8E8Bwk2GY8OJBIizfe77N4ZHQlA/jXkx51NCx8IwhEuAc v/dEcidSWR/SIEiACfsSsGwurLVdz1njqxM1LoZi9KD23FUUQ6JnJlA6/aXBOLbb ush7GOlJ1pamnkiG+0Km5YMBpHxogclTIEH5X/mlnm8FvPTjeSBfD3F0S8Mq/eqA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudduledguddvvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefofgggkfgjfhffhffvufgtsehttdertderredtnecuhfhrohhmpedfffgr nhcunfgrnhhgihhllhgvfdcuoegurghnsehlrghnghhilhhlvgdrohhrgheqnecuggftrf grthhtvghrnhepleejgfegueehueeiveelvdejvefgudetueekgffgffeiveehudfgffev heejveffnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id A465E218006A; Mon, 30 Aug 2021 14:10:42 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.5.0-alpha0-1125-g685cec594c-fm-20210825.001-g685cec59 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 Message-Id: <9c66fdd4-89ab-4125-b5da-575248a1cc7d@www.fastmail.com> In-Reply-To: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> References: <79c48cc2-5341-2cea-7d6e-4372f7c93245@freebsd.org> Date: Mon, 30 Aug 2021 14:10:22 -0400 From: "Dan Langille" To: "Application Certification Support via freebsd-cloud" Subject: Re: RFC: Switching FreeBSD/EC2 images to UEFI boot Content-Type: text/plain X-Rspamd-Queue-Id: 4Gyyzt4kl1z4ZDJ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b="r//XeuhU"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=YsnE3WOj; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 64.147.123.21 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-4.09 / 15.00]; XM_UA_NO_VERSION(0.01)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.21]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.21:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; RWL_MAILSPIKE_POSSIBLE(0.00)[64.147.123.21:from]; MID_RHS_WWW(0.50)[] X-ThisMailContainsUnwantedMimeParts: N On Sun, Aug 29, 2021, at 12:05 AM, Colin Percival wrote: > So, as I started this email saying: We have a decision to make and I'm > looking for input from the community. Faster booting, or support for older > EC2 instance types in our published AMIs? If my instance is older, how will this change affect it? -- Dan Langille dan@langille.org From nobody Tue Aug 31 15:17:25 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A989217A06FC for ; Tue, 31 Aug 2021 15:17:32 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from mail.tarsnap.com (mail.tarsnap.com [54.86.246.204]) by mx1.freebsd.org (Postfix) with SMTP id 4GzW5S2sgRz4pQ7 for ; Tue, 31 Aug 2021 15:17:32 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: (qmail 29125 invoked from network); 31 Aug 2021 15:17:26 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by mail.tarsnap.com with SMTP; 31 Aug 2021 15:17:26 -0000 Received: (qmail 26299 invoked from network); 31 Aug 2021 15:17:25 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by localhost with SMTP; 31 Aug 2021 15:17:25 -0000 To: "freebsd-cloud@freebsd.org" From: Colin Percival Subject: EC2 AMI Systems Manager Public Parameters Message-ID: <1913e7d6-9eed-b9f5-3e8a-5cf9f70d568f@freebsd.org> Date: Tue, 31 Aug 2021 08:17:25 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GzW5S2sgRz4pQ7 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:14618, ipnet:54.86.0.0/16, country:US] X-ThisMailContainsUnwantedMimeParts: N Hi cloudy people, You can now look up EC2 AMI Ids in the Systems Manager Parameter Store: https://www.daemonology.net/blog/2021-08-31-FreeBSD-AMI-SSM-Public-Parameters.html Right now just RELEASEs from 12.0 onwards; later this week 14.0-CURRENT will be added (when the next snapshots are built), and in a few weeks the 12-STABLE and 13-STABLE snapshots will start showing up there as well. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From nobody Thu Sep 23 08:49:42 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4E08117D18C8 for ; Thu, 23 Sep 2021 08:49:54 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HFTPY2zdXz3ll7 for ; Thu, 23 Sep 2021 08:49:50 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 8C8F25C01BC; Thu, 23 Sep 2021 04:49:44 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 23 Sep 2021 04:49:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=from:to:subject:date:message-id:mime-version:content-type :content-transfer-encoding; s=fm1; bh=BJX6BIdnaszb9DMB62IAsoKwxZ EBABKnOdPPfzWNHyw=; b=FCJEJsWA1TKsJvLkS6capWSZVsQo+93PewgVxVWZ6g OE4TlAzjM7u+MHI02wC1aadU1xy0d8aIHOZezXOxPmuPBtFYR4TcFlJ8nrPWAocC VV1jQy6RJ8FVJbBzxY7Qrl4h95J1aWWj/vg129sTBC+wiGypJ/+3ngUXqdqY+kMv my3WmLV+GXWyQOkC1o2okI/d0u6oQCNbOwbMLLItFD0wiZzKOGjAYBBBxqKUQ4gc vFHubiVk8NizX1MKvebbnyGpuq33wphARTbHaiNKM8abwQm2pDq0ddcT7IpdbG3W vt46gLHaqN52nUZXmKz+27J2YL6E1sYs3idRbqy4/j7w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=BJX6BI dnaszb9DMB62IAsoKwxZEBABKnOdPPfzWNHyw=; b=mLLZPyd3ffrjrp4Y/HKOiP SKfCE+bIS334m+BweY3e4uTeGRaYZWLcCScvzTpvbvMW/sqBjxLXxBulVh48p/FH W5HmC4CQjE+tYwMPYpdo573B/rvNZXNP0G5F4uPVFpnxMu/j6gq3N5KuO5AYKjOB R8iwk9rbgeHvwH5hVx38F2ZmJSJdh6qRJMOGZ3+D/99ThKoCyocQqXBam68ucrX6 TVxpBLQUdY3Unp9Jke34tXz1umsl8GN4ONA97r/9bQhG4kujy7gjSZsIPjGs3vVM 6t20KIuSU+Jao9TBagObrOjwZHffCM+5FJmoPMlHNDbJ/b8vge8VP162DhgonTFA == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudeiledgtdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffokfggtgfgsegrkehmre ertdejnecuhfhrohhmpedfrfgrthcuofgrugguohigfdcuoehprghtsehprghtmhgruggu ohigrdgtohhmqeenucggtffrrghtthgvrhhnpeevtdevgedvheeuvdekudeggeelfffgud duveetgedtudelhfefgfffkeffheehffenucffohhmrghinhepfhhrvggvsghsugdrohhr ghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehprg htsehprghtmhgrugguohigrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Thu, 23 Sep 2021 04:49:44 -0400 (EDT) From: "Pat Maddox" To: freebsd-cloud@FreeBSD.org Subject: Current status of ZFS AMIs on EC2? Date: Thu, 23 Sep 2021 01:49:42 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_MailMate_2131FD5B-E26E-42AF-8ADC-75490D76EF96_=" Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4HFTPY2zdXz3ll7 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm1 header.b=FCJEJsWA; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=mLLZPyd3; dmarc=none; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 66.111.4.29 as permitted sender) smtp.mailfrom=pat@patmaddox.com X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:11403, ipnet:66.111.0.0/20, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm1,messagingengine.com:s=fm3]; FREEFALL_USER(0.00)[pat]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; DMARC_NA(0.00)[patmaddox.com]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.111.4.29:from] X-ThisMailContainsUnwantedMimeParts: Y --=_MailMate_2131FD5B-E26E-42AF-8ADC-75490D76EF96_= Content-Type: text/plain; charset="UTF-8"; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit Hi there, I came across Colin’s 2019 post announcing ZFS AMIs for EC2 [1]. I haven’t seen any more recent information regarding ZFS on EC2. I’ve launched one instance, and it appears to work fine from initial tests (including recovering a known working boot env using beadm). I have also installed the official 12 and 13 releases, and see that they are UFS. I have two questions: 1. Is anyone using the ZFS AMIs for production? 2. Why has ZFS not been incorporated into the official releases? Thanks, Pat [1] https://lists.freebsd.org/pipermail/freebsd-cloud/2019-February/000200.html --=_MailMate_2131FD5B-E26E-42AF-8ADC-75490D76EF96_=-- From nobody Thu Sep 23 09:42:46 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3518C17D8038 for ; Thu, 23 Sep 2021 09:42:49 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HFVZc4cVvz3sMN for ; Thu, 23 Sep 2021 09:42:48 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 8A53B5C01E0; Thu, 23 Sep 2021 05:42:48 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 23 Sep 2021 05:42:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=from:to:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm1; bh= pbWrvWvQzCKlYi4fJBHfVN3BgOo9cwNcAVAWQdRE8YA=; b=RM5TnWHMkyTpb9LA PdfEgvxLiX8CYrCfe7zgaVvFWp0Di4j+EnZStjygVxV++dP+iW/rB5sQ7FkMIgaY OjJzR2zMju2ALltHXdHlAIy68BjoZL2HnxQxk0+NrOukcGjvtQmB9P7P1Z2LKUxD sjHZPDKmRZK2vzJdqvCnG4STMuPHxKpJuzmWZZT4OnWzkpg1RYC4DmmX2Q3QpFIM sgRdZH2vjNfMq1d8/Zo1W51bCH/Ou1juBOwiq33A1Do12b2rcybwLxHIj1J8+rC3 3i8oRyJl4944bMatvMLAiHOrdn8ELFWrY/pTXzbYlkoUpSH/2z1TP2okS/4VwtNc qrz8tQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=pbWrvWvQzCKlYi4fJBHfVN3BgOo9cwNcAVAWQdRE8 YA=; b=SlFWMOkXeHNNSeaca7xMHBDS5amwzRIzAClCDMSFBs+uq2sWNSwHmXvnY cXNbw6xoJsq/E0sNWQb69tClSwcrkZcl6yBA80i2UatGTNMRxss7ijqCPYilkB8I PIHTWMk1OY6Fn7Zb2UmbnzsIAoGxdzqeuc5Hy5XKOLhdkmJCRZAvcRowi6axUANl 8Lpg5XEXe9IXrSkvL84z7sN9G1PpiOLy2olh2c0rCEQ+XSkPISS9wU0eidqeRekQ cBudXRDVgRyGSW3exTywY6oN0bVVWSveXPWCumVZ+QzlcRr5xeSx2Bz1j2NHaike +bIk+hkfXpPjU3Sc7VSuN4quzkbLQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudeiledgudekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffokfgjfhggtgfgsegrke hmreertdejnecuhfhrohhmpedfrfgrthcuofgrugguohigfdcuoehprghtsehprghtmhgr ugguohigrdgtohhmqeenucggtffrrghtthgvrhhnpeeiveevgeejfeehgedvleeufeelff etfedthffhhfekveehleduiefgieekueeitdenucffohhmrghinhepfhhrvggvsghsugdr ohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hprghtsehprghtmhgrugguohigrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Thu, 23 Sep 2021 05:42:48 -0400 (EDT) From: "Pat Maddox" To: freebsd-cloud@FreeBSD.org Subject: Re: Current status of ZFS AMIs on EC2? Date: Thu, 23 Sep 2021 02:42:46 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: <60E1F943-9C3C-4265-B7EC-F49680B9262B@patmaddox.com> In-Reply-To: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> References: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_MailMate_E0509714-84B9-4F4E-BF2B-1048A8CCCC17_=" Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4HFVZc4cVvz3sMN X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm1 header.b=RM5TnWHM; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=SlFWMOkX; dmarc=none; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 66.111.4.29 as permitted sender) smtp.mailfrom=pat@patmaddox.com X-Spamd-Result: default: False [-3.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29:c]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.0.0/20, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm1,messagingengine.com:s=fm3]; FREEFALL_USER(0.00)[pat]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; DMARC_NA(0.00)[patmaddox.com]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.111.4.29:from] X-ThisMailContainsUnwantedMimeParts: Y --=_MailMate_E0509714-84B9-4F4E-BF2B-1048A8CCCC17_= Content-Type: text/plain; charset="UTF-8"; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit Also, is there a 13.0-release ZFS AMI? On 23 Sep 2021, at 1:49, Pat Maddox wrote: > Hi there, I came across Colin’s 2019 post announcing ZFS AMIs for > EC2 [1]. I haven’t seen any more recent information regarding ZFS on > EC2. I’ve launched one instance, and it appears to work fine from > initial tests (including recovering a known working boot env using > beadm). > > I have also installed the official 12 and 13 releases, and see that > they are UFS. > > I have two questions: > > 1. Is anyone using the ZFS AMIs for production? > 2. Why has ZFS not been incorporated into the official releases? > > Thanks, > Pat > > [1] > https://lists.freebsd.org/pipermail/freebsd-cloud/2019-February/000200.html --=_MailMate_E0509714-84B9-4F4E-BF2B-1048A8CCCC17_=-- From nobody Thu Sep 23 20:22:22 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 80BF517D3D17 for ; Thu, 23 Sep 2021 20:22:29 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from mail.tarsnap.com (mail.tarsnap.com [54.86.246.204]) by mx1.freebsd.org (Postfix) with SMTP id 4HFmmj1lmmz4Wk0 for ; Thu, 23 Sep 2021 20:22:29 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: (qmail 41656 invoked from network); 23 Sep 2021 20:22:23 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by mail.tarsnap.com with SMTP; 23 Sep 2021 20:22:23 -0000 Received: (qmail 36366 invoked from network); 23 Sep 2021 20:22:22 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by localhost with SMTP; 23 Sep 2021 20:22:22 -0000 From: Colin Percival Subject: Re: Current status of ZFS AMIs on EC2? To: Pat Maddox , freebsd-cloud@FreeBSD.org References: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> Message-ID: <12428f52-ea5a-a2e5-7eb6-e170c97f0e17@freebsd.org> Date: Thu, 23 Sep 2021 13:22:22 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 In-Reply-To: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4HFmmj1lmmz4Wk0 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 9/23/21 1:49 AM, Pat Maddox wrote: > Hi there, I came across Colin’s 2019 post announcing ZFS AMIs for EC2 [1]. I > haven’t seen any more recent information regarding ZFS on EC2. I’ve launched > one instance, and it appears to work fine from initial tests (including > recovering a known working boot env using beadm). > > I have also installed the official 12 and 13 releases, and see that they are UFS. > > I have two questions: > > 1. Is anyone using the ZFS AMIs for production? Yes. (Not me, but I've heard from several people who are. I'll let them identify themselves if they choose to do so.) > 2. Why has ZFS not been incorporated into the official releases? ZFS, and other "flavoured" AMIs, were blocked waiting for this: https://www.daemonology.net/blog/2021-08-31-FreeBSD-AMI-SSM-Public-Parameters.html Now it's just a matter of finding time; my current top priority for EC2 is speeding up the FreeBSD boot process. -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From nobody Thu Sep 23 21:00:52 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C51D917D8ED8 for ; Thu, 23 Sep 2021 21:01:02 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HFndB3XLqz4dR2; Thu, 23 Sep 2021 21:01:02 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 6221732007E8; Thu, 23 Sep 2021 17:00:55 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Thu, 23 Sep 2021 17:00:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=fm1; bh= J/BS2F3zE2251hGoIBwExqW1W4OIcmnaTkNb7Yf5GSM=; b=Fkz8Ncbv2asF/TnD foBQNJxOwMlax9gfDkzusqBE70oLGS0yAK5t/jyI/Xe5kDPu6Zcs1sL4YLdl04uB WW8HXCXYgBuQTv2K+4IY93DJCVXcwLRH8HJQRuPLLYViiDAwCZS7qsoHoyapOTYJ aM6yQGSzCpMVU79/r20DbLPLG23djlbXIXAHLpaXC/ZhDzKDykm9mBI2JWeSDWTy uYQeW9bK2YWU1KHhL94hht7jD5IP7qB3Z9AUv/jsDUVv57RSQjMChTYg269jFJ9o I6yKDmflJVravaGTuso4jUjoxiN0zGsk+BqKShNBarFi269nfFUJC4Wp1MRtj4vi fOyJaA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=J/BS2F3zE2251hGoIBwExqW1W4OIcmnaTkNb7Yf5G SM=; b=PFG0D+rTIq9euE/Szupbh0XjhkHYse3msr0JTILpG7mtfyVxY6a3aVYeU VIDTyKUg1JucVKDU5RhtzkVs0l5LxO2oaYZFjHTyPqvxfP/C+FH2ynODXeljpDeN Mv8U16420HGnSeb1oWOmypWQnn/IgoJZuoEXWsizFZHO2OzlsNRVfr9wxctHO8/L ibGilqWYkyLWbjhbG1eogoIY+U/k2SgEdNQrx4FrOwIcJz6TjC4fp/Z+ok5aYJoH 1ylE5/1wHUJ7uu4fcRWeTVIF630OHSpt2WqQh2dkg5UnJDzwN+vBadxktim0TJc2 J4TmvndAC1vFJT16vk6G66bncmWvg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudeiledgudehhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufffokfgjfhggtgfgsehtqhhmtdertdejnecuhfhrohhmpedfrfgr thcuofgrugguohigfdcuoehprghtsehprghtmhgrugguohigrdgtohhmqeenucggtffrrg htthgvrhhnpeegveejjeevleetfedvleegvefftdelhfefudfhgfeiheektdelueelveeh ffehteenucffohhmrghinhepuggrvghmohhnohhlohhghidrnhgvthenucevlhhushhtvg hrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehprghtsehprghtmhgruggu ohigrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 23 Sep 2021 17:00:54 -0400 (EDT) From: "Pat Maddox" To: "Colin Percival" Cc: freebsd-cloud@FreeBSD.org Subject: Re: Current status of ZFS AMIs on EC2? Date: Thu, 23 Sep 2021 14:00:52 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: In-Reply-To: <12428f52-ea5a-a2e5-7eb6-e170c97f0e17@freebsd.org> References: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> <12428f52-ea5a-a2e5-7eb6-e170c97f0e17@freebsd.org> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4HFndB3XLqz4dR2 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 23 Sep 2021, at 13:22, Colin Percival wrote: > On 9/23/21 1:49 AM, Pat Maddox wrote: >> Hi there, I came across Colin=E2=80=99s 2019 post announcing ZFS AMIs = for = >> EC2 [1]. I >> haven=E2=80=99t seen any more recent information regarding ZFS on EC2.= = >> I=E2=80=99ve launched >> one instance, and it appears to work fine from initial tests = >> (including >> recovering a known working boot env using beadm). >> >> I have also installed the official 12 and 13 releases, and see that = >> they are UFS. >> >> I have two questions: >> >> 1. Is anyone using the ZFS AMIs for production? > > Yes. (Not me, but I've heard from several people who are. I'll let = > them > identify themselves if they choose to do so.) > >> 2. Why has ZFS not been incorporated into the official releases? > > ZFS, and other "flavoured" AMIs, were blocked waiting for this: > > https://www.daemonology.net/blog/2021-08-31-FreeBSD-AMI-SSM-Public-Para= meters.html > > Now it's just a matter of finding time; my current top priority for = > EC2 > is speeding up the FreeBSD boot process. Great, thanks for the info. I=E2=80=99ve done a 12.0 -> 12.2 -> 13.0 upgr= ade, = all seems to be good. Pat From nobody Thu Sep 23 22:13:21 2021 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8DE7B17DEE42 for ; Thu, 23 Sep 2021 22:14:53 +0000 (UTC) (envelope-from erobison@gmail.com) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HFqGP3B1Dz4jsg for ; Thu, 23 Sep 2021 22:14:53 +0000 (UTC) (envelope-from erobison@gmail.com) Received: by mail-wr1-x42b.google.com with SMTP id u18so21448235wrg.5 for ; Thu, 23 Sep 2021 15:14:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HysEjHIL0Si8GQBUnw78vzhljn4cAikabaWpmnazAp8=; b=SrxWT97b5FQ27O4at/ZRRVU7UGDXPrJiVfMB5Gf7JKA+BYwtbRMD5X2GD+96u+zgK3 2qEfJAHo/Q2XsLkQNlhHwxWLpU0WecORqeqJ6kmOQkgBZXyYG9XSX1YWgj6IOrC3K22W A7c+snUrgdT6sScrjMufbrwCBbDG5UEoFzp/igj/EiBu2SdZRQriJXmNC4SdhjBbWQtW McgAELNhKEt/zM5yUcfzMVcZ2vLOf594QsUNpdJ8zeJDXrsVIxJmlPqE8XkO8CojrFEj dZ/qnPK8I9zjk9GlWJfphK+Wrwps25IVWq/yciEnluddqa9bFLiqL0DucVsUICHeudx2 O8LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HysEjHIL0Si8GQBUnw78vzhljn4cAikabaWpmnazAp8=; b=YesVvcAm09L9j4PibvOdUuXlMb3zy+cM9tWhnN+glt3NxkbVZeLcM6C/BFsRPL72jt iyar/xlg4ioRYPCCpJFrxvmXkjtd8gYmkT/Em3PiwoT/oZ/KFBacuXpMYCzfSZCY2WFQ 5yzwGBTdnVxJEeLDgXmSF0eXI8Y+kuRsKj+CbPREz/Qm30354WeQ+kysn6GDM4mOnPg0 Bx5FVzLcynXcbPqffWsgB/I6axEv8SNCyz7/L6FzsH6G3NLFz/0aPSnOH6EreyCxKdXV snAEOnnl/FlYmfHuT8fvxNWU+MH/WmVaM8MXjVEdL9bF2lZkcxZBP48mOxa4V3C1FvDS NfHw== X-Gm-Message-State: AOAM532dp6VehufzA9bS6HuWTrhiy7450uVGJsvlyxIutT5Vp7fmnEZu si1HosET5sXwBRReSgxIsThZIJPrOFIto3Pimas= X-Google-Smtp-Source: ABdhPJyTNxTLm/DuJbo8G65LliSBghuXJO0myjUV3C9fOAPZq+/Ut/ZUQYS2vWwc1f2FfFhEX3rt/CLwEwa/PNtZSKc= X-Received: by 2002:a5d:6dce:: with SMTP id d14mr8059591wrz.363.1632435292301; Thu, 23 Sep 2021 15:14:52 -0700 (PDT) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 References: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> In-Reply-To: <65D39FF2-C14D-430E-A83B-0C3606E0D7AF@patmaddox.com> From: ericr Date: Thu, 23 Sep 2021 16:13:21 -0600 Message-ID: Subject: Re: Current status of ZFS AMIs on EC2? To: Pat Maddox Cc: freebsd-cloud@freebsd.org Content-Type: multipart/alternative; boundary="000000000000b76f2505ccb0f532" X-Rspamd-Queue-Id: 4HFqGP3B1Dz4jsg X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --000000000000b76f2505ccb0f532 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Sep 23, 2021 at 2:50 AM Pat Maddox wrote: > Hi there, I came across Colin=E2=80=99s 2019 post announcing ZFS AMIs for= EC2 > [1]. I haven=E2=80=99t seen any more recent information regarding ZFS on = EC2. > I=E2=80=99ve launched one instance, and it appears to work fine from init= ial > tests (including recovering a known working boot env using beadm). > > I have also installed the official 12 and 13 releases, and see that they > are UFS. > > I have two questions: > > 1. Is anyone using the ZFS AMIs for production? > Yes. ~30 in five regions, mostly 12.2-RELEASE. > 2. Why has ZFS not been incorporated into the official releases? > $COLIN =3D 1 Hopefully some day soon ZFS will be the default filesystem for all installs and images. For now, we use the 12.0-RELEASE AMI Colin built (Thanks Colin!) and upgrade to 12.2, then make an AMI of it. If we want 13.0, we'll upgrade a copy and create another AMI. It's not painful, and it doesn't take long. There are other ways to do it, but being lazy I find that freebsd-update -r takes the least effort. Once you make a 'base' AMI, you don't have to deal with it again. [1] > https://lists.freebsd.org/pipermail/freebsd-cloud/2019-February/000200.ht= ml --000000000000b76f2505ccb0f532-- From nobody Wed May 18 13:21:38 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 12BC21AE443D for ; Wed, 18 May 2022 13:22:12 +0000 (UTC) (envelope-from raf@rafal.net) Received: from mailtransmit04.runbox.com (mailtransmit04.runbox.com [IPv6:2a0c:5a00:149::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4L3DDM239bz4bG0 for ; Wed, 18 May 2022 13:22:11 +0000 (UTC) (envelope-from raf@rafal.net) Received: from mailtransmit02.runbox ([10.9.9.162] helo=aibo.runbox.com) by mailtransmit04.runbox.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1nrJcn-00Bxua-OE for freebsd-cloud@freebsd.org; Wed, 18 May 2022 15:22:01 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=rafal.net; s=selector2; h=To:Cc:Date:Message-Id:Subject:Mime-Version:Content-Type:From; bh=sQRupGhXxwJ2U1fgjM3zQxPork8GTQpT1T9qpRJxz+g=; b=j7o86m19Ltg6fy5voi6N3/GMW JH57JNl8gyrG/p7mzU8RQbuA0hfBxns+lwDf4cowFehXEWnndfC9txuzJD4EMaZIRNSsvusi/pZqO orxR1aL3z1E+u/7jX84im8GodGmgsZOrfwXN+obMaUSOtecHs4fFSWzPnQv4EVKMF2WSr8GefiUGH vmSuu5i1FEVdH/jBRsqz1AawZtrp3ybDHlQH1KMoIw4G0H5Wn9E7rN0Ll3Qz1hfVMTvjf0amLBjDT C0/qFyi+10Q8PctL40T3z5PlmpY4VaxZiR9w/M20O7CAEgorlRiPpDnApNKm0g/jYrJzNzbGFoLP2 Vbe8jWqLA==; Received: from [10.9.9.73] (helo=submission02.runbox) by mailtransmit02.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1nrJcn-00078d-6h; Wed, 18 May 2022 15:22:01 +0200 Received: by submission02.runbox with esmtpsa [Authenticated ID (1087189)] (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) id 1nrJcR-0001IF-4r; Wed, 18 May 2022 15:21:39 +0200 From: Rafal Lukawiecki Content-Type: multipart/alternative; boundary="Apple-Mail=_BC5FE89C-2C15-4EB3-BE66-F4A34FA0B771" List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.80.82.1.1\)) Subject: AMI Builder 13.1? Message-Id: Date: Wed, 18 May 2022 14:21:38 +0100 Cc: Colin Percival To: freebsd-cloud@freebsd.org X-Mailer: Apple Mail (2.3696.80.82.1.1) X-Rspamd-Queue-Id: 4L3DDM239bz4bG0 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=rafal.net header.s=selector2 header.b=j7o86m19; dmarc=pass (policy=reject) header.from=rafal.net; spf=pass (mx1.freebsd.org: domain of raf@rafal.net designates 2a0c:5a00:149::25 as permitted sender) smtp.mailfrom=raf@rafal.net X-Spamd-Result: default: False [-2.60 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[rafal.net:s=selector2]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a0c:5a00:149::25]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[rafal.net:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[rafal.net,reject]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MLMMJ_DEST(0.00)[freebsd-cloud]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:50304, ipnet:2a0c:5a00::/29, country:NO]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[2a0c:5a00:149::25:from] X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_BC5FE89C-2C15-4EB3-BE66-F4A34FA0B771 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hello Colin, I am working on our FreeBSD today and this week=E2=80=94after a long = break. Do you have new AMI Builders for the latest 13 release? I intend = to run the builder on c6g.8xlarge and to use the produced AMIs on = t4g.medium. Many thanks, Rafal -- Rafal Lukawiecki Data Scientist=20 Project Botticelli Ltd --Apple-Mail=_BC5FE89C-2C15-4EB3-BE66-F4A34FA0B771 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hello= Colin,

I am working = on our FreeBSD today and this week=E2=80=94after a long break. Do you = have new AMI Builders for the latest 13 release? I intend to run the = builder on c6g.8xlarge and to use the produced AMIs on = t4g.medium.

Many= thanks,
Rafal
--
Rafal Lukawiecki
Data Scientist 
Project Botticelli = Ltd

= --Apple-Mail=_BC5FE89C-2C15-4EB3-BE66-F4A34FA0B771-- From nobody Wed May 18 18:49:52 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4DB861B3F6F4 for ; Wed, 18 May 2022 18:50:18 +0000 (UTC) (envelope-from raf@rafal.net) Received: from mailtransmit05.runbox.com (mailtransmit05.runbox.com [IPv6:2a0c:5a00:149::26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4L3MVx1bmQz3jng for ; Wed, 18 May 2022 18:50:17 +0000 (UTC) (envelope-from raf@rafal.net) Received: from mailtransmit02.runbox ([10.9.9.162] helo=aibo.runbox.com) by mailtransmit05.runbox.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1nrOkJ-00CdoV-Vm for freebsd-cloud@freebsd.org; Wed, 18 May 2022 20:50:08 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=rafal.net; s=selector2; h=To:References:Message-Id:Content-Transfer-Encoding:Cc:Date: In-Reply-To:From:Subject:Mime-Version:Content-Type; bh=rubAmpzLg/5zY/67hhsy8QoqBgauYNYRX8CsVugt6zo=; b=Kle5nrNoEWaWIixl6VcfQxz0oX 2k5vLXu55gHNXbFSZDi5HUBMCEJ/Hj9RkDhgeKKLwLJKBs8IuRUP8coAtJ8jVrk67Yy36bF3gTSTo uTwqKlsmoJo5lS9bdjr/dlDEIAwT0eQsTm2jnpc+31fIwYSiSL1ZK+1XUj+nk0SAphMVaf7PYTlSj NzEAun6HP3wody9GPr4zm8sLTvYNH0rrv7PnUjSUcxinB/l+KrvoYrp4ILB1ln3Ekuq1OipbI7W6s f84QFzZE8LcSUY4lYsZWMHMgXoDxhXO7XbXN7aKE3rIyV+8//sqgucNJkBpSD9g5CVtYqYnbjCpFE fA6es2Ng==; Received: from [10.9.9.74] (helo=submission03.runbox) by mailtransmit02.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1nrOkJ-0003Th-Cf; Wed, 18 May 2022 20:50:07 +0200 Received: by submission03.runbox with esmtpsa [Authenticated ID (1087189)] (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) id 1nrOk5-0006lk-B3; Wed, 18 May 2022 20:49:53 +0200 Content-Type: text/plain; charset=utf-8 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) Subject: Re: AMI Builder 13.1? From: Rafal Lukawiecki In-Reply-To: <01000180d86594b7-d1deb1c3-0663-48ba-a9e2-a115f7a08fd9-000000@email.amazonses.com> Date: Wed, 18 May 2022 19:49:52 +0100 Cc: freebsd-cloud@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <19999F85-BE49-4B58-AECC-73873E8A5AAD@rafal.net> References: <01000180d86594b7-d1deb1c3-0663-48ba-a9e2-a115f7a08fd9-000000@email.amazonses.com> To: Colin Percival X-Mailer: Apple Mail (2.3696.100.31) X-Rspamd-Queue-Id: 4L3MVx1bmQz3jng X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=rafal.net header.s=selector2 header.b=Kle5nrNo; dmarc=pass (policy=reject) header.from=rafal.net; spf=pass (mx1.freebsd.org: domain of raf@rafal.net designates 2a0c:5a00:149::26 as permitted sender) smtp.mailfrom=raf@rafal.net X-Spamd-Result: default: False [-2.60 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[rafal.net:s=selector2]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a0c:5a00:149::26]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[rafal.net:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[rafal.net,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MLMMJ_DEST(0.00)[freebsd-cloud]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:50304, ipnet:2a0c:5a00::/29, country:NO]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[2a0c:5a00:149::26:from] X-ThisMailContainsUnwantedMimeParts: N > On 18 May 2022, at 19:19, Colin Percival wrote: >=20 > On 5/18/22 06:21, Rafal Lukawiecki wrote: >> I am working on our FreeBSD today and this week=E2=80=94after a long = break. Do you have new AMI Builders for the latest 13 release? I intend = to run the builder on c6g.8xlarge and to use the produced AMIs on = t4g.medium. >=20 > Not yet but it will happen. I only have a few "paid" hours for = FreeBSD/EC2 > work (cf. https://patreon.com/cperciva ) and my top priority right now = is to > prepare for BSDCan (where I'll be talking about speeding up the boot = process). No worries, Colin. I have used the 13.0 and built 13.1 from a git clone = of the repo. Testing under way=E2=80=A6 -- Rafal Lukawiecki Data Scientist=20 Project Botticelli Ltd From nobody Mon May 30 16:11:14 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 921091B58140 for ; Mon, 30 May 2022 16:11:17 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from mail.tarsnap.com (mail.tarsnap.com [54.86.246.204]) by mx1.freebsd.org (Postfix) with SMTP id 4LBgPv4q94z4f31 for ; Mon, 30 May 2022 16:11:15 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: (qmail 58117 invoked from network); 30 May 2022 16:11:15 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by mail.tarsnap.com with SMTP; 30 May 2022 16:11:15 -0000 Received: (qmail 95611 invoked from network); 30 May 2022 16:11:14 -0000 Received: from unknown (HELO ?127.0.0.1?) (127.0.0.1) by localhost with SMTP; 30 May 2022 16:11:14 -0000 Message-ID: <79cce2d6-52b8-9451-3210-57e5439b5000@freebsd.org> Date: Mon, 30 May 2022 09:11:14 -0700 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 From: Colin Percival Subject: Re: AMI Builder 13.1? To: freebsd-cloud@freebsd.org, Rafal Lukawiecki References: <2e967b48-0521-a5bd-2230-a1caecf665b1@freebsd.org> Content-Language: en-US In-Reply-To: <2e967b48-0521-a5bd-2230-a1caecf665b1@freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4LBgPv4q94z4f31 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=softfail (mx1.freebsd.org: 54.86.246.204 is neither permitted nor denied by domain of cperciva@freebsd.org) smtp.mailfrom=cperciva@freebsd.org X-Spamd-Result: default: False [2.45 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[cperciva]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; GREYLIST(0.00)[pass,meta]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-0.52)[-0.515]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; VIOLATED_DIRECT_SPF(3.50)[]; NEURAL_HAM_SHORT(-0.53)[-0.534]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[54.86.246.204:from]; MLMMJ_DEST(0.00)[freebsd-cloud]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:54.86.0.0/16, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N [Re-sending since my first email didn't get through to the list...] On 5/18/22 11:21, Colin Percival wrote: > On 5/18/22 06:21, Rafal Lukawiecki wrote: >> I am working on our FreeBSD today and this week—after a long break. Do you >> have new AMI Builders for the latest 13 release? I intend to run the builder >> on c6g.8xlarge and to use the produced AMIs on t4g.medium. > > Not yet but it will happen.  I only have a few "paid" hours for FreeBSD/EC2 > work (cf. https://patreon.com/cperciva ) and my top priority right now is to > prepare for BSDCan (where I'll be talking about speeding up the boot process). AMI Builder AMIs are now available: https://www.patreon.com/posts/67054688 -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From nobody Wed Jun 8 00:20:02 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AB8D183F5C7 for ; Wed, 8 Jun 2022 00:20:12 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LHntM1XGgz4lkT for ; Wed, 8 Jun 2022 00:20:08 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id EA7073200BD6; Tue, 7 Jun 2022 20:20:04 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 07 Jun 2022 20:20:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:sender:subject :subject:to:to; s=fm1; t=1654647604; x=1654734004; bh=UfcqAs1Nsy b3LAJPGQY40c5jLn6j/menkt1BgpGaNks=; b=WrhmFDvOqeTB/ErjCimHJZ0CiA ndlbaeM+yHSAUAcqowEmigUqi2AW+4y+zi3Ix6MJSDXd2mjk5LGTiw54o7V88RSm gLz8VZBZhYiHCaCiB/sWutiL2vjrorjMNcAX4hGPe46hivzYhzSzcrDi4HUcdXb4 2zuJl1GaRFOQqxu0ubluXs6OvQq7eHgTWTDPoDuLue0eixy129cukWhUp+wSe1qa 3cIN6g1narixlpXn1QuGlVQYNr8LeP7zufllXMZ8PFj1cIO1bYCZRHlBGA7QmQuJ dj9Oqi9eRiDFP2+BeMeu/kRzD5pHixG5Ub9XS9B7s8o1CE3cxKLn8VWlTaAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1654647604; x=1654734004; bh=UfcqAs1Nsyb3LAJPGQY40c5jLn6j /menkt1BgpGaNks=; b=Sz9NdPCndYdIXKToKlw0tvePorZ1LqQgc/j0YwuDSRrU Py+/2k0x3jPUyFQEsOU4FPUmhxbDSLwLfltKX2fDYpRJa/f9s7khFiHyVOny0NrM sWnC9YhLHwRy977RceVhvG+qZO8HV1g1OeUaulr21beITUH5vR1BJBJUOMrTxM32 YgUbyo5NOJDctzWIiRLOwnYCxdCNIY+dHsawdX5bnzg+AnrP2MYAKt4sYHMgS+hm df5CF+fp2yRfH6LWKEC8zfrQVg93oZdj0RGByD5KgiPcXk1VhuwHD/WPCCSMG0oR N94KE3CFjzHxbT9euGi/fQ+N6sKM/ul1yYyM/Llnzg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedruddtiedgfedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffokfggtgfgsehtkehmtd ertdejnecuhfhrohhmpedfrfgrthcuofgrugguohigfdcuoehprghtsehprghtmhgruggu ohigrdgtohhmqeenucggtffrrghtthgvrhhnpedvkeehffekvdfgjefggedtgfdugeejte evkeehudeuveetgeehgeeukedttddvgfenucffohhmrghinheplhhotggrlhhsthgrtghk rdgtlhhouhgunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homhepphgrthesphgrthhmrgguughogidrtghomh X-ME-Proxy: Feedback-ID: i8b6c40f9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 7 Jun 2022 20:20:03 -0400 (EDT) From: "Pat Maddox" To: freebsd-cloud@FreeBSD.org Subject: anything like localstack on FreeBSD? Date: Tue, 07 Jun 2022 17:20:02 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4LHntM1XGgz4lkT X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm1 header.b=WrhmFDvO; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=Sz9NdPCn; dmarc=none; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 64.147.123.19 as permitted sender) smtp.mailfrom=pat@patmaddox.com X-Spamd-Result: default: False [-3.58 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-0.98)[-0.976]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm1,messagingengine.com:s=fm2]; FREEFALL_USER(0.00)[pat]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; DMARC_NA(0.00)[patmaddox.com]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; MLMMJ_DEST(0.00)[freebsd-cloud] X-ThisMailContainsUnwantedMimeParts: N This feels like quite a long shot, but here we go... We are developing applications that run in AWS. So this question isn’t quite about running FreeBSD on AWS, but rather about developing applications that run on FreeBSD on AWS. We use localstack [1] to mock AWS services we care about locally. It uses docker. Is there anything like localstack on FreeBSD, that simulates AWS services (S3, SQS, secrets manager)? Pat [1] https://localstack.cloud/ From nobody Wed Jun 29 10:15:48 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 205EC878DCF for ; Wed, 29 Jun 2022 10:15:52 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LXy5z2HYmz3jng for ; Wed, 29 Jun 2022 10:15:51 +0000 (UTC) (envelope-from mpp302@gmail.com) Received: by mail-ej1-f54.google.com with SMTP id cw10so31600160ejb.3 for ; Wed, 29 Jun 2022 03:15:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:in-reply-to :content-transfer-encoding; bh=pyqtG7ePPCP97uJrlnk1sySIBRG+XBz3/aPQwdEjHOQ=; b=Rk5CwfLECWvggz4/9Ew57+5RzSHr1j8uzvTNTX7BExYXFMvr4eoVljt9LraqwVFcYu at+dBI6rUI6aupsbXf9mh5EHmlWhygyqjJsSez3/QFtOBsX3cnMjN0BRzIFdrD/QO5xs qG5ZRxlVFPTdg6Mh++T1z2/hrONkbTN8JTYMyIrEXDubu3iRZi3fEfogX/3rnPOOwDLJ M7QPTrlkdl8kJEJtR2w4X68Eex0qmmtVR+vBsNBsFo/49j6+wEalQSaKzW1s/3qaGW2A BSawQN1EzblpZUNoCtpqwlHijiO/3Ee5vQIoYkH0a3DJhNaTs4eRMyz6piTJnT8+UV8m 7w7Q== X-Gm-Message-State: AJIora8VNcP9z7zEx+v1dtFJargiyrUd9vpdXr+C+jEowu6AanaTBH+l JVXdGqAZZqCdYtSVCEpRdclI0KWNqTpFKg== X-Google-Smtp-Source: AGRyM1tObxsHa6JvrXAGQFrhFDYQEfSwCvU37k4e6qo/WzEfLu6D0LnwhqUF09xHCB39EPonE81bxg== X-Received: by 2002:a17:906:c151:b0:726:95c0:85dc with SMTP id dp17-20020a170906c15100b0072695c085dcmr2522800ejc.118.1656497750404; Wed, 29 Jun 2022 03:15:50 -0700 (PDT) Received: from ?IPV6:2a02:8109:8680:1304:5e5f:67ff:fef4:ffd8? ([2a02:8109:8680:1304:5e5f:67ff:fef4:ffd8]) by smtp.gmail.com with ESMTPSA id q22-20020a170906941600b0072660d8d83fsm5524782ejx.188.2022.06.29.03.15.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 29 Jun 2022 03:15:49 -0700 (PDT) Message-ID: <2fd47497-6be4-df0d-6482-5cda5e1c2d98@FreeBSD.org> Date: Wed, 29 Jun 2022 12:15:48 +0200 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: anything like localstack on FreeBSD? Content-Language: en-US To: Pat Maddox , freebsd-cloud@FreeBSD.org References: From: Mateusz Piotrowski <0mp@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4LXy5z2HYmz3jng X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mpp302@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=mpp302@gmail.com X-Spamd-Result: default: False [-0.07 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; RWL_MAILSPIKE_GOOD(0.00)[209.85.218.54:from]; RCVD_COUNT_THREE(0.00)[3]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[0mp@FreeBSD.org,mpp302@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[0mp@FreeBSD.org,mpp302@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.94)[-0.937]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.87)[0.868]; NEURAL_HAM_LONG(-1.00)[-0.998]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_NA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; SUBJECT_ENDS_QUESTION(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.218.54:from]; MLMMJ_DEST(0.00)[freebsd-cloud]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N 'Hey Pat, On 08/06/2022 02:20, Pat Maddox wrote: > Is there anything like localstack on FreeBSD, that simulates AWS services (S3, SQS, secrets manager)? > > [1] https://localstack.cloud/ > Hmm, I'm not aware of anything like that for FreeBSD. People had some successes with running the docker-machine the same way you can do it on macOS. Other than that, I've heard there is some movement in the containerd world related to FreeBSD. Best, Mateusz Piotrowski From nobody Sat Jul 23 05:06:50 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LqZ6Q15D3z4WcH4 for ; Sat, 23 Jul 2022 05:06:54 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LqZ6P1T0Gz3Ln4; Sat, 23 Jul 2022 05:06:53 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 1F8145C00CA; Sat, 23 Jul 2022 01:06:52 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Sat, 23 Jul 2022 01:06:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=cc:cc:content-transfer-encoding:content-type:date:date:from :from:in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1658552812; x= 1658639212; bh=sBgJkCzn2IHBTCAdM+FXsA9nF+/fWuGQXBwowut+paA=; b=F HZzzNseuVkCJLMV8NHUGnuku0iUkKLP3FL8C93owfeyjvMMx+TXBzke2BnXrgF0x u5+ZpmWzvxSXB4x+REUlazt3SHN+fv24sip5NXaAC1ocnlqcanyrmQqjAto+93hg O8cywkjweSVU/JHP9ylMCZBMSzlBR12+aY08ndLCTWpd9UmWe8PrM2ydo9NGT7EU CWw8PVaC4Uc5VXxUR8cJ5mrsDGBQXmlU2Ws42O57DArRT0rNKUbLtlKzB0nNMzd4 zTMv9KcPcw+w/9cAOK0gUwZp9suI2USGs2Y/AFabEncDjaX7GMEAt1Z94qM4dapi rLKgauuhMnAgVJcxSYwJw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1658552812; x= 1658639212; bh=sBgJkCzn2IHBTCAdM+FXsA9nF+/fWuGQXBwowut+paA=; b=c nMHy6iNBwlMLfQl+4/mgZaPR3BERGXjn1rENkM7Ezgm9FFMLhYQnwTM5eW2wad4z CfzH54SnTjNeef/FDxdHGF2oyZ6b5pbfK8ykuOMcFGT3TZ8HK0HwXkf/ihlW9kN5 gh/pFYfLhFrecN6vQYkaPwO8aTDYWaez6IHkZe4L1ta4zh/enJhnMb6w7EgkMa79 kMefp8jg22iktYZHKDigGHyAcFPseDHCtGmeZ57PeqLCzbWN6jR1ptrj65eD7mkU +hBg5FRD+wUpWGdZE5XU62hq2LdsZU0hyz/HWx2eRw9X/m5erstkjDFLWKP7tPJX YswQv5Sn3tHc3iTRi4GfA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvddtfedgleegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffoffkjghfgggtgfesth ekmhdtredtjeenucfhrhhomhepfdfrrghtucforgguughogidfuceophgrthesphgrthhm rgguughogidrtghomheqnecuggftrfgrthhtvghrnhepvdegleefgffgteduleelledvie eihfeuudehueefudegkedvgfegieekudekleegnecuffhomhgrihhnpehlohgtrghlshht rggtkhdrtghlohhuugenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrih hlfhhrohhmpehprghtsehprghtmhgrugguohigrdgtohhm X-ME-Proxy: Feedback-ID: i8b6c40f9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 23 Jul 2022 01:06:51 -0400 (EDT) From: "Pat Maddox" To: "Mateusz Piotrowski" <0mp@FreeBSD.org> Cc: freebsd-cloud@FreeBSD.org Subject: Re: anything like localstack on FreeBSD? Date: Fri, 22 Jul 2022 22:06:50 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: <08687776-B3E8-4DC7-8C37-3603ECE76364@patmaddox.com> In-Reply-To: <2fd47497-6be4-df0d-6482-5cda5e1c2d98@FreeBSD.org> References: <2fd47497-6be4-df0d-6482-5cda5e1c2d98@FreeBSD.org> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4LqZ6P1T0Gz3Ln4 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm2 header.b="F HZzzNs"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="c nMHy6i"; dmarc=none; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=pat@patmaddox.com X-Spamd-Result: default: False [-3.70 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.26:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[pat]; DMARC_NA(0.00)[patmaddox.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org] X-ThisMailContainsUnwantedMimeParts: N On 29 Jun 2022, at 3:15, Mateusz Piotrowski wrote: > 'Hey Pat, > > On 08/06/2022 02:20, Pat Maddox wrote: >> Is there anything like localstack on FreeBSD, that simulates AWS >> services (S3, SQS, secrets manager)? >> >> [1] https://localstack.cloud/ >> > Hmm, I'm not aware of anything like that for FreeBSD. People had some > successes with running the docker-machine the same way you can do it > on macOS. > > Other than that, I've heard there is some movement in the containerd > world related to FreeBSD. Yes I’ll be curious to see how containerd works on FreeBSD and if it supports localstack. In the mean time, we’ll be running a dedicated docker server for localstack... a remote localstack. Pat From nobody Sat Jul 23 08:33:03 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LqfhL43msz4X4FN; Sat, 23 Jul 2022 08:33:06 +0000 (UTC) (envelope-from bogdan-lists@neant.ro) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LqfhK69Lfz3Zfp; Sat, 23 Jul 2022 08:33:05 +0000 (UTC) (envelope-from bogdan-lists@neant.ro) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 67CD35C00BC; Sat, 23 Jul 2022 04:33:05 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Sat, 23 Jul 2022 04:33:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neant.ro; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:sender:subject :subject:to:to; s=fm3; t=1658565185; x=1658651585; bh=UXBBzg3cvI owHKiAVE84bpcjil0hhP+WhPNmicmAEaQ=; b=nvdlyQnAsWzSZC5sTC399+rKTS puENflWsweOm2dncfrjBl1kS6THU1G3NzNQsZ77azlu5e+k0Fpdl1DuK6OqiRGOS Ym/HQqbRVg6KvmlNBAdt4LURlcaxRECsupQccFVRza2DngvmS4UhHyuMgb8vkf1H /RfF6tdMaCmToo4qahBMLmP9vVMVrgUTec/Lyfeflv7Iv8vG1QHgTEZF3SgAyKEB mXX6BxHKnFmthtwd9u24jwEc/Kg/lViQkFmD05eRapZAOB3ErPATH1HhS1XJfg4l KxAmymNYNVJRsM/kichZywPrwrLvTZQ8JwhjqPWC/sYwli5B1pFdm+MF0HQQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1658565185; x=1658651585; bh=UXBBzg3cvIowHKiAVE84bpcjil0h hP+WhPNmicmAEaQ=; b=PPKCtEdRrPcy+nL9c9oWJKHlbO3Npm6KcrMCJQO6V7e9 G8NXck2QsW5X/ykoaeVIweNk6rreBE30cKiaUjQ3J8FXUFc39iyGWVWsT2jwXHl/ 13FuSRyIPPYtmi6A0nA/sXp4PegJ63SIct0U+tV9rQOHBdN0nP9heozzxng+dngL 0BN21mPDx0i8f/W64JDqacvDmhAlTDn+QpJUlYA0MwRtWs0RXxJgGhbEDoZYhcAn WRCHUdYCvItAR7PHv1hLD9qjVPXTU2SAacBFNzbCY9Nsb64ZUa3+uhyo+UErae9j 3I5wN10K9ut6VxfYxWNQz0BKMyWZ5/lGI5D5Bu5CqA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvddtgedgtdejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhtgfgggfukfffvffosehtqhhmtd hhtdejnecuhfhrohhmpegsohhguggrnhdqlhhishhtshesnhgvrghnthdrrhhonecuggft rfgrthhtvghrnhepkeevvdekheehleefteefheetteefueehkeffgeeitdefueegteeuje dvueevueejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhepsghoghgurghnqdhlihhsthhssehnvggrnhhtrdhroh X-ME-Proxy: Feedback-ID: i61d94637:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 23 Jul 2022 04:33:04 -0400 (EDT) From: bogdan-lists@neant.ro Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) Subject: AWS - UFS corrupted when restoring from AWS Backup service Message-Id: <11F07F6C-E93B-42E3-BD27-3FEC4E342B1A@neant.ro> Date: Sat, 23 Jul 2022 11:33:03 +0300 To: freebsd-fs@freebsd.org, freebsd-cloud@freebsd.org X-Mailer: Apple Mail (2.3696.100.31) X-Rspamd-Queue-Id: 4LqfhK69Lfz3Zfp X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=neant.ro header.s=fm3 header.b=nvdlyQnA; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=PPKCtEdR; dmarc=none; spf=pass (mx1.freebsd.org: domain of bogdan-lists@neant.ro designates 66.111.4.29 as permitted sender) smtp.mailfrom=bogdan-lists@neant.ro X-Spamd-Result: default: False [-4.10 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[neant.ro:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org,freebsd-fs@freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; FROM_NO_DN(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[neant.ro:+,messagingengine.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DMARC_NA(0.00)[neant.ro]; RCVD_VIA_SMTP_AUTH(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hello, TL;DR: We have a bunch of EC2 machines in AWS running FreeBSD. AMI from = Market, file system is UFS. We have AWS Backup service taking hourly = snapshots of these machines (AMI + EBS snapshots I believe). After a few = months of snapshots we had to restore one of them and found out that the = file system is corrupted and fsck was not able to recover it. We are = going to enable sync in fstab, see if that helps, but it=E2=80=99s hard = to know because it is hard to reproduce the problem, and details about = how everything works are fuzzy to me. Longer version: We use FreeBSD on web servers in AWS. Until January we were doing weekly = AMI snapshots by running a script that would shut down the machine, = create the AMI, then start the machine back up. Which worked for a long = time, but is less than ideal and shutting down production more often = than weekly is rude. At the start of this year we switched to running AWS Backup hourly. It = takes snapshots of a running machine without stopping it. I believe = it=E2=80=99s the same as creating an AMI and checking the =E2=80=9CNo = reboot=E2=80=9D checkbox. It should use the same API call, but I = wouldn=E2=80=99t know. We ran a few recovery tests, we read the docs, we = confirmed with support, everything looked like it should work with no = issues. A couple of weeks ago the EBS disk on one of the machines failed and we = needed to restore it. When we did, it ran fsck on boot (which it = didn=E2=80=99t on our previous tests) and failed to recover it, so the = machine was effectively dead. I know we can mount the disk on a = different machine and recover (some) data, that=E2=80=99s not the point. = We tried a few backups going back two weeks, same issue. We tried a few = more instances, about 5, all of them ran fsck on boot. A couple were = recovered, but it doesn=E2=80=99t matter, it still means it=E2=80=99s = not working as we thought. So now we=E2=80=99re effectively running = without backups on EC2 instances. I=E2=80=99m not sure why it happens. Information is sparse and I=E2=80=99m= making a lot of assumptions. Basically I believe that the snapshot = process is equivalent to cutting off power to the machine and that = happens every hour for months. The docs on UFS soft updates say that = there=E2=80=99s a small chance of data loss, but since that = power-cutting snapshot happens every hour over a time of months, that = chance isn=E2=80=99t that small any more. Still, apparently Linux = doesn=E2=80=99t have this problem, and everywhere I read it says that = data might be lost, but the file system should not be corrupted. And yet = fsck isn=E2=80=99t always able to recover it. As far as I understand, with soft updates and =E2=80=9Cnoasync=E2=80=9D = in fstab (default), data is flushed to disk about every 30 seconds = (according to syncer man page), asynchronously, while metadata is = written synchronously. I=E2=80=99m thinking that maybe that=E2=80=99s an = issue and turning on sync in fstab might help. On the other hand, the = man page for syncer says =E2=80=9CIt is possible on some systems that a = sync(2) occurring simultaneously with a crash may cause file system = damage.=E2=80=9D, which means it might make it worse? I don=E2=80=99t = know. We were not able to reproduce the problem reliably so that we can test. = I=E2=80=99m not sure if or how anyone can help. I just wanted to send = this message so that at least some other people are aware that AWS = Backup doesn=E2=80=99t play nice with FreeBSD.= From nobody Sat Jul 30 22:07:23 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LwJQm08WDz4Xs35 for ; Sat, 30 Jul 2022 22:07:28 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LwJQl00WDz471G for ; Sat, 30 Jul 2022 22:07:26 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id C32B05C0052; Sat, 30 Jul 2022 18:07:25 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Sat, 30 Jul 2022 18:07:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:sender:subject :subject:to:to; s=fm2; t=1659218845; x=1659305245; bh=Bp2qAXuFww bBcLuJ/iXqYFAab1D+UDoekZzrczvJRog=; b=As0XoKfwpWcFwcVwt1SergwmeN 2nWJYjwu2UgnAOMIZ++mzLFEFX0hLRc17oAu3QaDtIJMDOjMLEmAm9SS+6rgxVQR SAlQLMAGZ1DpJmMSOMMpheb8TqojCFnffI3CPeqiJLaP1Z0d2kPn7KjqjgKoGHy3 QNnmsTWD8hV6xPyFgMj/G1fyvxedPz8CcHKitL9l0BMd9eMI00c5ojroXk1wmZNs ++1yJeeOig9uvg1muttsvT6KVo8opqQd3pxSO/g9pxoF7KJjk6JXykfynC7GfF4B bhNMg1LJExOEGRRH5b7WPi15nm0XtSk2ZsIlIxefLPA8Ne+S+WvEBUOdLtnA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1659218845; x=1659305245; bh=Bp2qAXuFwwbBcLuJ/iXqYFAab1D+ UDoekZzrczvJRog=; b=QK+j8brqr9uoFwNPgdxVQuuCT59UyWIbM3ALZkHOWP8G I55ZPXQeNaU83VfJrL7uHCbotdVf4W41zs2l/bAIGvXUiQOPhA8UDKXC2/weLBzh AWph7lP452x0LKL6//qTh4DvTP3JAWSXJT3yPmDpHs6jvgS9TPwB3tbeeLpkWjFZ n2REQJ7OlzpfWBuGiQQTDQtc+ZtMy/BTe3oOdhAtiFz94OXvLJ4nrbjy++/f9SG4 mVWlKnLKKXR+OgVt3qf6e4HzRS9Dfw50SK4aNWwktGQryxxGzLR82MrQmpjboyyx CixY6KY6fBvGSr9KYirDyOet6y8PcwXCkkpbdmfB5A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvddvtddgtdeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffokfggtgfgsehtkehmtd ertdejnecuhfhrohhmpedfrfgrthcuofgrugguohigfdcuoehprghtsehprghtmhgruggu ohigrdgtohhmqeenucggtffrrghtthgvrhhnpeegheeugfekffejudelueegteevueefve dvfeejtedvkefhtdeujedtueeifeekieenucffohhmrghinhepghhithhhuhgsrdgtohhm necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphgrth esphgrthhmrgguughogidrtghomh X-ME-Proxy: Feedback-ID: i8b6c40f9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 30 Jul 2022 18:07:25 -0400 (EDT) From: "Pat Maddox" To: freebsd-cloud@FreeBSD.org Subject: GCP ZFS-on-Root (working, and some questions) Date: Sat, 30 Jul 2022 15:07:23 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: <360A7428-8A37-428D-BACE-6335A7AA6554@patmaddox.com> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4LwJQl00WDz471G X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm2 header.b=As0XoKfw; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=QK+j8brq; dmarc=none; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=pat@patmaddox.com X-Spamd-Result: default: False [-4.70 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.26:from]; FREEFALL_USER(0.00)[pat]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[patmaddox.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org] X-ThisMailContainsUnwantedMimeParts: N Hey there, I have taken some of the concepts and scripts that Colin used to make the AWS AMI Builder (thank you!!), and used them to configure a ZFS-on-Root image for GCP. Here’s the script: https://gist.github.com/patmaddox/1a0be973ecfce4f056c517abb7746286 I did it slightly differently from the AMI builder. Instead of working in a mem disk, I just fetch base.txz and expand it to a dir, edit the files, etc. One other difference is that because base.txz doesn’t include a kernel, I `cp -Rp /boot/kernel/ /mnt/boot/kernel/`. It boots, I can SSH and do all the usual things, so far I am very happy. My questions: 1. Is there anything I might be missing with the approach of extracting base.txz, and copying over the current instance’s kernel? 2. If I change the GCP disk size and reboot, the partition table is corrupt. I need to `geom part recover da0 && geom part resize -i 2 da0` to resize it. How can I have that happen automatically during the boot process, the way that the AMI does? Pat From nobody Sat Jul 30 22:50:15 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LwKN95sXgz4XyT6 for ; Sat, 30 Jul 2022 22:50:17 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LwKN90wQlz3CL0 for ; Sat, 30 Jul 2022 22:50:17 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 0B46D5C0097; Sat, 30 Jul 2022 18:50:17 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Sat, 30 Jul 2022 18:50:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1659221417; x= 1659307817; bh=WfyYtrrALxivqeLDfP1Xde4VkJ/c6BfJQsrjc8RZdWQ=; b=r 3mkjPaYpkrSfOXusZtUp9YBdOeMlnCQXbrP2qU69P4UBK9kMdOuwtanr3fZ8jb2A ddgCyvrVzDdAsR43sM1nitY024F+hWhgaKIE6MfERs5XktSUiHok8aMXk0lo84KF cHxLzEMprdLU3xaku3T1ilRhB6O9VsD/DAxpzJXuzPaVrksKv61m1jHUwQMXs7VJ rPo3aIbvsFfyCkaDmMzWsP182RQ1tzFvorCncKX6LQau4NVpKRX6WLMlIhWpt3Jl Xqfn5BlRkkausVP//HlVY46PSUrWMdA7YzF9jU3Sum/6rZshsghhWlKOw+Iy+Cke DoDbc1OM7vp5VTLU2hLiA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; t=1659221417; x=1659307817; bh=W fyYtrrALxivqeLDfP1Xde4VkJ/c6BfJQsrjc8RZdWQ=; b=AxssVMPc9hfNHKli3 TdIHV9RJhm3CNoZcQNxfi1PVc50brNZJEFyymH3Rh0qEXYe/0LEwp9ryglcnpAnb AcnzfeouqjiCa4wO+u8PLbP5sqGwkHRKO8hmtWgstQxH1hLBUWO2X/ixii26yjEy zfKyf5oLTc8oGa542ex3wqLzLUYDmyKpD5De/iA4RWxf6GbEPdPfmpcHgJp74U7H g4je7HliijvZPV2EMmGPU7NOHY/pTwuy4jp6XSGRd+kope3UhEXcjU3fQNgmnRN5 pWelR6nhenqSETmWGnli3MWWa3MqmEW0rVavrZ5B0PgBtZtNKLR9Hqkbr2BwvZiL ql38Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvddvtddgudehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffokfgjfhggtgfgsehtke hmtdertdejnecuhfhrohhmpedfrfgrthcuofgrugguohigfdcuoehprghtsehprghtmhgr ugguohigrdgtohhmqeenucggtffrrghtthgvrhhnpeeifeehkeeggeefheehffeijeffte efgfduteelieevteevvefhhfdukeeltdethfenucevlhhushhtvghrufhiiigvpedtnecu rfgrrhgrmhepmhgrihhlfhhrohhmpehprghtsehprghtmhgrugguohigrdgtohhm X-ME-Proxy: Feedback-ID: i8b6c40f9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 30 Jul 2022 18:50:16 -0400 (EDT) From: "Pat Maddox" To: freebsd-cloud@FreeBSD.org Subject: Re: GCP ZFS-on-Root (working, and some questions) Date: Sat, 30 Jul 2022 15:50:15 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: <94280CD0-37C5-44E3-88CC-6DA580E41229@patmaddox.com> In-Reply-To: <360A7428-8A37-428D-BACE-6335A7AA6554@patmaddox.com> References: <360A7428-8A37-428D-BACE-6335A7AA6554@patmaddox.com> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4LwKN90wQlz3CL0 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm2 header.b="r 3mkjPa"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=AxssVMPc; dmarc=none; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=pat@patmaddox.com X-Spamd-Result: default: False [-4.70 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.26:from]; FREEFALL_USER(0.00)[pat]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[patmaddox.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org] X-ThisMailContainsUnwantedMimeParts: N On 30 Jul 2022, at 15:07, Pat Maddox wrote: > 1. Is there anything I might be missing with the approach of > extracting base.txz, and copying over the current instance’s kernel? Silly - I can download and extract kernel.txz instead of copying the current kernel. Pretty sure base.txz and kernel.txz are all I need. Pat From nobody Wed Aug 10 19:38:40 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M30c56pdmz4XxdG; Wed, 10 Aug 2022 19:38:45 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4M30c45KHjz3jr5; Wed, 10 Aug 2022 19:38:44 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 4C7F53200488; Wed, 10 Aug 2022 15:38:43 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 10 Aug 2022 15:38:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=cc:content-type:date:date:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to; s=fm3; t= 1660160322; x=1660246722; bh=HkXG15+if3L2qyHkddcrE4hM1WtFOwgujgM onASsiK0=; b=fuAWNhg3ry9rzUQ68t1krY0iVgxwZE4/0Di0795lf/GFjtmGRBI IGmUYz6t6hj8Punvi4hc5gHaVagWKq2+wS/VKUH9lQqM27Go1jO3fcGxhIyFnLU5 ZCj71T6Ulk49PTtKV9O/mHgS8hBA3JfCQBRLV3YZ6HkWLaP3/VxVk0yLoY7ZwMmW /yhbqLFzOW0BlyCZ8afPInjUKwdU3Bly52fHf7XDfrZLzTQux/RndjBiK7aQ8MDJ iQ48I3xQWkx5L0ghouK63GVHZ39g9evvfw6uyt+9wgbVPcrp5N0BFnIdWQWA82Lp CLo3brEnLTKvOqxps9nzGjHt4CCOjn05PxA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:message-id:mime-version :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1660160322; x= 1660246722; bh=HkXG15+if3L2qyHkddcrE4hM1WtFOwgujgMonASsiK0=; b=r n1c4lJ4T9dhQ6twOtl12/5aigwh9XUpUoIts2PfstEVbMSYYVdzTMBaf35cY/8Xd qARHY57t1olCVUnn6/dIg4SF2KSDEgiIFLVhecg3l6qhX2V6M6g/E4ARfV/hlTxH f5bYLIbBMOiqugdJj2Bja+nlSwBwMKpDqGEpFaBJZy+oPc546ZXNzeOceqKwEsXx fLaj7APSrGox/0xY2XPCJgNJ+niOUnYAippTUdSQTEScA437RU6uew+EB+PSTpp3 EVLNC0grweXAxncVrfR4+og194A7gNp+DwewtARBFgt7zMMPZFKbCT8DJcgfTgvs Mh9fkgNzkys/AHt36WN1A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdegvddgudefkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufffokfggtgesthdtmhdtredttdenucfhrhhomhepfdfrrghtucfo rgguughogidfuceophgrthesphgrthhmrgguughogidrtghomheqnecuggftrfgrthhtvg hrnhepueduleeuvedvkeehuedtjeegieehheeltdefjeegtdekheegtdeiheefhedvueeh necuffhomhgrihhnpegrmhgriihonhdrtghomhdpghhoohhglhgvrdgtohhmnecuvehluh hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphgrthesphgrthhm rgguughogidrtghomh X-ME-Proxy: Feedback-ID: i8b6c40f9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 10 Aug 2022 15:38:42 -0400 (EDT) From: "Pat Maddox" To: "Freebsd Questions" , freebsd-cloud@FreeBSD.org Subject: How are the official cloud images created? (are there scripts available?) Date: Wed, 10 Aug 2022 12:38:40 -0700 X-Mailer: MailMate (1.13.2r5673) Message-ID: <02105388-B73F-4075-B4BD-F0A6734E2C79@patmaddox.com> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; format=flowed; markup=markdown X-Rspamd-Queue-Id: 4M30c45KHjz3jr5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm3 header.b=fuAWNhg3; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="r n1c4lJ"; dmarc=none; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 64.147.123.24 as permitted sender) smtp.mailfrom=pat@patmaddox.com X-Spamd-Result: default: False [-4.59 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.99)[-0.992]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm3,messagingengine.com:s=fm1]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.24:from]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org,freebsd-questions@freebsd.org]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_THREE(0.00)[4]; SUBJECT_HAS_QUESTION(0.00)[]; FREEFALL_USER(0.00)[pat]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[patmaddox.com]; RCVD_VIA_SMTP_AUTH(0.00)[] X-ThisMailContainsUnwantedMimeParts: N AWS has an official FreeBSD AMI [1], as does GCP [2]. How are those images created? I assume someone runs scripts to configure the machine / disk and then makes an image. Are those scripts available in git? Who is responsible for making those images, that I could contact for more info? Pat [1] https://aws.amazon.com/marketplace/pp/prodview-ukzmy5dzc6nbq [2] https://cloud.google.com/compute/docs/images#freebsd From nobody Wed Aug 10 19:49:31 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M30rg1wxQz4Y1cj; Wed, 10 Aug 2022 19:49:39 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M30rg1RDgz3lnS; Wed, 10 Aug 2022 19:49:39 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660160979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=2gvYH2Y0zVX9ri1JwYOME8jKKBoLCzaG8iBDpmzjHeI=; b=yb7NHVxDJBkopKEc3/TqBMtsyKCcyQyZtFOa+g8IC+stcrJRQPc+oHGYOQsGdib7+14Q+n SbxmcmwJ+moy8clFyXbg5g5K4i//LZH01YZ5T2zvWC4PAMSnicjvI4BAQMQ9h5cK4BeInP uFO/Er+T27fHpxH1x77rmHbEAyIZDfwWoq+hMe8HZ9sgPdvQ0vDexsOtE4xSY46+eyF1yz /3fFw9CA4164SNkxbiO8x6u6iwWzvnA4leYqugNSPsN+ezxptnxHx8zCHn1yRh7WfpfFro dkA2IpVzW8F5O5AdcD/iSqP+wX19+TCq9Y3qDj8PX7h2PzbeGUiCal13iJ+BhQ== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id A8A2B19D52; Wed, 10 Aug 2022 19:49:38 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Wed, 10 Aug 2022 19:49:31 +0000 From: Glen Barber To: Pat Maddox Cc: Freebsd Questions , freebsd-cloud@freebsd.org Subject: Re: How are the official cloud images created? (are there scripts available?) Message-ID: <20220810194931.GI30607@FreeBSD.org> References: <02105388-B73F-4075-B4BD-F0A6734E2C79@patmaddox.com> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="OmxNZ+k2SuGb8jfc" Content-Disposition: inline In-Reply-To: <02105388-B73F-4075-B4BD-F0A6734E2C79@patmaddox.com> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1660160979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=2gvYH2Y0zVX9ri1JwYOME8jKKBoLCzaG8iBDpmzjHeI=; b=bnG64KxLTf89ojmeZvb9Afme13zZTlbFslMUVS+FOBkceZtrlOBNuodmOy5ZRF5T8cpKXb FK7VuQAPxlZQ4xN8dxE9yJw9hc3d38vDdMTTzH5wMtb4cVL6xF/7Ez3qsV3tFAQv+JBdQk Li7aN7usXh4mC6MoQwz8I66iAgUXsCBR+wR0OzSqD+bQlKM/JBRYhBUbpTxU3IAuoR0zDd dQMSWmuzPs+yP8O/m1+FZqd7uRvub3ZCLAJMTUHjaHH73khfFwri3WSSQ9S6IZHmEvJCgf +Hp9WNfgEEvajtTlof2jmna5Jb7cIv4/c4rT27yQNIMq4+taHvFGcVxlFR/mGA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1660160979; a=rsa-sha256; cv=none; b=nFNMqu9GGZRIWisVMbV8S4aSaYq6ziIOmmElK52xH/ROsgCrbtKz91ZT+8zqZJAYUN8HFT nCjum9eXX1E68X3VHbXKja8cxsb6LLSKFEB0opSCc+NozZJ9DINvPiqf6oYTOSpONs/S60 kHRQqI3TY7gLAVhphhwq1xcIfPkno3QvrELZDHIDGEmm/5qe9ABmF7W7dBfMjXXs6BV7/6 qbIRqV8/BIjHJaVZMrAjv2RlaOSDQymvdLTqvHJ7SKFlZxF/INurBSM3nZThGSFy0/4WAC qN/hDW1nLYHXZ5OMUlmvDALXzAen38+W6t6G5hR25J3gdkwojcZ537nJc913FQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N --OmxNZ+k2SuGb8jfc Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 10, 2022 at 12:38:40PM -0700, Pat Maddox wrote: > AWS has an official FreeBSD AMI [1], as does GCP [2]. >=20 > How are those images created? I assume someone runs scripts to configure = the > machine / disk and then makes an image. >=20 > Are those scripts available in git? >=20 > Who is responsible for making those images, that I could contact for more > info? >=20 > Pat >=20 > [1] https://aws.amazon.com/marketplace/pp/prodview-ukzmy5dzc6nbq > [2] https://cloud.google.com/compute/docs/images#freebsd >=20 They are in git. src/release/Makefile.vm. The 'make list-cloudware' target will give information on the valid CLOUDWARE values. 'make vm-cloudware' is the main target. A short example of the usage is: # make WITH_CLOUDWARE=3D1 CLOUDWARE=3DEC2 vm-cloudware Note, you will need to provide the API keys and other variables yourself, such as AWSKEYFILE, AWSREGION, etc. See src/release/Makefile.ec2 for details on this particular platform. Glen --OmxNZ+k2SuGb8jfc Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmL0C8YACgkQAxRYpUeP 4pOzFRAAoCevt5+aqOnpgLy7bm1JheC4y78GgZrPatI0YEs5hk26t+UCzLrHkBAh 3I9lnAmc/bRyBzkQ+kiUTkQeIKWSIzrm2VhMt+VbgmlSDAQJwwPCWUk7Fk0c2utl AnQXfAABlFExFHhz8qRcqqqyMo9svdShdSyXzuYfHO8xHa5trwy1KpPrOFquNSdg WNR8GsQFRKZk3hNozMoKYCTUWkXziQ+CnWjDi3rcxIcOlfdzs+Sl5kEwTtm6mm2O kiHHmQJvUVYCqjNm9Z4ALzTMQoUGdPtNlMhei7PpOEVRqdTQbNxnsUCXKslwrR9V 0GHppCqSpyBSsir/mwj3H21zSrpHSxzvx++8GN7Ipk3yULBnatbC5TS7W8lchEcg s/91PHBiKOUxcsQHagLoYvmOEjdLPS+1lYxTBs/Kh0K3+zl5p21ARx+4oZ9D8Fl1 hV9vezopfCCdsQHjyRhRa5G8nHFn4++FUr/jW9L5kL4VXzfjUeFI0IZgSdIbTSHc hgWLtIKUXb2zKUKQrrSPdfxt3Iv2mHXeDDqQGu6Wv0+Pj/fWfdTVPKvVb333KY+l GX+FK3/nPovzmqRITBihUVawECDIZW+RRf43dl++BiOcpDLhmtxun3qRuqVPl1xX l/1EUAuFlBk/gCRL/Vezuk19MqUQGeXZk09isKdplXZdpO6xKKk= =mMFk -----END PGP SIGNATURE----- --OmxNZ+k2SuGb8jfc-- From nobody Wed Aug 17 11:49:35 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M75sc2Q7Fz4ZFFc for ; Wed, 17 Aug 2022 11:49:40 +0000 (UTC) (envelope-from bogdan-lists@neant.ro) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4M75sb47LFz44X8 for ; Wed, 17 Aug 2022 11:49:39 +0000 (UTC) (envelope-from bogdan-lists@neant.ro) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id B2A965C00E4 for ; Wed, 17 Aug 2022 07:49:38 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Wed, 17 Aug 2022 07:49:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neant.ro; h=cc :content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:sender:subject :subject:to:to; s=fm1; t=1660736978; x=1660823378; bh=QoWTOzEf+V tWntwfSkzsVQrH8LH6xp0381M3fiQBXO4=; b=fY28ALBqQ295OEZ1qFFDqIj3aA /W5ZVgiT+8MaRXexc6s8tXjhd/u37vYXkhkNii3ojpDq+1XxuX2dz3mqZOPWMnAU R3GAz/VyrHRkxxKBKoea5uV6JheKWrvaAQWuZsw0dOSwmzSURx3e2f07pxSk+IK2 zVxxP2Wy+IJsd2aIRLbkRz53NjekXntx8yMSN91Q2WuX0LhUaCsm3LYLtwpvuglG r1kmT05NjPv0mIzP+UKiZjqZ5ptrNbrwIe5p8nJQToxm3C9JTHZvLrgj7YfJImFi PvA4FJuVIraFBr43lMsjVePaBbzZD+aPR+7xa7aRkLB6x4/XPB5/v2qjOI2A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1660736978; x=1660823378; bh=QoWTOzEf+VtWntwfSkzsVQrH8LH6 xp0381M3fiQBXO4=; b=fEV9WOuImT2sAfERSYw1V0oIGFfOIO5btkUN8KCZi18w VBNPs6KmVBe83tQAOIv7yV1pUCsQDCCYJaoPfKPai5QRvz/35lAbniyX2z0ITPgY BdBphmZ9ETk23WpS9U0xTmG+duXHLuUYlfc8xfGslkkRurWi8SsYvVbogl9NMfUU 6sCDzg2r1HVFE89UqvliPWaak40Y3oU7mmPL04AHxVh+M9oXq+mrm3y4ndrg715w XWisvPFQoq1NLf5L7FkoUo/66DG1AabNOHQ9sWnE3wdbFUrcS72tOO+EGJQAn8AO nmdNuDIfCjXBBnVK8k9Zp7v498whjynvV+TsRci0KQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdehiedggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhtgfgggfukfffvffosehtqhhmtd hhtdejnecuhfhrohhmpegsohhguggrnhdqlhhishhtshesnhgvrghnthdrrhhonecuggft rfgrthhtvghrnhepvdfhjeetheeuvddugfeugffgudetkefgveeuffffieegtdeludefhe ffkeeihfdunecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgnecuvehluhhsthgvrhfu ihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepsghoghgurghnqdhlihhsthhsse hnvggrnhhtrdhroh X-ME-Proxy: Feedback-ID: i61d94637:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 17 Aug 2022 07:49:37 -0400 (EDT) From: bogdan-lists@neant.ro Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: FBSD 13.1 ACPI shutdown on AWS Graviton (ARM) Message-Id: <272F7648-C7F9-40EE-BD25-E014DC009A94@neant.ro> Date: Wed, 17 Aug 2022 14:49:35 +0300 To: freebsd-cloud@freebsd.org X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4M75sb47LFz44X8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=neant.ro header.s=fm1 header.b=fY28ALBq; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=fEV9WOuI; dmarc=none; spf=pass (mx1.freebsd.org: domain of bogdan-lists@neant.ro designates 66.111.4.26 as permitted sender) smtp.mailfrom=bogdan-lists@neant.ro X-Spamd-Result: default: False [-4.20 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[neant.ro:s=fm1,messagingengine.com:s=fm1]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.26:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[neant.ro]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NO_DN(0.00)[]; DKIM_TRACE(0.00)[neant.ro:+,messagingengine.com:+]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N Hi, FreeBSD 13.1 running on ARM in AWS doesn=E2=80=99t shut down when = receiving a signal through EC2 API. It can be an issue for instances in = autoscaling groups because the longer shutdown time might mess with the = scaling policies. And it=E2=80=99s an issue for =E2=80=9Cpermanent=E2=80=9D= instances because it can result in corrupted data on disk. I found https://reviews.freebsd.org/D24065, which seems to be included = in 13.1, although I can=E2=80=99t find it in any release notes. What am = I missing? Should shutdown through AWS Console work?= From nobody Sat Oct 15 06:14:11 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MqCdN73hpz4gHWQ for ; Sat, 15 Oct 2022 06:14:16 +0000 (UTC) (envelope-from gspurki@gmail.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MqCdN1srxz46vv for ; Sat, 15 Oct 2022 06:14:16 +0000 (UTC) (envelope-from gspurki@gmail.com) Received: by mail-ej1-x635.google.com with SMTP id ot12so14642111ejb.1 for ; Fri, 14 Oct 2022 23:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=hLcKVF+BxQ3itxoR2a3BPtIkYjZPD6Z9IqO22mswfA4=; b=PE0JcHYaSK8kjInM5wYONK7JXlVbcBf47hbrKzSPLg0DkzxlXS4soh3/NxcFgyT9aE DgateqYJ8NmNaCuVEeK1DKFeRSthNMLz4IupgonAc5bz6xLRX7Dzp3I4B0uCYPii2F2K KSeF4RoA/UKfFju+biGFVxpcUwtFqHzbXh31aPRuwhFN6v1s4EKIJNZHL4nSl6B9vXlt 4OWI+TtVn1RDmQaNjwtCXHfHNrvBCHK5nwT4LojILurOG4ymL6aiZ9d7lyJu2mReF4Mq BtYKeK3z61a0HjsYtHVcCNEjAFF5E88E+ZALcAm2GrpBXTxFrNFcnaPCFUYvOmMLMt4m TlSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=hLcKVF+BxQ3itxoR2a3BPtIkYjZPD6Z9IqO22mswfA4=; b=y+kXR2CZ+KBg89R92/o3MkB0d+LwushDqcsmrHSkZh2IA6BJzgbKQm8tS4CcazHmOG tvk0yMdTH+QQPQNvjI67tJZP05UUUXY2RhSsZBeVqQsxcB8HZuCw2wZ9iD62GqOQZCDY UusJE6nON4IwJQJgpX8wvEHwJh2W+49kYvukP/A68IKB1SjXQASjRoVfP0EGYlOagcBI 41zjBb7CObm0wu5MiWdOPmsUcyaazV+/WCrkKPIdckb9zA2+gCDAMVduAQyvFeeahZMi 6MXXmAADU4l7A7jLnFfYjeUElzVLkBcF+cdrvsc+kh2m/JM7OyPtXAEh6Ba0IfdEAQvZ BQ+g== X-Gm-Message-State: ACrzQf3TcvoWmbPP1jDrirvBND+GVEfoBTI4Y5SVXLPWR8dPsVF5h/jy deoOIDxd0tjx5qAme9VEczX0A6pPGlo= X-Google-Smtp-Source: AMsMyM5gkP9MH2cGSRIG/6swpNa+RQ0ldmgOrUZkdHQLe5ANcq52tlxCLEgOphNHFofcULprH6e2Cw== X-Received: by 2002:a17:906:2f87:b0:78a:c0d5:d019 with SMTP id w7-20020a1709062f8700b0078ac0d5d019mr1037332eji.158.1665814453246; Fri, 14 Oct 2022 23:14:13 -0700 (PDT) Received: from ?IPV6:2a02:1210:2e31:8000:3491:fc9d:e58f:c591? ([2a02:1210:2e31:8000:3491:fc9d:e58f:c591]) by smtp.gmail.com with ESMTPSA id m3-20020a170906160300b0078194737761sm2590561ejd.124.2022.10.14.23.14.12 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 14 Oct 2022 23:14:12 -0700 (PDT) Message-ID: Date: Sat, 15 Oct 2022 08:14:11 +0200 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.3.2 To: freebsd-cloud@FreeBSD.org Content-Language: en-US From: Gian-Simon Purkert Subject: Oracle Cloud FreeBSD image, some stuff no Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4MqCdN1srxz46vv X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=PE0JcHYa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gspurki@gmail.com designates 2a00:1450:4864:20::635 as permitted sender) smtp.mailfrom=gspurki@gmail.com X-Spamd-Result: default: False [-3.86 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-0.86)[-0.860]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::635:from]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N Hy, First of all, thank you very much for providing that image, However i found some points that could use a cleanup/fix to and act more like the images from: https://download.freebsd.org/releases/VM-IMAGES/13.1-RELEASE/aarch64/Latest/ * ZFS Module gets loaded from loader.conf and rc.conf, but why? there is not zfs and top shows a empty arc, the std images don't do that. * The installed pkg like panicmail, is that why the image is called *_test? * When running "freebsd-update IDS" there is probably not a single checksum that matches the release, why is that? * The 1G Swap partition...why? The std images don't have that, if one needs a swap then a swapfile would be preferred. * DHCP gives me 1000's of errors like "dhclient[40246]: 5 bad udp checksums in 5 packets" * User root AND freebsd gets the ssh-key. Both can login, no "PermitRootLogin no" or "AllowUsers freebsd" Have a great Day and again Thanks, Gian-Simon From nobody Wed Nov 16 00:22:48 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NBkKL5P8zz4fks2 for ; Wed, 16 Nov 2022 00:23:02 +0000 (UTC) (envelope-from me@igalic.co) Received: from mail-4022.proton.ch (mail-4022.proton.ch [185.70.40.22]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NBkKJ34pXz3kT6 for ; Wed, 16 Nov 2022 00:23:00 +0000 (UTC) (envelope-from me@igalic.co) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=igalic.co header.s=protonmail3 header.b=itWqa0wA; spf=pass (mx1.freebsd.org: domain of me@igalic.co designates 185.70.40.22 as permitted sender) smtp.mailfrom=me@igalic.co; dmarc=none Date: Wed, 16 Nov 2022 00:22:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igalic.co; s=protonmail3; t=1668558178; x=1668817378; bh=GoczDEhHvSGnvGKNsX3JwM6W7y+jPayiTCBSC/al1kI=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=itWqa0wAU37x/30INSXmot1Y6coRRjHg5Lu3Pq6feUmIget21Pv6PXZEvN/aMv3vc QfqQoEoyaE9LVLdVQG5Ox8MPDHyRcpMOC1mrwL1eHVkOWcKF3F4dCagtg6Za2P0hbP DbGiNVvV350KUSuoVz8xZifXGFh90BXvhppTRAFtS1EUTBfH8RJIUTG/phXW6bFf6G ocUY4KCvmgzCX7So1/6zWmFk7cfO4HPhHLuyKSLSTG0kpXSUimFvWcLOMXdiaALQ86 hQ8T0mCc+zHhS/jHv/AKd3QP+2QonEQaRiGPS2qSbHO5V/9lFBAZvdcKl031de18ZJ nFlR5gvIF/g3w== To: "freebsd-cloud@FreeBSD.org" From: =?utf-8?Q?Mina_Gali=C4=87?= Subject: cloud-init development Message-ID: Feedback-ID: 13937434:user:proton List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-1.48 / 15.00]; R_MIXED_CHARSET(1.00)[subject]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.978]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; R_DKIM_ALLOW(-0.20)[igalic.co:s=protonmail3]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_EQ_ADDR_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; FREEFALL_USER(0.00)[me]; DKIM_TRACE(0.00)[igalic.co:+]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[igalic.co]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4NBkKJ34pXz3kT6 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N Hi folks, as you may have read in the FreeBSD Quarterly Update[1], I have started working (again) on bringing better BSD support to cloud-init. There's a now net/cloud-init-devel port, that is regularly updated with the new changes. The latest update (22.3-112-g7b049855)[2] will include the ifconfig(8)[3] parser that I've been working on for the past month. If you're working on creating VM images for different clouds providers, I would hugely appreciate if you could help test cloud-init-devel, before it's released, and give feedback - praise and/or bug reports! Thank you very much in advance. Kind regards, Mina Gali=C4=87 ---- 1: https://www.freebsd.org/status/report-2022-07-2022-09/#_freebsd_as_a_tie= r_1_cloud_init_platform 2: https://bugs.freebsd.org/267785 3: https://github.com/canonical/cloud-init/pull/1779 Web: https://igalic.co/ PkgBase Repo: https://alpha.pkgbase.live/ From nobody Thu Dec 15 22:03:32 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NY5pn069jz4dLyd for ; Thu, 15 Dec 2022 22:03:45 +0000 (UTC) (envelope-from robballantyne3@gmail.com) Received: from mail-vs1-xe31.google.com (mail-vs1-xe31.google.com [IPv6:2607:f8b0:4864:20::e31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NY5pm0cfHz4P71 for ; Thu, 15 Dec 2022 22:03:44 +0000 (UTC) (envelope-from robballantyne3@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=KOw8o25e; spf=pass (mx1.freebsd.org: domain of robballantyne3@gmail.com designates 2607:f8b0:4864:20::e31 as permitted sender) smtp.mailfrom=robballantyne3@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-vs1-xe31.google.com with SMTP id h26so613077vsr.5 for ; Thu, 15 Dec 2022 14:03:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=8sP0x8eaP1jPeehchKh8X+So3ro+p361K7i7/fA4wy0=; b=KOw8o25elrX09JWDLQrEMfr6/CKDpnX/0KEf8PKIsy+675ssBoaU7U7+qHuV2EjZe8 fhG/fpmyc4RNaarYMdNM8YUgWv3UHHF0vZAPdWENDA2vaThvKdSN+dPoLGy09aUNXMYU HKFwbatv7CwnFw7HbyH392WUWS4+N9O9RT21FjCku6wpRbEzWJtsKGVvaynG+umuPAuH AgXJ7eg8T5rHMP2qjM2Pg16Hwkqa5x9o0P2PyiY/RJteEwbgQGIoakRqJmZZeI+nq4kk V7D3w3BJbQSQHZGvP58vJYWatvslxX6uWXmEnwZRkXHpqy7rzxOgeFPn8/iGCXQTQ3bu 6gcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=8sP0x8eaP1jPeehchKh8X+So3ro+p361K7i7/fA4wy0=; b=KuZliLRD8k2ikWvxzbG8uw6s3PyU70+pRgN+7wZhxbwxFYueVX95zrE5yVneIZqh+d 5f1DndR90dxheEoH6R35vLgpLl1Wsg3e/FhEpJ1jhBADTUmK0mN8gEKgdUjlBhLsFxf4 53Ow3b47Cm/sHGQ6IqaZWfKgRTUhRvHK+FfxGKaS5dXQRAGi32c50fQtY5epJyrsA5Qk 4awGe9w7kcLqHZN2SDp62nfjgfv+7CmRffK7m6sxKNmNRdF32M5VX5Tzz3y+uEIcYQtD RXVbo0AcW2XzCuyr6aPBbEnfBqqulRKeO+5snnRr3HO5x5fedfiBIT6BOIc3SUqc0zz4 ozbA== X-Gm-Message-State: ANoB5pnC5icxyzlUAEvzR3uXBbRUSqlsiqcp9nG8LNGu5hVyfnu0CF+f 32ngAPwmiFn3mS1G4wgJYPJcw5UmUBsMOy5RkwGDz7u78IY= X-Google-Smtp-Source: AA0mqf4xKqbsUaoUCshmKIQDAOyejdsnYTmoNXpTvC56Q3sZPahuh+m5P2VWKm87QNB6QleOCFfflHp4h20Awow/ejc= X-Received: by 2002:a67:1744:0:b0:3b0:8e7c:5f72 with SMTP id 65-20020a671744000000b003b08e7c5f72mr35654231vsx.27.1671141823423; Thu, 15 Dec 2022 14:03:43 -0800 (PST) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 From: Rob Ballantyne Date: Thu, 15 Dec 2022 14:03:32 -0800 Message-ID: Subject: What is a VPC (google's specifically but it could be more general) really? To: freebsd-cloud@freebsd.org Content-Type: multipart/alternative; boundary="000000000000c128d905efe506ec" X-Spamd-Result: default: False [-1.97 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.974]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::e31:from]; FREEMAIL_FROM(0.00)[gmail.com]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4NY5pm0cfHz4P71 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N --000000000000c128d905efe506ec Content-Type: text/plain; charset="UTF-8" Hello, I have a question about what the internal structure and forwarding is within Google's VPCs. I started into a project using OpenVPN to bind my home network to an isolated VPC in Google's Cloud when I discovered the routing didn't work quite the way I thought. I had assumed that VPCs would look like a private VLAN (Layer2) into which Google's infrastructure would inject L3 router interfaces and/or ip/ethernet filters. I set up a private VPC and two test FreeBSD boxes to test and see exactly how VPC configures routing. First, I just used a standard install of 13.1 and the routing table after everything is up and configured looks like: ---- Internet: Destination Gateway Flags Netif Expire default 10.1.1.1 UGS vtnet0 10.1.1.1 link#1 UHS vtnet0 10.1.1.20 link#1 UH lo0 127.0.0.1 link#2 UH lo0 ---- This looked a little unusual to me so (there was no link local route for all the addresses in the VPC), I commented out the rc.conf entry 'google_network_daemon_enable=YES' and setup the vtnet0 interface up manually with: 'ifconfig_vtnet0="inet 10.1.1.20 netmask 255.255.255.0"' The resulting routing table: ---- Internet: Destination Gateway Flags Netif Expire 10.1.1.0/24 link#1 U vtnet0 10.1.1.20 link#1 UHS lo0 127.0.0.1 link#2 UH lo0 ---- This configuration wasn't able to communicate. The latter routing table looks more usual though, with a 10.1.1.0/24 route to the local link. So, it appears to me that VPCs are really configured to be a point-to-point (star really) network where the Google router interface (10.1.1.1 in this case) has to handle all forwarding between nodes of a network. I've searched around the web to try and confirm this but there is scant detail on how exactly forwarding works within a single VPC. My VPN project involved using a bastion VPN host that would have terminated the VPN/SSL tunnel and routed traffic between my home network and the isolated network behind the bastion. Before I make final decisions on configuration, I wanted to know if my understanding is correct and whether there is any documentation on this that I've somehow missed. FreeBSD is, of course, the host of choice for this operation! If anyone does know any details, any info would be greatly appreciated. Many Thanks, Rob Ballantyne --000000000000c128d905efe506ec Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello,

=C2=A0 I have a question about what the internal= structure and forwarding is within Google's VPCs.

=C2=A0 I star= ted into a project using OpenVPN to bind my home network to an isolated VPC= in Google's Cloud when I discovered the routing didn't work quite = the way I thought.=C2=A0 I had assumed that VPCs would look like a private = VLAN (Layer2) into which Google's infrastructure would inject L3 router= interfaces and/or ip/ethernet filters.

=C2=A0 I set up a private VP= C and two test FreeBSD boxes to test and see exactly how VPC configures rou= ting.=C2=A0=C2=A0

=C2=A0 First, I just used a standard install of 13= .1 and the routing table after everything is up and configured looks like:<= /div>

----
Internet:
Destination =C2=A0 =C2=A0 =C2=A0 =C2=A0Gateway = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 =C2=A0 Netif Expiredefault =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A010.1.1.1 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 UGS =C2=A0 =C2=A0 =C2=A0vtnet0
10.1.1.1 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 U= HS =C2=A0 =C2=A0 =C2=A0vtnet0
10.1.1.20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0lo0
127.0.0.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#2 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0lo0
----

=C2=A0 This looked a little unusual to me so (there= was no link local route for all the addresses in the VPC), I commented out= the rc.conf entry 'google_network_daemon_enable=3DYES' and setup t= he vtnet0 interface up manually with: 'ifconfig_vtnet0=3D"inet 10.= 1.1.20 netmask 255.255.255.0"'=C2=A0 The resulting routing table:<= /div>

----
Internet:
Destination =C2=A0 =C2=A0 =C2=A0 =C2=A0Gateway = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 =C2=A0 Netif Expire10.1.1.0/24 =C2=A0 =C2=A0 =C2=A0 =C2= =A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 U =C2=A0 =C2=A0 =C2=A0 = =C2=A0vtnet0
10.1.1.20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#1 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UHS =C2=A0 =C2=A0 =C2=A0 =C2=A0 lo0
1= 27.0.0.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#2 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lo0
----
<= div class=3D"gmail_default" style=3D"font-family:monospace,monospace">
<= /div>
=C2=A0 This configuration wasn't able to communicate. The latter routi= ng table looks more usual though, with a 10.= 1.1.0/24 route to the local link.

=C2=A0 So, it appears to me th= at VPCs=C2=A0are really configured to be a point-to-point (star really) net= work where the Google router interface (10.1.1.1 in this case) has to handl= e all forwarding between nodes of a network.

=C2=A0 I've searche= d around the web to try and confirm this but there is scant detail on how e= xactly forwarding works within a single VPC.

=C2=A0 My VPN project i= nvolved using a bastion VPN host that would have terminated the VPN/SSL tun= nel and routed traffic between my home network and the isolated network beh= ind the bastion.

=C2=A0 Before I make final decisions on configurati= on, I wanted=C2=A0to know if my understanding is correct and whether there = is any documentation on this that I've somehow missed.

=C2=A0 F= reeBSD is, of course, the host of choice for this operation!

=C2=A0 = If anyone does know any details, any info would be greatly appreciated.
Many Thanks,
Rob Ballantyne
--000000000000c128d905efe506ec-- From nobody Thu Dec 15 22:27:06 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NY6LT5C0hz4gd8h for ; Thu, 15 Dec 2022 22:27:45 +0000 (UTC) (envelope-from bilbo@hobbiton.org) Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NY6LT3JPNz4Q0x for ; Thu, 15 Dec 2022 22:27:43 +0000 (UTC) (envelope-from bilbo@hobbiton.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x633.google.com with SMTP id vv4so2053122ejc.2 for ; Thu, 15 Dec 2022 14:27:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ofwilsoncreek-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=lJo9wZe2pXKdwomLd1sLMj8x934VtEq+oZ7v2bmsUgk=; b=CaJ5v8EBVE6MmuoK349WEUz84eeIJPviyC79pngaKa10ppCF28KfyGUhG8XLVF74Ho 93svjpaVRulu/ohEBM+tVwDQqut+QHiXvb5BBbS6IJtzY2PJWeoNqBELNG3PNhrea527 BTa8Je1KWq5lyitQYOTJaKuc5+Nf9FWSRC4jQTUkU4z/Eoq2qKaimYmKfmp4EVcbcDA/ NqLtWyoLu7RRcs57PAuAWisxy/1theze88VLACs5Py8jp89Sdf0IlVtHjGy2l9yn2FEB ufRdk2lJD8Y97ME/cRv62jtajH7fpZzsxrk9VEtVClUDregMLYsiW5b0v7NPWiy1CBom qVpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lJo9wZe2pXKdwomLd1sLMj8x934VtEq+oZ7v2bmsUgk=; b=ZUAhB9pd62D1hGyWYeaxAw57Cz4Q+k1acdkIIICQUHxZFtC9goSZIuYt5YNYZoDhKq AOlOvRWBfqsihkt3Tdd8LjmS8hnhJ3RjCAfJVMVyPFmm/2hBehs7al8LfNXXpBmzoLnG XPivMnHdI17Bkz/u5q2WfFZtQViU5SjN/uuj+ySRfyULuoQy38sBw8xa/WacNNZAoSDY PfUYEQCUKflxNrLqn2xXyxdmyvYvWzY4Wzpw1aO9fgnQeR6r7jMaDtDAZkflNdl9fuN5 ILe9arbqDK8mO4vz0Js1ykZXfFqdUzY/0U+VRFFmDCU0K6yD8IO7LRC9lmkcTOQvVU18 ObXw== X-Gm-Message-State: ANoB5plak0O/Tnu7IOAv4vnGQnzcCCotnWaHNRDuVP1WUK4bmfMz4E9s hVrVK289caSUhYpYVqJ3LyIO4d5b9YHs2qTNlVFgnA== X-Google-Smtp-Source: AA0mqf4gOPI/3kMS3H8xasXVkndc75m0GBsbHXjP/4n9jhLiX+J3kCgo/Gj/c0rY1EjN4aaub0Y8lyQ1O/OCZD3uqz8= X-Received: by 2002:a17:906:2854:b0:7ae:3684:84b0 with SMTP id s20-20020a170906285400b007ae368484b0mr73066163ejc.622.1671143262705; Thu, 15 Dec 2022 14:27:42 -0800 (PST) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Leif Pedersen Date: Thu, 15 Dec 2022 16:27:06 -0600 Message-ID: Subject: Re: What is a VPC (google's specifically but it could be more general) really? To: Rob Ballantyne Cc: freebsd-cloud@freebsd.org Content-Type: multipart/alternative; boundary="0000000000008aeee105efe55c4d" X-Rspamd-Queue-Id: 4NY6LT3JPNz4Q0x X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --0000000000008aeee105efe55c4d Content-Type: text/plain; charset="UTF-8" Hi, I don't have a direct answer, but as a user I can confirm that OpenVPN in layer 3 mode works for me. I simply haven't tried it in layer 2 mode with GCE (because I've no need for layer 2 and it incurs the extra overhead of broadcast packets). Layer 2 mode probably won't work anyway because the MTU has to be reduced to 1460, unless you do that on all participating hosts. Point is, if that's an option for you it might be worth exploring. As a side note, I configure the tun devices with the same IP address at the vtnet device. That actually works perfectly, even though the two endpoints are on wildly different networks, and avoids maintaining DNS entries and routes for the point to point network. For example: vtnet0: flags=8943 metric 0 mtu 1460 inet 10.1.2.3 netmask 0xffffffff broadcast 10.1.2.3 inet 130.x.x.x netmask 0xffffffff broadcast 130.x.x.x tun5000: flags=8051 metric 0 mtu 1500 inet 10.1.2.3 --> 10.16.0.1 netmask 0xffffffff (The internal IP on my home router's ethernet interface is 10.16.0.1/16, overlapping with its tun interface.) -Leif On Thu, Dec 15, 2022 at 4:03 PM Rob Ballantyne wrote: > Hello, > > I have a question about what the internal structure and forwarding is > within Google's VPCs. > > I started into a project using OpenVPN to bind my home network to an > isolated VPC in Google's Cloud when I discovered the routing didn't work > quite the way I thought. I had assumed that VPCs would look like a private > VLAN (Layer2) into which Google's infrastructure would inject L3 router > interfaces and/or ip/ethernet filters. > > I set up a private VPC and two test FreeBSD boxes to test and see > exactly how VPC configures routing. > > First, I just used a standard install of 13.1 and the routing table > after everything is up and configured looks like: > > ---- > Internet: > Destination Gateway Flags Netif Expire > default 10.1.1.1 UGS vtnet0 > 10.1.1.1 link#1 UHS vtnet0 > 10.1.1.20 link#1 UH lo0 > 127.0.0.1 link#2 UH lo0 > ---- > > This looked a little unusual to me so (there was no link local route for > all the addresses in the VPC), I commented out the rc.conf entry > 'google_network_daemon_enable=YES' and setup the vtnet0 interface up > manually with: 'ifconfig_vtnet0="inet 10.1.1.20 netmask 255.255.255.0"' > The resulting routing table: > > ---- > Internet: > Destination Gateway Flags Netif Expire > 10.1.1.0/24 link#1 U vtnet0 > 10.1.1.20 link#1 UHS lo0 > 127.0.0.1 link#2 UH lo0 > ---- > > This configuration wasn't able to communicate. The latter routing table > looks more usual though, with a 10.1.1.0/24 route to the local link. > > So, it appears to me that VPCs are really configured to be a > point-to-point (star really) network where the Google router interface > (10.1.1.1 in this case) has to handle all forwarding between nodes of a > network. > > I've searched around the web to try and confirm this but there is scant > detail on how exactly forwarding works within a single VPC. > > My VPN project involved using a bastion VPN host that would have > terminated the VPN/SSL tunnel and routed traffic between my home network > and the isolated network behind the bastion. > > Before I make final decisions on configuration, I wanted to know if my > understanding is correct and whether there is any documentation on this > that I've somehow missed. > > FreeBSD is, of course, the host of choice for this operation! > > If anyone does know any details, any info would be greatly appreciated. > > Many Thanks, > Rob Ballantyne > --0000000000008aeee105efe55c4d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

I don't have a direct answer, b= ut as a user I can confirm that OpenVPN in layer 3 mode works for me. I sim= ply haven't tried it in layer 2 mode with GCE (because I've no need= for layer 2 and it incurs the extra overhead of broadcast packets). Layer = 2 mode probably won't work anyway because the MTU has to be reduced to = 1460, unless you do that on all participating hosts. Point is, if that'= s an option for you it might be worth exploring.

A= s a side note, I configure the tun devices with the same IP address at the = vtnet device. That actually works perfectly, even though the two endpoints = are on wildly different networks, and avoids maintaining DNS entries and ro= utes for the point to point network.

For example:<= /div>
vtnet0: flags=3D8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULT= ICAST> metric 0 mtu 1460
=C2=A0 =C2=A0 inet 10.1.2.3 netma= sk 0xffffffff broadcast 10.1.2.3
=C2=A0 =C2=A0 inet 130.x.x.x netmask 0x= ffffffff broadcast 130.x.x.x
tun5000: flags=3D8051<UP,POINTOPOINT,RUN= NING,MULTICAST> metric 0 mtu 1500
=C2=A0 =C2=A0 inet 10.1.2.3 --> = 10.16.0.1 netmask 0xffffffff

(The internal IP = on my home router's ethernet interface is 10.16.0.1/16, overlapping with its tun interface.)

-Leif





On Thu, Dec 15, 2022 at 4:03 PM Rob Ballantyne <robballantyne3@gmail.com> = wrote:
Hello,

=C2=A0 I have a question about what the internal structu= re and forwarding is within Google's VPCs.

=C2=A0 I started into= a project using OpenVPN to bind my home network to an isolated VPC in Goog= le's Cloud when I discovered the routing didn't work quite the way = I thought.=C2=A0 I had assumed that VPCs would look like a private VLAN (La= yer2) into which Google's infrastructure would inject L3 router interfa= ces and/or ip/ethernet filters.

=C2=A0 I set up a private VPC and tw= o test FreeBSD boxes to test and see exactly how VPC configures routing.=C2= =A0=C2=A0

=C2=A0 First, I just used a standard install of 13.1 and t= he routing table after everything is up and configured looks like:

-= ---
Internet:
Destination =C2=A0 =C2=A0 =C2=A0 =C2=A0Gateway =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 =C2=A0 Netif Expire
default = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A010.1.1.1 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 UGS =C2=A0 =C2=A0 =C2=A0vtnet0
10.1.1.1 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UHS =C2=A0 = =C2=A0 =C2=A0vtnet0
10.1.1.20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#1 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0lo0
127.0.0.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#2 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lo0
----

=C2=A0 This looked a little= unusual to me so (there was no link local route for all the addresses in t= he VPC), I commented out the rc.conf entry 'google_network_daemon_enabl= e=3DYES' and setup the vtnet0 interface up manually with: 'ifconfig= _vtnet0=3D"inet 10.1.1.20 netmask 255.255.255.0"'=C2=A0 The r= esulting routing table:

----
Internet:
Destination =C2=A0 =C2=A0= =C2=A0 =C2=A0Gateway =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags =C2=A0= =C2=A0 Netif Expire
10= .1.1.0/24 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 U =C2=A0 =C2=A0 =C2=A0 =C2=A0vtnet0
10.1.1.20 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH= S =C2=A0 =C2=A0 =C2=A0 =C2=A0 lo0
127.0.0.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0link#2 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0lo0
----

=C2=A0 This configuration wasn't = able to communicate. The latter routing table looks more usual though, with= a 10.1.1.0/24 route t= o the local link.

=C2=A0 So, it appears to me that VPCs=C2=A0are rea= lly configured to be a point-to-point (star really) network where the Googl= e router interface (10.1.1.1 in this case) has to handle all forwarding bet= ween nodes of a network.

=C2=A0 I've searched around the web to = try and confirm this but there is scant detail on how exactly forwarding wo= rks within a single VPC.

=C2=A0 My VPN project involved using a bast= ion VPN host that would have terminated the VPN/SSL tunnel and routed traff= ic between my home network and the isolated network behind the bastion.
=C2=A0 Before I make final decisions on configuration, I wanted=C2=A0to= know if my understanding is correct and whether there is any documentation= on this that I've somehow missed.

=C2=A0 FreeBSD is, of course,= the host of choice for this operation!

=C2=A0 If anyone does know a= ny details, any info would be greatly appreciated.

Many Thanks,
Rob= Ballantyne
--0000000000008aeee105efe55c4d-- From nobody Fri Dec 16 00:10:53 2022 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NY8dk2CMGz4jZwM for ; Fri, 16 Dec 2022 00:11:06 +0000 (UTC) (envelope-from robballantyne3@gmail.com) Received: from mail-vs1-xe29.google.com (mail-vs1-xe29.google.com [IPv6:2607:f8b0:4864:20::e29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NY8dj5XCxz4X0w for ; Fri, 16 Dec 2022 00:11:05 +0000 (UTC) (envelope-from robballantyne3@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-vs1-xe29.google.com with SMTP id a66so855958vsa.6 for ; Thu, 15 Dec 2022 16:11:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=YD2tUvFCkegdykr91Qi6Exh0FiHXzH4PIzAcIXo8ckE=; b=e+ibecgphUOAEsefyRW037jTy5aL4OkcEAqmnhUpUfXuQ1XXR6ScZu+fNm6UR0uyBa Tu4TJR/g5hEMTCxYwylJG46Srw3cZhQiiWsBb8xm8QsI6MYtlpqJWC8IZ0jVEeMfUNTQ lt3Skexd1Mzo12GKk1RbGi4QqeTHn84FfJASrFaTTDg0Cjqob3+rRXKjRdjjZcZT1mBP xBNGd/6h0vX4jAOIIBHDzCX+v6SkfnVxngWIvmYvQL9RPKzC7iE4MqlsbYfwLexY73lZ 17fMD/nO5zIpbHmxYdWUZD6eKb4FVjA33HWyawcD5JPvWyAfukI9jruoWZ+SFF4RAFW2 gFww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YD2tUvFCkegdykr91Qi6Exh0FiHXzH4PIzAcIXo8ckE=; b=RdielJp3qsFzL/oGi7GMz3vc7nV7UtMJIF57hHP/R3qWLgOeuHBbs8Qq3jRlX3azaH sWJNfXyLX44gdb1NdObNYLtxc3sym2kqltPXtjE81LKoe62bUk50wp0DemcuvfU6z23q ecvUGI47wmfdlWx8A+qbKAKJAMjDCP9tod4cnPb6IKC4Cpv2kBojWJA3GDpGHeFKhb9q lfpGpqSPMNKkgEcjxDfChJZj6aaiUve2QSmiE6webkSVqPfquhezR8EGAbqXsaMt5fOC E6q21GBNrRms62cTTzn0/L9AnxovH3N931li4Z50w4gs40xqKzpw7es5Fm+sdQ6huZsu gaFA== X-Gm-Message-State: ANoB5pnnQj4gBiWuURe6EQOvSyh054GHgUjOJqsC0fJUsQbQpz8rEzc9 G06hmE7d/VzhJ6dwkDeGcovBD6M+xKTc8/MSD6lUCi5j X-Google-Smtp-Source: AA0mqf5SDCAjQU8xhqHKPPLbYD19keA6ATkTiJyyt7rgRFjqgX+REHvrzaedDN/2jixX5U7/QoQtd4cEX4LAYNTDYDU= X-Received: by 2002:a05:6102:807:b0:3b0:cc5c:a3f7 with SMTP id g7-20020a056102080700b003b0cc5ca3f7mr26410867vsb.1.1671149464890; Thu, 15 Dec 2022 16:11:04 -0800 (PST) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Rob Ballantyne Date: Thu, 15 Dec 2022 16:10:53 -0800 Message-ID: Subject: Re: What is a VPC (google's specifically but it could be more general) really? To: leif@ofwilsoncreek.com Cc: freebsd-cloud@freebsd.org Content-Type: multipart/alternative; boundary="00000000000038b1af05efe6ce04" X-Rspamd-Queue-Id: 4NY8dj5XCxz4X0w X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000038b1af05efe6ce04 Content-Type: text/plain; charset="UTF-8" Thank you Leif, I probably should have mentioned that I got the OpenVPN tunnel working as well. I was confused as to what was going on until I looked carefully at what Google had installed in the routing table and saw what ought to be a link local route (which would normally just be directed at a link#k entry in the Gateway Field) was actually directed to what I believe is the VPC router interface in the subnet (10.1.1.1 above). It's working now but I've got an uneasy feeling I haven't done it 'right.' If this were ordinary VLAN/Ethernet stuff, it would work like this too (I think) but it would be incurring an extra L3 hop through the router when it could have gone over the VLAN/Ethernet fabric direct. Thanks again! Rob On Thu, Dec 15, 2022 at 2:27 PM Leif Pedersen wrote: > Hi, > > I don't have a direct answer, but as a user I can confirm that OpenVPN in > layer 3 mode works for me. I simply haven't tried it in layer 2 mode with > GCE (because I've no need for layer 2 and it incurs the extra overhead of > broadcast packets). Layer 2 mode probably won't work anyway because the MTU > has to be reduced to 1460, unless you do that on all participating hosts. > Point is, if that's an option for you it might be worth exploring. > > As a side note, I configure the tun devices with the same IP address at > the vtnet device. That actually works perfectly, even though the two > endpoints are on wildly different networks, and avoids maintaining DNS > entries and routes for the point to point network. > > For example: > vtnet0: flags=8943 metric > 0 mtu 1460 > inet 10.1.2.3 netmask 0xffffffff broadcast 10.1.2.3 > inet 130.x.x.x netmask 0xffffffff broadcast 130.x.x.x > tun5000: flags=8051 metric 0 mtu 1500 > inet 10.1.2.3 --> 10.16.0.1 netmask 0xffffffff > > (The internal IP on my home router's ethernet interface is 10.16.0.1/16, > overlapping with its tun interface.) > > -Leif > > > > > > On Thu, Dec 15, 2022 at 4:03 PM Rob Ballantyne > wrote: > >> Hello, >> >> I have a question about what the internal structure and forwarding is >> within Google's VPCs. >> >> I started into a project using OpenVPN to bind my home network to an >> isolated VPC in Google's Cloud when I discovered the routing didn't work >> quite the way I thought. I had assumed that VPCs would look like a private >> VLAN (Layer2) into which Google's infrastructure would inject L3 router >> interfaces and/or ip/ethernet filters. >> >> I set up a private VPC and two test FreeBSD boxes to test and see >> exactly how VPC configures routing. >> >> First, I just used a standard install of 13.1 and the routing table >> after everything is up and configured looks like: >> >> ---- >> Internet: >> Destination Gateway Flags Netif Expire >> default 10.1.1.1 UGS vtnet0 >> 10.1.1.1 link#1 UHS vtnet0 >> 10.1.1.20 link#1 UH lo0 >> 127.0.0.1 link#2 UH lo0 >> ---- >> >> This looked a little unusual to me so (there was no link local route >> for all the addresses in the VPC), I commented out the rc.conf entry >> 'google_network_daemon_enable=YES' and setup the vtnet0 interface up >> manually with: 'ifconfig_vtnet0="inet 10.1.1.20 netmask 255.255.255.0"' >> The resulting routing table: >> >> ---- >> Internet: >> Destination Gateway Flags Netif Expire >> 10.1.1.0/24 link#1 U vtnet0 >> 10.1.1.20 link#1 UHS lo0 >> 127.0.0.1 link#2 UH lo0 >> ---- >> >> This configuration wasn't able to communicate. The latter routing table >> looks more usual though, with a 10.1.1.0/24 route to the local link. >> >> So, it appears to me that VPCs are really configured to be a >> point-to-point (star really) network where the Google router interface >> (10.1.1.1 in this case) has to handle all forwarding between nodes of a >> network. >> >> I've searched around the web to try and confirm this but there is scant >> detail on how exactly forwarding works within a single VPC. >> >> My VPN project involved using a bastion VPN host that would have >> terminated the VPN/SSL tunnel and routed traffic between my home network >> and the isolated network behind the bastion. >> >> Before I make final decisions on configuration, I wanted to know if my >> understanding is correct and whether there is any documentation on this >> that I've somehow missed. >> >> FreeBSD is, of course, the host of choice for this operation! >> >> If anyone does know any details, any info would be greatly appreciated. >> >> Many Thanks, >> Rob Ballantyne >> > --00000000000038b1af05efe6ce04 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thank you Leif,

=C2=A0 I probably should have mentione= d that I got the OpenVPN tunnel working as well.=C2=A0 I was confused as to= what was going on until I looked carefully at what Google had installed in= the routing table and saw what ought to be a link local route (which would= normally just be directed at a link#k entry in the Gateway Field) was actu= ally directed to what I believe is the VPC router interface in the subnet (= 10.1.1.1 above).

=C2=A0 It's working now but I've got an une= asy feeling I haven't done it 'right.'=C2=A0 If this were ordin= ary VLAN/Ethernet stuff, it would work like this too (I think) but it would= be incurring an extra L3 hop through the router when it could have gone ov= er the VLAN/Ethernet fabric direct.

=C2=A0 Thanks again!
Rob
<= /div>
O= n Thu, Dec 15, 2022 at 2:27 PM Leif Pedersen <leif@ofwilsoncreek.com> wrote:
Hi,

I don't have a direct answer, but as a user I can confirm that OpenV= PN in layer 3 mode works for me. I simply haven't tried it in layer 2 m= ode with GCE (because I've no need for layer 2 and it incurs the extra = overhead of broadcast packets). Layer 2 mode probably won't work anyway= because the MTU has to be reduced to 1460, unless you do that on all parti= cipating hosts. Point is, if that's an option for you it might be worth= exploring.

As a side note, I configure the tun de= vices with the same IP address at the vtnet device. That actually works per= fectly, even though the two endpoints are on wildly different networks, and= avoids maintaining DNS entries and routes for the point to point network.<= /div>

For example:
vtnet0: flags=3D8943<UP,= BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1460
=
=C2=A0 =C2=A0 inet 10.1.2.3 netmask 0xffffffff broadcast 10.1.2.3
= =C2=A0 =C2=A0 inet 130.x.x.x netmask 0xffffffff broadcast 130.x.x.x
tun5= 000: flags=3D8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500=
=C2=A0 =C2=A0 inet 10.1.2.3 --> 10.16.0.1 netmask 0xffffffff

(The internal IP on my home router's ethernet int= erface is 10.16.0.1/16, overlapping with its tun interface.)


On Thu, Dec 15= , 2022 at 4:03 PM Rob Ballantyne <robballantyne3@gmail.com> wrote:
Hello,

= =C2=A0 I have a question about what the internal structure and forwarding i= s within Google's VPCs.

=C2=A0 I started into a project using = OpenVPN to bind my home network to an isolated VPC in Google's Cloud wh= en I discovered the routing didn't work quite the way I thought.=C2=A0 = I had assumed that VPCs would look like a private VLAN (Layer2) into which = Google's infrastructure would inject L3 router interfaces and/or ip/eth= ernet filters.

=C2=A0 I set up a private VPC and two test FreeBSD bo= xes to test and see exactly how VPC configures routing.=C2=A0=C2=A0

= =C2=A0 First, I just used a standard install of 13.1 and the routing table = after everything is up and configured looks like:

----
Internet:Destination =C2=A0 =C2=A0 =C2=A0 =C2=A0Gateway =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0Flags =C2=A0 =C2=A0 Netif Expire
default =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A010.1.1.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UGS =C2= =A0 =C2=A0 =C2=A0vtnet0
10.1.1.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 link= #1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UHS =C2=A0 =C2=A0 =C2=A0vtnet0=
10.1.1.20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#1 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lo0
127.0.0.1= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#2 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lo0
----

=C2=A0 This looked a little unusual to me so (th= ere was no link local route for all the addresses in the VPC), I commented = out the rc.conf entry 'google_network_daemon_enable=3DYES' and setu= p the vtnet0 interface up manually with: 'ifconfig_vtnet0=3D"inet = 10.1.1.20 netmask 255.255.255.0"'=C2=A0 The resulting routing tabl= e:

----
Internet:
Destination =C2=A0 =C2=A0 =C2=A0 =C2=A0Gateway= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Flags =C2=A0 =C2=A0 Netif Expire<= br>10.1.1.0/24 =C2=A0 = =C2=A0 =C2=A0 =C2=A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 U =C2= =A0 =C2=A0 =C2=A0 =C2=A0vtnet0
10.1.1.20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0link#1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UHS =C2=A0 =C2=A0 =C2= =A0 =C2=A0 lo0
127.0.0.1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0link#2 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 UH =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lo= 0
----

=C2=A0 This configuration wasn't able to communicate= . The latter routing table looks more usual though, with a 10.1.1.0/24 route to the local link.
=
=C2=A0 So, it appears to me that VPCs=C2=A0are really configured to b= e a point-to-point (star really) network where the Google router interface = (10.1.1.1 in this case) has to handle all forwarding between nodes of a net= work.

=C2=A0 I've searched around the web to try and confirm thi= s but there is scant detail on how exactly forwarding works within a single= VPC.

=C2=A0 My VPN project involved using a bastion VPN host that w= ould have terminated the VPN/SSL tunnel and routed traffic between my home = network and the isolated network behind the bastion.

=C2=A0 Before = I make final decisions on configuration, I wanted=C2=A0to know if my unders= tanding is correct and whether there is any documentation on this that I= 9;ve somehow missed.

=C2=A0 FreeBSD is, of course, the host of choic= e for this operation!

=C2=A0 If anyone does know any details, any in= fo would be greatly appreciated.

Many Thanks,
Rob Ballantyne
<= /div>
--00000000000038b1af05efe6ce04-- From nobody Mon Mar 6 02:15:33 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVMcZ4kyxz3wGHx for ; Mon, 6 Mar 2023 02:15:42 +0000 (UTC) (envelope-from meta@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVMcZ4GPvz3BnN; Mon, 6 Mar 2023 02:15:42 +0000 (UTC) (envelope-from meta@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678068942; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=VW9dChPKi8BCYUhnrEJBgEegap7zASsRJGQF4e7pxSg=; b=SBTszqp/z/IwHdIl/mdpKPnMZdSFT+co+Gg8QnO+7ZBm7wZOVI3d8gICRCtwQKQhDVKVgl 7OreeLmYYbetJyNdmCYGBPb1uCkDOsTSnjfERVVaiemQPTuHODQrpJ/VbY23AgUnusmeVS TqSCi8soCYbNB9R4gmIdo5u6eae9OlpaYbbGQBksFJnEeIP+a0jc8z1QceREwMrWHMRTVe wulC36eWVFKIlqvHEYitBFZjIylROj/OnA/123Bi5hralyDKk7txltILyJgACmvJRR0nOM 29eYO4RBbWMynAAA8Mhzl9F14O3hOZzIfUhuit/o62szLtdkPsCSmMgWeDgIqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678068942; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=VW9dChPKi8BCYUhnrEJBgEegap7zASsRJGQF4e7pxSg=; b=s8GQ55ArFE8te7Bthy504aBB8BCGXaq7Ge4/WMXDRZwP8Xhgm/I2Z4tVl12WQ3W5tfQCOD lBJtj+mDOowlxSJ8iHP3bfGtBf1jSx0J+EODuOl28gg7VHNgwoAN7qcB2e0LH9DQjR7Zmz J6v9fyKQjTnffu35omJWg77mtu8Y+IefEQ2U5Hf322WSdmUchXMWsO2u2A7RGVfWVMWOZO SColpVIG/HBP6NW+g0ARfO/Dz0EK62Mn0kMviBxiieRZQl268tlAjhNs1psS6JYGuu/o3/ fixcEa7j42luaDTZHQiYWNVqOFFT6KYvG0l/RoPLOKErGem/E0Y7S0lORFE+IQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678068942; a=rsa-sha256; cv=none; b=ymsyKzmfC5B13WvKfH4xrG7k55o/7peLN3j6Ctwoljf4WfkNUtKYNvZNQAqd34jLcdHeIL FRkMeAKptElm1MI3HLpa/8f/JjTCaxtn9i3iTL7zuHpywWO5BRNgUNuhlxaU0hNkYKIIJ9 CUMVvTJuIrVuG11t+PSoDWMifExQVDoVPyb0FPxOIkqImMi+MVKEyjAhorLJHwZnHKk5Ng gOM7Jzmo55YIpm2j0iRl8Oc8j0YDXM6PiYbOkqP7iuqpPHyfiRRH//KwQHM+oSEV4iVwD8 wJIou+PkVGmIgm8xEdk2+VLhSNCYY36QgN/6dqVJQ8u6YfwXcKQxay5um9XMZQ== Received: from icepick.vmeta.jp (unknown [IPv6:2403:bd80:c100:411:911b:1dab:8985:dd4a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: meta/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PVMcY4XV4ztSw; Mon, 6 Mar 2023 02:15:41 +0000 (UTC) (envelope-from meta@FreeBSD.org) Date: Mon, 6 Mar 2023 11:15:33 +0900 From: Koichiro Iwao To: freebsd-cloud@FreeBSD.org Subject: How are official EC2 images built? Message-ID: <20230306021533.ght5ouvxg6zrihfr@icepick.vmeta.jp> X-Operating-System: FreeBSD 13.2-STABLE amd64 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-ThisMailContainsUnwantedMimeParts: N Hi, Does anyone how offical EC2 images [1] are built? I mean if the build script is public. I would like to create private custom images with some customization based on official image. [1] https://aws.amazon.com/marketplace/seller-profile?id=92bb514d-02bc-49fd-9727-c474863f63da&ref=dtl_prodview-ukzmy5dzc6nbq -- meta From nobody Mon Mar 6 02:30:40 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVMyF6xlTz3wH3N for ; Mon, 6 Mar 2023 02:31:01 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVMyF130kz3Dsh for ; Mon, 6 Mar 2023 02:31:01 +0000 (UTC) (envelope-from pat@patmaddox.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm3 header.b=M7pClutx; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="a gVNFiu"; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 66.111.4.28 as permitted sender) smtp.mailfrom=pat@patmaddox.com; dmarc=none Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 8A3605C00F5; Sun, 5 Mar 2023 21:31:00 -0500 (EST) Received: from imap41 ([10.202.2.91]) by compute2.internal (MEProxy); Sun, 05 Mar 2023 21:31:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1678069860; x=1678156260; bh=1acgjJJImjGQuq2pmqIHYttPwksntoZ9848 PM/Xbkzc=; b=M7pClutxpCuq7QLuCC8WWwN5iN2mOQgYic/SG/HP9w4gdp/Kxuu 9he723uOMqpRklNlAwaoy3TRQ5tJusex09L2DicK9PQbBwByQkJpDoPxVfA4x8Wq QmdUSluTL5O5vwTfNztYZ7jZf7pzzOJFq79xA5UXDH8+LSgq6tQTcFUGe+3H50Tf Ak/pa4lCCG+2YtA9EemrcGGQxEghlcVVMXL1h/94BATZLh2sDoz6XsF0iXsTtol+ YyTpbciA6FIwz+K9AH1BdRhm84qHos8bzTkBNu9OyaBT6TXERTZp+/rMSSJQs2vg B4PucCpjjqgRTW9E9jN6sD+/KGXek2bhY3w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1678069860; x= 1678156260; bh=1acgjJJImjGQuq2pmqIHYttPwksntoZ9848PM/Xbkzc=; b=a gVNFiuet0WUVSAh/ra1clhXC0akcC7Ptd5alZEBavD2GE1bD8UEGX49R3na73zfe w7+MXOcT0AAT44t8rT+rfA8Hg9JpJKkWepbDOlCUIY6krsJcY+TeKe5rhjU9DGef ADNRHzeQKvYfYukggUNgCIqHzlR6umdG80jtXxaegeMuoyRZoYBwzbcYmHu7j7de o0Od/LcfqUwklkKYZwExIu3Q2RbzPI00p9Jqcidxph6IBxxQX0PAiqDKL3Gbojtv eLkC/QxxPJQQfrzpN6keG3BmXZzy/9Gv1KvZ+mAsrz4//DPBYSNDB+pozfbuWtJi 4wwuqsZ1wzfS2mUzrrE+w== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvddthedggeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdfrrghtucforgguughogidfuceophgrthesphgrthhm rgguughogidrtghomheqnecuggftrfgrthhtvghrnhepleduleetteefueeivdffleelle etjeetvdekudefgfeitedtuddtteejueevhedunecuffhomhgrihhnpehfrhgvvggsshgu rdhorhhgpdhgihhthhhusgdrtghomhdprghmrgiiohhnrdgtohhmnecuvehluhhsthgvrh fuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphgrthesphgrthhmrgguugho gidrtghomh X-ME-Proxy: Feedback-ID: i8b6c40f9:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 33570234007B; Sun, 5 Mar 2023 21:31:00 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-183-gbf7d00f500-fm-20230220.001-gbf7d00f5 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 Message-Id: In-Reply-To: <20230306021533.ght5ouvxg6zrihfr@icepick.vmeta.jp> References: <20230306021533.ght5ouvxg6zrihfr@icepick.vmeta.jp> Date: Sun, 05 Mar 2023 18:30:40 -0800 From: "Pat Maddox" To: freebsd-cloud@freebsd.org Subject: Re: How are official EC2 images built? Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.19 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm3,messagingengine.com:s=fm1]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; XM_UA_NO_VERSION(0.01)[]; FREEFALL_USER(0.00)[pat]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[patmaddox.com]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org] X-Rspamd-Queue-Id: 4PVMyF130kz3Dsh X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N https://cgit.freebsd.org/src/tree/release/Makefile.ec2 https://cgit.freebsd.org/src/tree/release/tools/ec2.conf I have opted to build images a bit differently, not using the stuff in s= rc/release. I=E2=80=99ve done two approaches. The first is to attach a second disk to an EC2 instance, install FreeBSD= onto it with customizations, and then image the disk (https://gist.gith= ub.com/patmaddox/9f20edcb50ff15fa9edd1d4e38d07e50). The way I=E2=80=99m doing it now is to use poudriere to make an img file= , and then upload that image to S3, snapshot it (aws ec2 import-snapshot= =E2=80=93disk-container Format=3Draw,Url=3Ds3://mybucket/myimage.img) a= nd finally make an AMI from that snapshot. The stuff in src/release is good for knowing what changes to make to con= fig files, or packages to install. Pat On Sun, Mar 5, 2023, at 6:15 PM, Koichiro Iwao wrote: > Hi,=20 > > Does anyone how offical EC2 images [1] are built? > I mean if the build script is public. I would like to create private > custom images with some customization based on official image. > > [1]=20 > https://aws.amazon.com/marketplace/seller-profile?id=3D92bb514d-02bc-4= 9fd-9727-c474863f63da&ref=3Ddtl_prodview-ukzmy5dzc6nbq > > --=20 > meta From nobody Mon Mar 6 02:31:29 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVMzB11Fsz3wGwS for ; Mon, 6 Mar 2023 02:31:50 +0000 (UTC) (envelope-from pat@patmaddox.com) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVMz95Gwdz3F37 for ; Mon, 6 Mar 2023 02:31:49 +0000 (UTC) (envelope-from pat@patmaddox.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=patmaddox.com header.s=fm3 header.b=k+K48mZf; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="E mEX5hf"; spf=pass (mx1.freebsd.org: domain of pat@patmaddox.com designates 66.111.4.28 as permitted sender) smtp.mailfrom=pat@patmaddox.com; dmarc=none Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id A23305C00C1; Sun, 5 Mar 2023 21:31:49 -0500 (EST) Received: from imap41 ([10.202.2.91]) by compute2.internal (MEProxy); Sun, 05 Mar 2023 21:31:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=patmaddox.com; h=cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1678069909; x=1678156309; bh=42FPRap9hSJ/tKtAnOLycS5Xj2+SagwAyV/ ofeBkbM0=; b=k+K48mZfmGdWMd7EC5HhAofv1Qxo4Tla8bmMBARdLEyCIcAwHdk zRe49IAlmRKlQl/fxZaY+BEgFfNMVlwjSZEJ8ITUF7AKvvAlInO+MN51FnC8nUsG +AhJ/F0niWNhGNwAiKMd8DB+BLmArBYVx+f2Auz5eJUfqyv9bokf+ZsGZCP2FTU5 znL2KOSk/fehRmhxVnzB2Zi/CzO6VPDtTwYpSxSsmMTvGeoz060/gq0iLyk2bS9r 6rnkVOo5BIYHITU3XN+yip+GPQmgxbzhPLMgBzOQ/fxE8NXTNvgMgoAGJtAgYH/T U8/JbD2lr3TOtqMBhe4IJuhIkNimFr9G6Jg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1678069909; x= 1678156309; bh=42FPRap9hSJ/tKtAnOLycS5Xj2+SagwAyV/ofeBkbM0=; b=E mEX5hf/vy9dni56XQ8+TvOW4EReB22rvhXVZWndT0XS1PDQS4J1t06azrlSorsPM GWm0cVdqtHyf4Y+4c2C+iFqKTfRG/cSa3vTeOL8qc1RmL9icx0f4nT88eIePgpTD 71j03FdsD1kCfANlZUokaIwalQ7G3ZUZ7cZrSS4Eb5/PLfi+igfb3CrpufKXaEB+ f7SeocNVcy+vVE6tntyvD7f+gJ2Ybx1GvgAjOjAQjXkrOyxh/zQd8d5v6pnePukL wJazWWBuIDcjsxm05/gPaCLa0KYPTNl+F2dZNBp2/t2qzZyB04jKxQHdMbbcNpg4 2z0tc4ZmSb8YE1M5q2izQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvddthedggeejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdfrrghtucforgguughogidfuceophgrthesphgrthhm rgguughogidrtghomheqnecuggftrfgrthhtvghrnhepleduleetteefueeivdffleelle etjeetvdekudefgfeitedtuddtteejueevhedunecuffhomhgrihhnpehfrhgvvggsshgu rdhorhhgpdhgihhthhhusgdrtghomhdprghmrgiiohhnrdgtohhmnecuvehluhhsthgvrh fuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphgrthesphgrthhmrgguugho gidrtghomh X-ME-Proxy: Feedback-ID: i8b6c40f9:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 67C77234007B; Sun, 5 Mar 2023 21:31:49 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-183-gbf7d00f500-fm-20230220.001-gbf7d00f5 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org Mime-Version: 1.0 Message-Id: <39256c80-e375-44cf-9840-66eb8a43db2e@app.fastmail.com> In-Reply-To: References: <20230306021533.ght5ouvxg6zrihfr@icepick.vmeta.jp> Date: Sun, 05 Mar 2023 18:31:29 -0800 From: "Pat Maddox" To: freebsd-cloud@freebsd.org Subject: Re: How are official EC2 images built? Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.19 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[patmaddox.com:s=fm3,messagingengine.com:s=fm1]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28:c]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; XM_UA_NO_VERSION(0.01)[]; FREEFALL_USER(0.00)[pat]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[patmaddox.com]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[patmaddox.com:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org] X-Rspamd-Queue-Id: 4PVMz95Gwdz3F37 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N One thing I left out - with the second approach I shared, you have to ma= ke the AMI with UEFI mode. -Pat On Sun, Mar 5, 2023, at 6:30 PM, Pat Maddox wrote: > https://cgit.freebsd.org/src/tree/release/Makefile.ec2 > > https://cgit.freebsd.org/src/tree/release/tools/ec2.conf > > I have opted to build images a bit differently, not using the stuff in=20 > src/release. I=E2=80=99ve done two approaches. > > The first is to attach a second disk to an EC2 instance, install=20 > FreeBSD onto it with customizations, and then image the disk=20 > (https://gist.github.com/patmaddox/9f20edcb50ff15fa9edd1d4e38d07e50). > > The way I=E2=80=99m doing it now is to use poudriere to make an img fi= le, and=20 > then upload that image to S3, snapshot it (aws ec2 import-snapshot=20 > =E2=80=93disk-container Format=3Draw,Url=3Ds3://mybucket/myimage.img) = and finally=20 > make an AMI from that snapshot. > > The stuff in src/release is good for knowing what changes to make to=20 > config files, or packages to install. > > Pat > > On Sun, Mar 5, 2023, at 6:15 PM, Koichiro Iwao wrote: >> Hi,=20 >> >> Does anyone how offical EC2 images [1] are built? >> I mean if the build script is public. I would like to create private >> custom images with some customization based on official image. >> >> [1]=20 >> https://aws.amazon.com/marketplace/seller-profile?id=3D92bb514d-02bc-= 49fd-9727-c474863f63da&ref=3Ddtl_prodview-ukzmy5dzc6nbq >> >> --=20 >> meta From nobody Mon Mar 6 02:54:24 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVNTM2pKQz3wJ75 for ; Mon, 6 Mar 2023 02:54:31 +0000 (UTC) (envelope-from meta@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVNTM2LX9z3G5Z; Mon, 6 Mar 2023 02:54:31 +0000 (UTC) (envelope-from meta@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678071271; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5aQ2bAfOugtNfqruYD551/BVB2mpqw/NhOC4KMwKpLY=; b=ME7pEY3Xz6cUpo5RzKptrrtR6k1HxkSK1r+cDpNfqSRmEwS4cQd2Xq9ulcfpni156nO4Ij 82UPoFC7C6ag1JZHo2x0JdAspIEqkhqnECaBDMPnRAkbZ+g6hgMryOeZl82PGTS09/jO+3 cejURPW8O44McrJ6bwyGKPAkDEL2ZK5Xf1+IB3U4d9cgutP+y6aVBz6QhdN4/5gYSYoWQT t0TK2OGdq2gBpAJ+vRLjE+tQORZ5cMryiAeaUXeTex0N16y+Sd7b3x0yC3dU0fM5M+CkQv ydh8AfygbVWbYdcQMgqIT/jN2kHOMJK8G22f6YLxQ7J6bIYf5YnnweF4/YxTDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678071271; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5aQ2bAfOugtNfqruYD551/BVB2mpqw/NhOC4KMwKpLY=; b=CQ/kXc7phn2uXssTxFlAT9eknJBna8ObPC9JaxCWVrkBS7izfiqbcA1Ns4WlAEZ5410Q28 iWWa22Xgsn2KP8kqp6cEDsUP5x/6aJ9O6NjB25/48Hm4pz+eCHqpKXUFChoEYg7kLvk6xN kusf9GV5a1V+PKdYk+hp8XYHTj1fgHD8zc8CwV8NtlCNXo7GMugcHLgXj6oidyVaqMtwko KbCIuYSoK0+xznHJqHKuEkZgnLttJdJojo8On6FNY4wEIFmXXwLt4C2cY6mQOaEQEjKeAO zVQcjpNAVZOxqJhpJ1xyeDFs+pz5IlFtXdzstjC74uvSC2EP/aLuWiTpiooM/Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678071271; a=rsa-sha256; cv=none; b=haTnIIkLKSv986u4cRIPnpc3xdkelhnCjVTa3FDX/2gsw51KD0uoE5zohjXxCaAk/6EqXE v3ewGW+dFu5MNMv6D+TnwMu1VbtW+5tqE7S6hWt7vszJ4N4JDSPJyrfwBKxIOVh+9LV9UH rEzmJmkqYuXuXkcNl4WRJ7tlHPatOby37y4cxl19Uh9vzVk55ZX6HaPyFyWkv8XuW0z/Im plvSC1sbGoVFVt/yyLNNzUdcbWyZKkacWLUUGoeGecu4j1W5FsG7s2spGsJAUizNPgfou6 6C3mojaRei4V6nalBhvmL03dE4jdBClMvPq/OPwAY8bXWdFwmp1oA44d0H2qkQ== Received: from icepick.vmeta.jp (unknown [IPv6:2403:bd80:c100:411:911b:1dab:8985:dd4a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: meta/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PVNTL1XxsztlB; Mon, 6 Mar 2023 02:54:30 +0000 (UTC) (envelope-from meta@FreeBSD.org) Date: Mon, 6 Mar 2023 11:54:24 +0900 From: Koichiro Iwao To: Pat Maddox Cc: freebsd-cloud@freebsd.org Subject: Re: How are official EC2 images built? Message-ID: <20230306025424.jx3rihykbswl4uk2@icepick.vmeta.jp> X-Operating-System: FreeBSD 13.2-STABLE amd64 References: <20230306021533.ght5ouvxg6zrihfr@icepick.vmeta.jp> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ThisMailContainsUnwantedMimeParts: N Thanks for the quick response! Both are definitely useful. On Sun, Mar 05, 2023 at 06:30:40PM -0800, Pat Maddox wrote: > https://cgit.freebsd.org/src/tree/release/Makefile.ec2 > > https://cgit.freebsd.org/src/tree/release/tools/ec2.conf > > I have opted to build images a bit differently, not using the stuff in src/release. I’ve done two approaches. > > The first is to attach a second disk to an EC2 instance, install FreeBSD onto it with customizations, and then image the disk (https://gist.github.com/patmaddox/9f20edcb50ff15fa9edd1d4e38d07e50). > > The way I’m doing it now is to use poudriere to make an img file, and then upload that image to S3, snapshot it (aws ec2 import-snapshot –disk-container Format=raw,Url=s3://mybucket/myimage.img) and finally make an AMI from that snapshot. > > The stuff in src/release is good for knowing what changes to make to config files, or packages to install. > > Pat > > On Sun, Mar 5, 2023, at 6:15 PM, Koichiro Iwao wrote: > > Hi, > > > > Does anyone how offical EC2 images [1] are built? > > I mean if the build script is public. I would like to create private > > custom images with some customization based on official image. > > > > [1] > > https://aws.amazon.com/marketplace/seller-profile?id=92bb514d-02bc-49fd-9727-c474863f63da&ref=dtl_prodview-ukzmy5dzc6nbq > > > > -- > > meta > -- meta From nobody Mon Mar 6 14:08:51 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PVgRc474xz3wwx0 for ; Mon, 6 Mar 2023 14:09:00 +0000 (UTC) (envelope-from meta@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PVgRc3WqLz3QPh; Mon, 6 Mar 2023 14:09:00 +0000 (UTC) (envelope-from meta@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678111740; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nsZEQVTjaZ4VD3FMO4Z2C19mPUsffqHHMt/Xq65xBAg=; b=Z2TzgzRP+k3kRVzhxHjeN5DFU1tRG2K6T9I9REfJEpYcrlE9F7DMuhfJgeE83A46A6bg6V qIP5FemROgT2Y8l1qyMNIisBL0kIyDeDqMgyImD9Cpj1oz5jNJMeZJdf+1JwogFZna0WIN rbdbMZRm57om4qCDhEX4APwOoPlf5q9t9IM5CMOvntjIDnzAe4i5JHaD3I4E0ci2WR64Se nuVSI/lIr9xeQ9XsUalZKQFmwQ4EwwThwsYjZPbxErtm/JgFA51JKAP7EFs0Rx9qcEUfff aAixFJqbHp5M7ats6i5xVsbzfXOasjZ1zWjmWUiFLCrdkl4phgWJhlRk+pbGRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1678111740; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nsZEQVTjaZ4VD3FMO4Z2C19mPUsffqHHMt/Xq65xBAg=; b=gwKb6OCCf70898A0Ecvr9SbQ83AJQenbqS25LOU+AgFlpE1Iwkr7203d9+c+up06+Ue7F4 ZQvRs7BJwuvYrF5vFMdCI5JSyrwilvuVzNmrB8fVKMkg4ldowjWundQgO0UidEMadFIQAI NDQnXV7M7WMBC4A9HRiz800oVO1UXnC12Ew59V4skBkAiJXHZjmGR4uveDgq6RfEtHUI+u jtZy/v4d/+5Gzg+uqbY6ukOWqpd2vemgwVbHAq/O+HRD65CVBDySOf0wcJ6PjZg1RPwH82 Mu2LKVPQm8jqawsXBam/mRb/CsPCdQJrDmT5JT9Fw2HqLwm9ztsr35tTGHel+g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1678111740; a=rsa-sha256; cv=none; b=HhDF7jIzHAU9J4W1/X5LivkIrOmGgCkwC8S1jxLgK8lwlf9IzRSgyAmeycY3Gr6PG0x5Pq 69XblVnWLKvSr983tHtqKhx0T5KgWY2I5DAWXYJhhu7nldKCcjiwumLwyBTzRkKAQth1dM rLp+gl7NaRAsL7HrEc4hx/2dlhcuOxLak0GZT5Y0aRyQVJ/8grrLtYMRH7Z/aUD+mHbmLb 3VSmHq7PfiWVbD0V2aDVfzmktNmW5mvHRl321yIVqC0dfQuX08AvKVI/6NiykSYjayY7j0 JKaOnBNKREgin7z3zoPbtsphnQlv8B/BcoMcA6k/G0z/foK6jzn/kfncLx9G0g== Received: from icepick.vmeta.jp (unknown [IPv6:2403:bd80:c100:411:e10f:5f62:d735:93a1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: meta/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PVgRb2McZz16WN; Mon, 6 Mar 2023 14:08:59 +0000 (UTC) (envelope-from meta@FreeBSD.org) Date: Mon, 6 Mar 2023 23:08:51 +0900 From: Koichiro Iwao To: Colin Percival Cc: freebsd-cloud@FreeBSD.org Subject: Re: How are official EC2 images built? Message-ID: <20230306140851.hduau66beidb7rne@icepick.vmeta.jp> X-Operating-System: FreeBSD 13.2-STABLE amd64 References: <20230306021533.ght5ouvxg6zrihfr@icepick.vmeta.jp> <01000186b5b4a976-9499f902-9004-4929-a9b3-6d6db764dc76-000000@email.amazonses.com> List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <01000186b5b4a976-9499f902-9004-4929-a9b3-6d6db764dc76-000000@email.amazonses.com> X-ThisMailContainsUnwantedMimeParts: N On Mon, Mar 06, 2023 at 06:56:01AM +0000, Colin Percival wrote: > On 3/5/23 18:15, Koichiro Iwao wrote: > > Does anyone how offical EC2 images [1] are built? > > They are built using the `make ec2ami` target in the src/release directory. > That target builds a disk image and then converts it into an AMI using the > bsdec2-image-upload utility (available in the ports tree). > > > I mean if the build script is public. I would like to create private > > custom images with some customization based on official image. > > I wrote about some of the options a few years ago: > https://www.daemonology.net/blog/2018-12-26-the-many-ways-to-launch-FreeBSD-in-EC2.html > > I'm guessing the AMI Builder option will best suit your need for "some > customization" but if you need to make changes to the base system then > you'll > want to use `make ec2ami`. Thanks, what I actually wanted to create is a qcow2 image with cloud-init/firstboot enabled to be run by vm-bhyve, removing EC2-specific stuffs from official EC2 images. I'll try to make EC2 images next. Anyway, it helps a lot. Thanks! -- meta From nobody Tue Apr 18 00:24:05 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0l6F3ZVWz45Q46 for ; Tue, 18 Apr 2023 00:24:21 +0000 (UTC) (envelope-from freebsd@igalic.co) Received: from mail-40136.proton.ch (mail-40136.proton.ch [185.70.40.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits)) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0l6D3R4tz3lxP for ; Tue, 18 Apr 2023 00:24:20 +0000 (UTC) (envelope-from freebsd@igalic.co) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=igalic.co header.s=protonmail header.b=xYd2rdWo; spf=pass (mx1.freebsd.org: domain of freebsd@igalic.co designates 185.70.40.136 as permitted sender) smtp.mailfrom=freebsd@igalic.co; dmarc=none Date: Tue, 18 Apr 2023 00:24:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igalic.co; s=protonmail; t=1681777459; x=1682036659; bh=WCuXFydo7+ywC7oabjl7OcQnc0wWrlPM3iUcwFGc1QU=; h=Date:To:From:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=xYd2rdWoFztmCxpG113Gvg+CoLTPc7lev04GyaHMtpxfqOn8YzhH+QhBKctmJGVHm FQ4/SYxrk0TdDHnH+y3p605/TbO5MbUSk9Lkp3bOl754Ymtrj/LwiPkp1jdynGlS1w thLF7GUvm1F1cj+w0MJSp/h0AD7EQuKlDaj2s7Xh9kanL/+pRohk9Ml8YEz5cdxzqY 7gfGxqoWzsRoKo6pdbluI6kHQOj0HzVwkEfmE5aTMsUISzakxJ0HpaAbiiPkS7brL7 SjM1PRxQLx1zw5HtmU2awBp3IP7DalpDygGrOw1DtR8jepASY0c1wELIXBhNAcxwFy a1EKLngl7AyYw== To: "freebsd-cloud@FreeBSD.org" From: =?utf-8?Q?Mina_Gali=C4=87?= Subject: Infiniband at Azure Message-ID: In-Reply-To: References: Feedback-ID: 66573723:user:proton List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.45 / 15.00]; R_MIXED_CHARSET(1.00)[subject]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.95)[-0.951]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24:c]; R_DKIM_ALLOW(-0.20)[igalic.co:s=protonmail]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; ARC_NA(0.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org]; DKIM_TRACE(0.00)[igalic.co:+]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_EQ_ADDR_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[igalic.co]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.136:from] X-Rspamd-Queue-Id: 4Q0l6D3R4tz3lxP X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N Hi folks! cross-posting because it concerns both topics! recently a cloud-init colleague working on Azure got me access to an Azure HPC instance =E2=80=94 which comes with two Infiniband Nvidia / Mellanox controllers. These show up with ibstat(8), but not under ifconfig(8). Under Linux, WA Agent is responsible for bringing Infiniband devices up to a base-line configuration: https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/enable-= infiniband On FreeBSD, WA Agent is rather poorly supported. So, how do I bring up these mlx5_0 devices? How do i make them show up in ifconfig? And is anyone interested in (me) fixing up WAAgent to work better on FreeBSD? Thank you for your help, Mina Gali=C4=87 Try PkgBase: https://alpha.pkgbase.live/ (as soon as i have a hardware sponsor again) From nobody Tue Jun 27 08:43:16 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qqysz6MxCz4jvGh for ; Tue, 27 Jun 2023 08:43:35 +0000 (UTC) (envelope-from sschols@lnwsoft.de) Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qqysz06lgz3pMX for ; Tue, 27 Jun 2023 08:43:35 +0000 (UTC) (envelope-from sschols@lnwsoft.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=lnwsoft-de.20221208.gappssmtp.com header.s=20221208 header.b="aQyOCJ/h"; spf=pass (mx1.freebsd.org: domain of sschols@lnwsoft.de designates 2a00:1450:4864:20::330 as permitted sender) smtp.mailfrom=sschols@lnwsoft.de; dmarc=none Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-3fb4146e8deso13834165e9.0 for ; Tue, 27 Jun 2023 01:43:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lnwsoft-de.20221208.gappssmtp.com; s=20221208; t=1687855413; x=1690447413; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=PxSvzse+SL89+Fv2k0k/tp2HMVI25XJC2B5hyp2DLRI=; b=aQyOCJ/hLEWE7XvQ1rpI6aOEwGIQ4yGmVnlNddMPJ036/Wjf3TdnP5Dm6tKZVP9bP7 hCzO1ZfvMEWZeyJ6X7V6biHJVsBU60ccPJvoRpF/h9IadQpkj2DyYhGTwq2K2hgBypKj FN+WTtULzKxcD8jU37sR6S89JyY4fyTeeDUQLA5o6p8XdWLbBNICvFX4XurlXdAzKwbL x5tZWITLr0e78nCnwb1ICKJLwcjsd58uuYTWyZ+bQwg8/EPp1gOToPEhFY2S/yYY0Hp8 N+SlS+75TkfLStL8WLOgdhie/fMHHRl3IhvtQFPEe0Bf9QrnvsjlA+SGaMrZzdIFKHXA ielw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687855413; x=1690447413; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=PxSvzse+SL89+Fv2k0k/tp2HMVI25XJC2B5hyp2DLRI=; b=BXF+LDcLzb9ZE0QnkLl5Rf9Tue3uNIZD0drAZSYv3Pi5pe85x/f7YD20duR89pxXKB x5yXuituIi0SfVMdap3UEWtSXQv9I0LFWotNHiSH9v67/WZPVuOAcYe+4ooDMJZzfMvZ rp35zLUsGk6m6K2NXclxYGBQHLI0dE7MLmz9URMfa1OrrOdupVqN3FIRrL/1nGwKGW5a lTIJlA8pR4fCrGY7ZWOWCh5yTU9wU11854A48tGM5LqMj/LTasKJvLkgkT1WCfVpoe/B 98Hf55Z8DWoehhim/xt7n0tEsv0amUoQrIrz7NAOuuFAUzsiupR5nJaPUEHMgaVEb5Xl D66g== X-Gm-Message-State: AC+VfDzLR9OcSNQuIo4a02WJvMJOA27qdJgObZ9jBlzJdX39SzVKAJ6B h3UY2wGt/KWlw9rcsl0lXxeOrt2HwGDAEO8M3rkaTnm/t3XIWXE/ X-Google-Smtp-Source: ACHHUZ5v3HbZyV87MA+5Ku+KuPtMBtmt3AC8DpV3+ZG9E5zBc/B/xl/Bz6iZYOLDyyccb2p7Roa14rQCg7Fldem0Xzk= X-Received: by 2002:a7b:c5d8:0:b0:3f1:789d:ad32 with SMTP id n24-20020a7bc5d8000000b003f1789dad32mr35172764wmk.11.1687855412761; Tue, 27 Jun 2023 01:43:32 -0700 (PDT) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 From: Severin Schols Date: Tue, 27 Jun 2023 10:43:16 +0200 Message-ID: Subject: FreeBSD in Azure within CSP subscriptions To: freebsd-cloud@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-3.22 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.72)[-0.722]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[lnwsoft-de.20221208.gappssmtp.com:s=20221208]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::330:from]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[lnwsoft-de.20221208.gappssmtp.com:+]; ARC_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[lnwsoft.de]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Qqysz06lgz3pMX X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Hi, we're trying to run some FreeBSD VMs in Azure, but found that we're not able to deploy the official FreeBSD marketplace images into CSP subscriptions. There seems to be a specific flag missing in the image publishing process. I've set this up for other marketplace offers, so I'd gladly help to get this sorted out. I've been trying to build my own images for Azure as a workaround, but couldn't get this to work either unfortunately (probably due to my lack of experience with the entire FreeBSD build ecosystem). Thanks, Severin From nobody Tue Jun 27 18:21:31 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QrCj50T5Hz4k0Zf for ; Tue, 27 Jun 2023 18:21:45 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [66.165.241.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.nomadlogic.org", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QrCj44mHfz4FBS for ; Tue, 27 Jun 2023 18:21:44 +0000 (UTC) (envelope-from pete@nomadlogic.org) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomadlogic.org; s=04242021; t=1687890095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jgFJy0Wzjxlcc8W20faxGODqI/pHFbaUYRNeP50cTPs=; b=wEixauWUq/p6NB0fBxB5rSuBF8Q1zBnLQI0RBonQZsdOFu2TBSI3tjkh62l/cq15AVxG/y hw0zgEDcUjvrYv1mcXqOcQ/Nf/OxaoOTyR6V/rk+mgQSp0q/Ht1sf+IhS9/vKWMZIOMr56 OV+oqQXHHF29BRaGTiorBq8IyKXz5iE= Received: from [192.168.1.160] (cpe-24-24-168-214.socal.res.rr.com [24.24.168.214]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id d8a7a2a1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 27 Jun 2023 18:21:32 +0000 (UTC) Message-ID: <349cae22-fede-bf51-80d6-d4d61fd7ea2b@nomadlogic.org> Date: Tue, 27 Jun 2023 11:21:31 -0700 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: FreeBSD in Azure within CSP subscriptions Content-Language: en-US To: Severin Schols , freebsd-cloud@freebsd.org References: From: Pete Wright In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4QrCj44mHfz4FBS X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:29802, ipnet:66.165.240.0/22, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 6/27/23 01:43, Severin Schols wrote: > Hi, > > we're trying to run some FreeBSD VMs in Azure, but found that we're > not able to deploy the official FreeBSD marketplace images into CSP > subscriptions. There seems to be a specific flag missing in the image > publishing process. I've set this up for other marketplace offers, so > I'd gladly help to get this sorted out. I would suggest filing a bug report to ensure enough eyes get on it.  i'm not familiar with Azure CSP subscriptions, is there a good link to learn more about it? > I've been trying to build my own images for Azure as a workaround, but > couldn't get this to work either unfortunately (probably due to my > lack of experience with the entire FreeBSD build ecosystem). i have a bit of experience with AWS images so happy to lend a second pair of eyes to help debug any issues you run into.  more of my customers are moving to Azure, so i have a vested interest in making this easy :) -pete -- Pete Wright pete@nomadlogic.org @nomadlogicLA From nobody Fri Oct 6 04:43:16 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4S1wm60HPdz4vnW9 for ; Fri, 6 Oct 2023 04:43:18 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from mail.tarsnap.com (mail.tarsnap.com [54.86.246.204]) by mx1.freebsd.org (Postfix) with SMTP id 4S1wm55Glzz4psw for ; Fri, 6 Oct 2023 04:43:17 +0000 (UTC) (envelope-from cperciva@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: (qmail 28201 invoked from network); 6 Oct 2023 04:43:16 -0000 Received: from unknown (HELO dell7390.daemonology.net) (127.0.0.1) by mail.tarsnap.com with SMTP; 6 Oct 2023 04:43:16 -0000 Received: (qmail 2134 invoked from network); 6 Oct 2023 04:43:16 -0000 Received: from unknown (HELO ?127.0.0.1?) (127.0.0.1) by localhost with SMTP; 6 Oct 2023 04:43:16 -0000 Message-ID: <22680d78-fbaa-9eb8-791a-44cb3925f8a7@freebsd.org> Date: Thu, 5 Oct 2023 21:43:16 -0700 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Subject: Re: New FreeBSD snapshots available: main (20231005 8818f0f1124e) Content-Language: en-US To: Glen Barber , freebsd-snapshots@freebsd.org, "freebsd-cloud@freebsd.org" References: <20231005214328.GB1307@FreeBSD.org> From: Colin Percival In-Reply-To: <20231005214328.GB1307@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:14618, ipnet:54.86.0.0/16, country:US] X-Rspamd-Queue-Id: 4S1wm55Glzz4psw On 10/5/23 14:43, Glen Barber wrote: > === Amazon EC2 AMI Images === > > FreeBSD/amd64 EC2 AMI IDs can be retrieved from the Systems Manager > Parameter Store in each region using the keys: > > /aws/service/freebsd/amd64/base/ufs/15.0/CURRENT > /aws/service/freebsd/amd64/base/zfs/15.0/CURRENT > > FreeBSD/aarch64 EC2 AMI IDs can be retrieved from the Systems Manager > Parameter Store in each region using the keys: > > /aws/service/freebsd/aarch64/base/ufs/15.0/CURRENT > /aws/service/freebsd/aarch64/base/zfs/15.0/CURRENT Also, we have images with all of those paths with "base" replaced by "cloud-init" -- we're now building (experimental!) cloud-init AMIs. These will also be present (hopefully!) in tomorrow's 14.0-BETA5 builds. On my to-do list (but almost certainly not arriving until after 14.0-RELEASE) are ami-builder (which I've built manually after the fact until now), minimal (no debug symbols, and possibly with other things removed, I haven't decided yet), and patched (automatically re-rolled to have the latest security updates applied). These are, of course, subject to me finding time to work on this stuff. -- Colin Percival FreeBSD Deputy Release Engineer & EC2 platform maintainer Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid From nobody Sat Dec 2 00:56:41 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Shs2R6P21z52nlZ for ; Sat, 2 Dec 2023 00:56:47 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Received: from mail-relay3.dca2.superb.net (mail-relay3.dca2.superb.net [66.148.95.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Shs2R2BBKz3FHx for ; Sat, 2 Dec 2023 00:56:47 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of postmaster@shaneware.biz has no SPF policy when checking 66.148.95.60) smtp.mailfrom=postmaster@shaneware.biz; dmarc=none Received: from ppp230-11.static.internode.on.net ([203.122.230.11] helo=[192.168.9.155]) by mail-relay3.dca2.superb.net with esmtpsa (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (envelope-from ) id 1r9EJI-0000gR-Pv for freebsd-cloud@FreeBSD.org; Fri, 01 Dec 2023 19:56:46 -0500 Message-ID: <9f04355d-4657-7781-0797-3518a164119d@ShaneWare.Biz> Date: Sat, 2 Dec 2023 11:26:41 +1030 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-US To: FreeBSD Cloud From: Shane Ambler Subject: Resizing EBS with ZFS on AWS Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; RWL_MAILSPIKE_VERYGOOD(-0.20)[66.148.95.60:from]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org]; ASN(0.00)[asn:14361, ipnet:66.148.95.0/24, country:US]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[postmaster]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[shaneware.biz]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Shs2R2BBKz3FHx X-Spamd-Bar: - Hi, I recently started an EC2 with the 14.0 ZFS AMI After filling the drive, I expanded it, recovered the gpart partition and expanded the partition and zpool size and kept going. Then I saw that I would need more space, so I have expanded the EBS vol again, while the EC2 was shutdown. This time the EC2 will not boot. Using the EC2 console I get to... nvme0: SET_FEATURES (09) sqid:0 cid:15 nsid:0 cdw10:0000000b cdw11:0000031f nvme0: INVALID_FIELD (00/02) crd:0 m:1 dnr:0 p:1 sqid:0 cid:15 cdw0:0 nda0 at nvme0 bus 0 scbus0 target 0 lun 1 nda0: nda0: Serial Number vol0f445f0597836b567 nda0: nvme version 1.4 nda0: 153600MB (314572800 512 byte sectors) Trying to mount root from zfs:zroot/ROOT/default []... Mounting from zfs:zroot/ROOT/default failed with error 2: unknown file system. GEOM: nda0: the secondary GPT header is not in the last LBA. What can I do at mountroot> to recover gpart? or boot from here? -- FreeBSD - the place to B...Serving Data Shane Ambler From nobody Sat Dec 2 04:30:37 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ShxnQ4D0Pz52XtV for ; Sat, 2 Dec 2023 04:30:50 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Received: from mail-relay2.dca2.superb.net (mail-relay2.dca2.superb.net [66.148.95.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ShxnP5xKFz4JV0 for ; Sat, 2 Dec 2023 04:30:49 +0000 (UTC) (envelope-from postmaster@shaneware.biz) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of postmaster@shaneware.biz has no SPF policy when checking 66.148.95.42) smtp.mailfrom=postmaster@shaneware.biz; dmarc=none Received: from ppp230-11.static.internode.on.net ([203.122.230.11] helo=[192.168.9.155]) by mail-relay2.dca2.superb.net with esmtpsa (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (envelope-from ) id 1r9HeM-00006A-V0 for freebsd-cloud@FreeBSD.org; Fri, 01 Dec 2023 23:30:43 -0500 Message-ID: Date: Sat, 2 Dec 2023 15:00:37 +1030 List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: Resizing EBS with ZFS on AWS Content-Language: en-US From: Shane Ambler To: FreeBSD Cloud References: <9f04355d-4657-7781-0797-3518a164119d@ShaneWare.Biz> In-Reply-To: <9f04355d-4657-7781-0797-3518a164119d@ShaneWare.Biz> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-0.53 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; FORGED_SENDER(0.30)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; NEURAL_SPAM_SHORT(0.27)[0.275]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14361, ipnet:66.148.95.0/24, country:US]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[freebsd-cloud@FreeBSD.org]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[postmaster]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[FreeBSD@ShaneWare.Biz,postmaster@shaneware.biz]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[shaneware.biz]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4ShxnP5xKFz4JV0 X-Spamd-Bar: / On 2/12/23 11:26, Shane Ambler wrote: > Hi, > > I recently started an EC2 with the 14.0 ZFS AMI > > After filling the drive, I expanded it, recovered the gpart partition > and expanded the partition and zpool size and kept going. > > Then I saw that I would need more space, so I have expanded the EBS vol > again, while the EC2 was shutdown. This time the EC2 will not boot. > > Using the EC2 console I get to... > > Trying to mount root from zfs:zroot/ROOT/default []... > Mounting from zfs:zroot/ROOT/default failed with error 2: unknown file > system. > GEOM: nda0: the secondary GPT header is not in the last LBA. > > > What can I do at mountroot> to recover gpart? or boot from here? > OK so it seems the expanding disk was a red herring, I had altered loader.conf to break boot. I came up with steps to attach the unbootable EBS to another instance, import the pool, fix loader.conf, then re-attached to the original instance and got it to boot. Might need to document these cloud recovery steps somewhere. -- FreeBSD - the place to B...Scaring Daemons Shane Ambler From nobody Tue Dec 12 16:48:42 2023 X-Original-To: freebsd-cloud@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SqPhT4c4Gz54B76 for ; Tue, 12 Dec 2023 16:48:57 +0000 (UTC) (envelope-from wschnr@googlemail.com) Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SqPhS5GwHz3DZV for ; Tue, 12 Dec 2023 16:48:56 +0000 (UTC) (envelope-from wschnr@googlemail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of wschnr@googlemail.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=wschnr@googlemail.com; dmarc=none Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-a1db99cd1b2so737462566b.2 for ; Tue, 12 Dec 2023 08:48:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702399734; x=1703004534; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=obgRjA15ra0BZTqLKFC2+ae/T9Y4sJ8OLnpnLW0AURE=; b=mmbd9EkRsyVC6M8tpc7sDnRAojH0T0zDPraKvAUyqmOAeiKGY+P1mS7FPyCIBsjJVv H9pP3eTyjriMFStBewjai7uUkUyne/kHVRYpD0vgVV1CBtlOLBJv2/fRcafIWYBM7u/o uVENlJuOvJMskWXxItGfhE+TaMkqOK6j/W8mVqAtvyBcZzWo5zAY10cSACD1/jcfmjNQ jcx44OI6Ih8tMvK9OVYjpJ2r2+6b9bewm1g9i+RntdMy5IWQ/38W+Y3Gy9OD6QtlcpBL YWetvmtF92Zap8DSKpYRtV0kM9zHv7vO8bDXsXk22wtYe/Qp+v0rLLYoeYOlZ/AIEFhE cTYg== X-Gm-Message-State: AOJu0YxsJVP92M1fGbGgb3fl/+GpEOSbkZEozA/iusghfZC/4+Sew/H0 49UvDkM5p56cyqFMPAm+DFHXrcNNrvI/RoV7P3xHeF8jpEye4Q== X-Google-Smtp-Source: AGHT+IGWY/If17aLTTi0yUfxpB9XpG+K1xWdE6AXIQZibpds5PxnHPUfqjJoSy1T76YbfRU7liMR/oIHRfNyet9zjiw= X-Received: by 2002:a17:907:d9e:b0:a19:b93c:d294 with SMTP id go30-20020a1709070d9e00b00a19b93cd294mr3206205ejc.78.1702399734252; Tue, 12 Dec 2023 08:48:54 -0800 (PST) List-Id: FreeBSD on cloud platforms (EC2, GCE, Azure, etc.) List-Archive: https://lists.freebsd.org/archives/freebsd-cloud List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-cloud@freebsd.org MIME-Version: 1.0 From: Wolfram Schneider Date: Tue, 12 Dec 2023 17:48:42 +0100 Message-ID: Subject: FreeBSD 14.0-RELEASE/amd64 runs fine on Hetzner cloud To: freebsd-cloud@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [1.54 / 15.00]; NEURAL_SPAM_LONG(1.00)[1.000]; NEURAL_SPAM_MEDIUM(0.99)[0.990]; NEURAL_HAM_SHORT(-0.35)[-0.346]; FORGED_SENDER(0.30)[wosch@freebsd.org,wschnr@googlemail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; RWL_MAILSPIKE_GOOD(-0.10)[209.85.218.42:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-cloud@freebsd.org]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; R_DKIM_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[wosch@freebsd.org,wschnr@googlemail.com]; RCVD_IN_DNSWL_NONE(0.00)[209.85.218.42:from]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[googlemail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-cloud@freebsd.org]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Rspamd-Queue-Id: 4SqPhS5GwHz3DZV X-Spamd-Bar: + Hi, I have tried FreeBSD 14.0-RELEASE/amd64 on the cloud servers of hetzner.com. The installation and update to stable/14 went without any problems. To install FreeBSD 14.0-RELEASE/amd64, you need to purchase an x86 server, select the ISO-IMAGE FreeBSD-14.0-RELEASE-amd64-bootonly.iso, mount the image, reboot the server and run the installation process on the console. have fun, Wolfram -- Wolfram Schneider https://wolfram.schneider.org