From nobody Sat Feb 24 21:47:52 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tj0sj6qrgz5Bg1Y for ; Sat, 24 Feb 2024 21:50:01 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail5.jnielsen.net (webmail5.jnielsen.net [69.87.218.172]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.freebsdsolutions.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tj0sf6fMKz4Vhk for ; Sat, 24 Feb 2024 21:49:58 +0000 (UTC) (envelope-from lists@jnielsen.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of lists@jnielsen.net designates 69.87.218.172 as permitted sender) smtp.mailfrom=lists@jnielsen.net Received: from smtpclient.apple (136-36-176-5.googlefiber.net [136.36.176.5] (may be forged)) (authenticated bits=0) by webmail5.jnielsen.net (8.17.2/8.17.1) with ESMTPSA id 41OLm6IM030217 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Sat, 24 Feb 2024 14:48:11 -0700 (MST) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail5.jnielsen.net: Host 136-36-176-5.googlefiber.net [136.36.176.5] (may be forged) claimed to be smtpclient.apple Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: John Nielsen List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: Best way to have a FreeBSD VM for automated testing? Message-Id: Date: Sat, 24 Feb 2024 14:47:52 -0700 Cc: freebsd-virtualization@freebsd.org To: Jo Durchholz X-Mailer: iPhone Mail (21C66) X-Spamd-Bar: / X-Spamd-Result: default: False [0.73 / 15.00]; THREAD_HIJACKING_FROM_INJECTOR(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; FAKE_REPLY(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.973]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; APPLE_IOS_MAILER_COMMON(0.00)[]; DMARC_NA(0.00)[jnielsen.net]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:6364, ipnet:69.87.218.0/24, country:US]; HAS_XAW(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; TO_DN_SOME(0.00)[] X-Rspamd-Queue-Id: 4Tj0sf6fMKz4Vhk =EF=BB=BF >=20 > On Feb 23, 2024, at 12:17=E2=80=AFPM, Jo Durchholz wrot= e: >=20 > I'm in repeatable build land, working in Linux and developing a FreeBSD ap= pliance. >=20 > For tests, I need to run a FreeBSD VM, put some Python code and test data i= nto it, run the script, and get the test results back. >=20 > Repeatability means: Everything done with the VM needs to be scriptable (u= sing a GUI for exploring is okay but things have to translate). > Which in turn means that every setup step for a FreeBSD image comes with a= pretty high coding and maintenance cost. >=20 > So my question is: > What's the FreeBSD image that has the least number of steps to get the bas= e system up and running? I suppose it's the VM-IMAGES section, but is this c= orrect? >=20 > Follow-up question: > The startup time needs to be as fast as possible. Sub-second would be grea= t ("don't disrupt the developer's thought stream"). > I see the boot process from a vanilla VM-IMAGES image takes multiple secon= ds; can this be sped up to just a few seconds, or do I need to run the setup= and create a VM snapshot at which the VM starts for each test run? I loved Colin=E2=80=99s presentation on Firecracker and reducing boot times.= If fast boot is your primary requirement then for sure pursue that as other= s have suggested. I wanted to mention two alternatives that might require le= ss lift up front: 1) AWS EC2 + cloud-init. The FreeBSD Project publishes AMIs for every releas= e (again, thanks cperciva). You can launch an EC2 instance from one of those= with a few clicks. You can additionally assign an IAM role to the instance t= o give it permission to write to an S3 bucket and drop a script in the user-= data for the instance to run when it boots. The script could download the Py= thon code to be run, run it, and upload the results to S3 all non-interactiv= ely. With a couple hours=E2=80=99 work you could adapt the stock image to add any= missing dependencies and disable unneeded drivers and services to speed the= boot, then snapshot that and use your custom image for future runs. You could also easily manage the whole thing with Terraform. Copy/paste the i= nfrastructure as code templates and tweak the variables to get an easily rep= eatable system. 2) KVM + libvirt on whatever Linux host(s) you have available. The VM-IMAGES= released by the Project include qcow volumes. You might need to tweak the i= mage to do your custom script(s) at startup (though the image may also have c= loud-init if your environment supports it) but qcow does support snapshots a= s well and the whole thing should have low boot times. JN