From owner-freebsd-virtualization@freebsd.org Fri Jul 19 02:53:46 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1553CB7453 for ; Fri, 19 Jul 2019 02:53:46 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [174.136.98.114]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.nomadlogic.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 91BD983A32 for ; Fri, 19 Jul 2019 02:53:44 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from [192.168.1.206] (cpe-23-243-162-239.socal.res.rr.com [23.243.162.239]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id 231fb6d8 TLS version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO for ; Fri, 19 Jul 2019 02:53:42 +0000 (UTC) Subject: Re: Custom Qcow2 Images To: freebsd-virtualization@freebsd.org References: <459c53d0-f00b-6ce3-dfab-de3ddac1c0bb@nomadlogic.org> From: Pete Wright Message-ID: <95a027df-46c3-24ba-8acd-ddb24579885e@nomadlogic.org> Date: Thu, 18 Jul 2019 19:53:42 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <459c53d0-f00b-6ce3-dfab-de3ddac1c0bb@nomadlogic.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 91BD983A32 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of pete@nomadlogic.org designates 174.136.98.114 as permitted sender) smtp.mailfrom=pete@nomadlogic.org X-Spamd-Result: default: False [3.37 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_LONG(-0.06)[-0.064,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(0.00)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_MEDIUM(0.67)[0.670,0]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; DMARC_NA(0.00)[nomadlogic.org]; MX_GOOD(-0.01)[cached: mail.nomadlogic.org]; NEURAL_HAM_SHORT(-0.90)[-0.902,0]; URIBL_SBL(6.50)[bsdrp.net]; IP_SCORE(-2.83)[ip: (-9.25), ipnet: 174.136.96.0/20(-4.56), asn: 25795(-0.27), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:25795, ipnet:174.136.96.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 02:53:46 -0000 On 7/14/19 4:37 PM, Pete Wright wrote: > Hey there folks, > I am wondering if anyone has any pointers on creating a custom Qcow2 > FreeBSD that is akin in size to nanobsd.  I have an environment where > we need to run a single binary and want to keep our disk image as > small as possible.  It will run inside a user-mode Qemu process.  > Reading through the nanobsd script it seems that it expects a raw disk > for installation, so I do think using the nanobsd script itself well > work well.  Has anyone else done something similar?  My goal is to > have a disk image that is around 500MB. > > Thanks in advance! > -pete > I wanted to close the loop on this discussion as I think I've found a workable solution for my use-case.  I'm using a three step process: 1) generate a raw disk image using "poudriere image".  This phase references a Jail I've built that has disabled lots of uneeded features as per the docs here: https://bsdrp.net/documentation/technical_docs/poudriere 2) next I use mkimg(1) to convert the raw disk image to a bootable qcow2 image that I can boot from Qemu.  this results in a disk image that's about 800MB. 3) the final phase is to use qemu-img to create a compressed qcow2 image.  the resulting artifact of that process is a ~325MB Qcow2 diskimage that actually has decent performance (on my SSD backed zfs filesystem). I attempted to use poudriere image to generate a usb image directly, and while that did work well sizing wise i ran into an issue where insufficient inodes were allocated.  I reference what I ran into in this github issue: https://github.com/freebsd/poudriere/issues/701 For me this current setup is fine for my purposes, although I'm sure others can optimize this workflow a bit :) Also, thanks again Allan and Dave Cottlehuber (who pinged me offlist) for giving me some hints and pointing me in the right direction! Cheers, -pete -- Pete Wright pete@nomadlogic.org @nomadlogicLA