From nobody Fri May 20 16:35:25 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 269841AE7870 for ; Fri, 20 May 2022 16:35:37 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qk1-x730.google.com (mail-qk1-x730.google.com [IPv6:2607:f8b0:4864:20::730]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4L4XQc0zxlz4Twc; Fri, 20 May 2022 16:35:36 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qk1-x730.google.com with SMTP id v11so7312041qkf.1; Fri, 20 May 2022 09:35:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=xYwmouT0CmVvLHmTVnU4uDnlJZVK6IlxKIiS7jvaFEQ=; b=eBWFK757nZE2oZtQ2Xwxu7kxmOC+76ModpkFpU74En998oTSUmKKXR98RJS8FNVN7n ZQZfIFCU0GvvniTsaCCNv44AyAdcvfH+VF8EDaHrkW07t6sZfHBeZ/jNvLXl3uyDV2iQ e9nHuhrmvlCaJBGQaP5tpHkuxaeWQIU5SZj/jLt0jnrmLAj2CnF/lz3syKPBEfEUOyOn SUzYl7opwzbmzdKwAzsqjhr1tvEKAzVW10wDHM4QX3w/XV5IAKJo6UKEJQ9BCtsGUb42 o8pT0LfLC02JnW1/VUoJm3dWdiMMSioXDwO88OwmoRFxLIWePcLni8KXjAhe6EXdQBmJ k1Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=xYwmouT0CmVvLHmTVnU4uDnlJZVK6IlxKIiS7jvaFEQ=; b=dUdwOwmgyZxfZm5AdLLUZV1sQT/CII5FuHskFIwyHUvrz7DgDFUd6mSFni+QBY+2DD q1baLd0qzYCmCnJGk6Gpx3gOYDc4koumEikGswykn0kb77R4ZbMLFHmsPRh2XnWU8ncm gkpaZOeoWQo7sZaZ7Rv/GKE34G8Hdj884Mba3EntiqNCX/DRvNaR9g5gihv2rjtS9ol5 pLGSbfn0QhzQU7Ze7edp9/yCuprKzHkCMHgAzqYyv46ntlzAxBkF2peZeAmwZmN8F4EZ aeE7/n3vAT055UlsegdTmQEXbfAZ4nQmrojk35/Yy11rgUe1Ya0k/iN86fWpJzwSvYS2 gY5g== X-Gm-Message-State: AOAM5304ApXzclPGIrgj1QT6vgkQVq+5Vdr6Z1CGhHSQTlvlW32rCNhK xjjc5m0DvY/mewM9GF1upatxU8oB30c= X-Google-Smtp-Source: ABdhPJw3i+eQNofw+Dfv/jYHzpFr0bXGtskmxUC9dJX2ycEqv8Agvcg6Zs/XZBtFg11c+TGskaBlXQ== X-Received: by 2002:ae9:e842:0:b0:69f:c3ea:8233 with SMTP id a63-20020ae9e842000000b0069fc3ea8233mr6769919qkg.320.1653064529095; Fri, 20 May 2022 09:35:29 -0700 (PDT) Received: from nuc (198-84-189-58.cpe.teksavvy.com. [198.84.189.58]) by smtp.gmail.com with ESMTPSA id h127-20020a376c85000000b006a10aa7908dsm2365954qkc.38.2022.05.20.09.35.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 09:35:28 -0700 (PDT) Date: Fri, 20 May 2022 12:35:25 -0400 From: Mark Johnston To: Brooks Davis Cc: Allan Jude , freebsd-hackers@freebsd.org Subject: Re: zfs support in makefs Message-ID: References: <20220518230427.GI15201@spindle.one-eyed-alien.net> <20220519182532.GJ15201@spindle.one-eyed-alien.net> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220519182532.GJ15201@spindle.one-eyed-alien.net> X-Rspamd-Queue-Id: 4L4XQc0zxlz4Twc X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=eBWFK757; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::730 as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.69 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; NEURAL_HAM_MEDIUM(-0.99)[-0.989]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_NA(0.00)[freebsd.org]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::730:from]; MLMMJ_DEST(0.00)[freebsd-hackers]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N On Thu, May 19, 2022 at 06:25:32PM +0000, Brooks Davis wrote: > On Thu, May 19, 2022 at 01:36:25PM -0400, Allan Jude wrote: > > On 5/18/2022 7:04 PM, Brooks Davis wrote: > > > On Wed, May 18, 2022 at 03:03:17PM -0400, Mark Johnston wrote: > > >> Hi, > > >> > > >> For the past little while I've been working on ZFS support in makefs(8). > > >> At this point I'm able to create a bootable FreeBSD VM image, using the > > >> standard FreeBSD ZFS layout, and run through the regression test suite > > >> in bhyve. I've also been able to create and boot an EC2 AMI. > > > > > > Very cool! > > > > > >> === Interface === > > >> > > >> Creating a pool with a single dataset is easy: > > >> > > >> $ makefs -t zfs -s 10g -o poolname=test ./zfs.img /path/to/input > > >> > > >> Upon importing such a pool, you'll get a dataset named "test" mounted at > > >> /test containing everything under /path/to/input. > > >> > > >> It's possible to set properties on the root dataset: > > >> > > >> $ makefs -t zfs -s 10g -o poolname=test -o fs=test:setuid=off:atime=on ./zfs.img /path/to/input > > >> > > >> It's also possible to create additional datasets: > > >> > > >> $ makefs -t zfs -s 10g -o poolname=test -o fs=test/ds1:mountpoint=/test/dir1 ./zfs.img /path/to/input > > >> > > >> The parameter syntax is > > >> "-o fs=[:=[:=[:...]]]". Only a > > >> few properties are supported, at least for now. > > >> > > >> Dataset mountpoints behave the same as they would if created with the > > >> standard ZFS tools. So by default the root dataset's mountpoint is > > >> /test, test/ds1's mountpoint is /test/ds1, etc.. If a dataset overrides > > >> its default mountpoint, its children inherit that mountpoint. > > >> > > >> makefs builds the output filesystem using a single input directory tree. > > >> Thus, makefs -t zfs requires that at least one of the dataset's > > >> mountpoints map to /path/to/input; that is, there is a "root" mount > > >> point. > > >> > > >> The -o rootpath parameter defines this root mount point. By default it's > > >> "/". All datasets in the pool must have their mountpoints > > >> under this path, and one dataset's mountpoint must be equal to this > > >> path. To build bootable images, one sets -o rootpath=/. > > >> > > >> Putting it all together, one can build a image using the standard layout > > >> with an invocation like this: > > >> > > >> makefs -t zfs -o poolname=zroot -s 20g -o rootpath=/ -o bootfs=zroot/ROOT/default \ > > >> -o fs=zroot:canmount=off:mountpoint=none \ > > >> -o fs=zroot/ROOT:mountpoint=none \ > > >> -o fs=zroot/ROOT/default:mountpoint=/ \ > > >> -o fs=zroot/tmp:mountpoint=/tmp:exec=on:setuid=off \ > > >> -o fs=zroot/usr:mountpoint=/usr:canmount=off \ > > >> -o fs=zroot/usr/home \ > > >> -o fs=zroot/usr/ports:setuid=off \ > > >> -o fs=zroot/usr/src \ > > >> -o fs=zroot/usr/obj \ > > >> -o fs=zroot/var:mountpoint=/var:canmount=off \ > > >> -o fs=zroot/var/audit:setuid=off:exec=off \ > > >> -o fs=zroot/var/crash:setuid=off:exec=off \ > > >> -o fs=zroot/var/log:setuid=off:exec=off \ > > >> -o fs=zroot/var/mail:atime=on \ > > >> -o fs=zroot/var/tmp:setuid=off \ > > >> ${HOME}/tmp/zfs.img ${HOME}/tmp/world > > >> > > >> I'll admit this is somewhat clunky, but it doesn't seem worse than what > > >> we have to do otherwise, see poudriere-image for example: > > >> https://github.com/freebsd/poudriere/blob/master/src/share/poudriere/image_zfs.sh#L79 > > >> > > >> What do folks think of this interface? Is there anything missing, or > > >> anything that doesn't make sense? > > > > > > I find it slightly confusing that -o options have a default namespace of > > > pool options unless they have an fs=*: prefix, but making users type > > > "pool:" for other options doesn't seem to make sense so this is probably > > > the best solution. > > > > > > The density of data in the filesystem specification does suggest that > > > someone might want to create a UCL config file format eventually, but > > > what's here already seems entirely workable. > > > > > > -- Brooks > > > > In normal `zpool create` they use -o for pool properties, and -O for > > dataset properties for the root dataset. I wonder if we might also want > > -o poolprop=value and -O zroot/var:mountpoint=/var:canmount=off > > > > just to avoid the conceptual collision of those 2 different items. > > Sadly -O is taken in makefs. Though, -O is already not supported for all filesystem types (cd9660 in particular). I'm not sure whether -O is at all useful anymore now that we have mkimg(1): I presume that -O is useful when you already have a partitioned disk image and want to fill in one of the partitions with a filesystem. There's a suggestion in the thread of having multiple hardlinks of makefs; we could add a makefs_zfs which handles -O as Allan suggests. > > One other possible issue: dataset properties can have a : in them, for > > user-defined properties. Do we maybe want to use a , to separate them > > instead? Although values can contain ,'s (the sharenfs property often > > does), so that probably doesn't work either. > > One solution would be to allow the same fs=foo: to be specified multiple > times (I've not checked if the current code allows this) to add options > instead of having a separator. That does make the command line even more > clunky though. The current code won't allow this, but it would be easy to add of course. Maybe we should support both modes. Or maybe the real solution is to introduce a UCL configuration format and keep the command-line interface simple. I didn't think about this much yet since makefs currently doesn't support setting arbitrary properties, just those few that I need to build FreeBSD images. I guess I'd want to see some specific use-cases for specifying additional dataset/pool properties before deciding what to do.