From owner-freebsd-arch@FreeBSD.ORG Mon Sep 17 21:34:03 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 89CF510656EE for ; Mon, 17 Sep 2012 21:34:03 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 49C478FC12 for ; Mon, 17 Sep 2012 21:34:03 +0000 (UTC) Received: by iea17 with SMTP id 17so7828829iea.13 for ; Mon, 17 Sep 2012 14:34:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=KFfRD4GRbPTRrEExqYGvPUIizV17XsbeQTCxsnJ6L/A=; b=em90d3eJ/kAqc5061ruZWRIhlaqkMktmnzCX+OkJwErrA68cmSA9juF490Tz1JrsfW UsUGodI1H7HHLj1tkfOBzRhCDP+93n1jezK4ysGOK0wgVqURljr+R4urIHJa7DQiiLZk ULI2QLXp8IDcZ171/xEr5cmm9y2jKPcrj6X8B+bHTH+Nl5wdcZEr/tz+gIdTOQYw1SeX i+9r/HEapspWrEWJ0l6XjGe5Xi8ZBVeMgcd/38t2jnEDhiOR2F1RLUr6f6Slc1l08Q7W MIFCG0dLNWR6gZ0M+Hl0E5xZ/mPN12xAYeAbMtSkYGlF3dOmi/YnA1/Sb8fDem2q6HeC 7f4A== Received: by 10.50.158.226 with SMTP id wx2mr8308148igb.70.1347917642768; Mon, 17 Sep 2012 14:34:02 -0700 (PDT) Received: from 63.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id q1sm19686951igj.15.2012.09.17.14.33.57 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 17 Sep 2012 14:34:00 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <20120917201919.GA43626@lor.one-eyed-alien.net> Date: Mon, 17 Sep 2012 15:33:57 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120917201919.GA43626@lor.one-eyed-alien.net> To: Brooks Davis X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQlkK0f1YY/HKiJGLKezxUIRgPNvrognMVINBwMVI6csBZck2zrG78lZpPWvNRVZMRXt/sfs Cc: arch@freebsd.org Subject: Re: Importing NetBSD's mtree (and install) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2012 21:34:03 -0000 On Sep 17, 2012, at 2:19 PM, Brooks Davis wrote: > As part of an effort to improve our ability to create full system = images > without root access I would like to import NetBSD's version of mtree. > By doing so we would gain the -C option which produces mtree files > compatible with libarchive and makefs (one line per file with full > path) and the -N option which allows a stand alone set of passwd and > group files. >=20 > When mated with the -U and -M options to NetBSD's install we will have > most[0] of the pieces require to allow installworld to run as a user = and > then build images containing proper permissions. The rest of this = post > will focus on my plans for mtree since it is the logical next step. >=20 > NetBSD's mtree is missing a few features present in our mtree. Most = of > them looks simple to implement and I plan to do so before any import. > The only exception is -i which implements indenting of old-style mtree > files to match the format of the files in /etc/mtree/. It will either > need another name or to be dropped. I honestly don't see the point in > it so I'm not sure if it's worth keeping if people will need to alter > their scripts regardless, but I'm willing to be convinced otherwise. >=20 > Importing mtree requires importing or implementing some enhancements = to > libc. First, the strsvis() function is required which is most > easily handled by importing NetBSD's vis/unvis implementations with = the > addition of the VIS_GLOB set of characters. Compatibly wrappers will > be required for existing vis(3) functions and for unvis() due to ABI > changes, but they will be minimal. >=20 > Second, pwcache_userdb(), uid_from_user(), pwcache_groupdb(), and > gid_from_group() provide useful enhancements to the uid_from_user() > and group_from_gid(). They appear to be entirely compatible with our > current implementation so simply importing the enhanced version seems > like the best course. >=20 > Finally, FreeBSD and Mac OS have the functions fflagstostr() and > strtofflags() in libc. NetBSD has flags_to_string() and > string_to_flags() in libutil. The latter could be a very thin wrapper > around fflagstostr() and the former is exactly strtofflags(). I think > the best course is probably to provide compatible wrappers for mtree's > internal use, but I could be convinced to make them more globally > available. >=20 > Does anything about this plan seem seriously objectionable? Looks good to me. I started doing this a while ago an ran into lots of = problems, many of which you've outlined here. They weren't hard = problems, just numerous enough for me to lose interest in the project = before I completed it. Glad to see somebody has pushed through. > I've written some of this up along with a list of missing features in > the wiki. I'll keep progress up to date there: >=20 > http://wiki.freebsd.org/NetBSDMtree >=20 > -- Brooks >=20 > [0] We also lack a tool to build disk images including partition = tables, > but Marcel is looking into this. makefs will build the disk image w/o the MBR/GPT tables. For most flash = devices, this is sufficient. For more complex situations, like where = the boot loader groks FAT but not UFS, that needs some help... Once upon = a time, you could use dd + fdisk to create an MBR image, but nothing = apart from the kernel understands using it. gpart is so darn easy to = use, it is a shame that you have to make a privileged trip to the kernel = to use it.=20 Warner=