From owner-freebsd-arch@freebsd.org Mon Nov 30 05:49:12 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0158FA3123E for ; Mon, 30 Nov 2015 05:49:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A312C17B3 for ; Mon, 30 Nov 2015 05:49:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by qkfo3 with SMTP id o3so55035405qkf.1 for ; Sun, 29 Nov 2015 21:49:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gqsigX2gmjuSDeOLlvZlBixwGucPHt7trIKLKPliU38=; b=wDOP3xccnv8qSeSxSgF/hsfSmTEW8hiekDTYZqVeKu74RqKdLz0Ry+JK96H2sNzAbr 6yzI3P/uCtunSjKQS4iBIr7r22q5t6h3dPTAw3Ex+bpY9Jv0QQZf8uSW2PVHAzcuAlrk b6dVGZOm6rQ4YKQXVVAcc6NO7u7i/BKuFl0ET363EiT64yNvyaBRDcNpg3+Sujl6jYek c7ZOrDxdD8aw4oCJ2+2Ag7oarLbMr6lfoLd8hfITLVoLjm0mJ28vypxI8Bpa1A7s6G6x Pd8fXvzDKvpNb7O3GTsfFFRp6UNXh60Ff2isXFOrG4Lj0KImi0Er549v27LLsQU2uFPu y2/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=gqsigX2gmjuSDeOLlvZlBixwGucPHt7trIKLKPliU38=; b=kTHzAdDNBQqr8LvJ1Tkr69NsNRni9rmDmtU+UXbfVuWQy/ekH6z+zZeJvOkWG7Xg4v CdT/cTS1t/qdcfb8Z0g8Ssy1eO35hG0IdfccZ+tEDrmBqwnQG3r+SRiN7gUCRo46k+FH bjGb/XbSieUYz89WlkCBkHjHtJzK3ooNg334gQn7P6LqEI8WOvhtIi+bQQpHhdqvS809 pdPdhH/XlHQ6afvEQbGTyIbJQthwjIaRch+TsHS8bu9uSbyPzSjRxawCvC0Nj8lwWgXC W4tyXkXBhciXOf0XQ0julJBYvJlqfvDtdPMtrC46So2kdU8SXoft4urvV3XW0IhS1FPH gulQ== X-Gm-Message-State: ALoCoQn8zF1xcjNKwSpB96jWiFk6PUDOy6UetKw/KJdfSorODkzARzCU7eBBjMj1tHku6BM+R5xo MIME-Version: 1.0 X-Received: by 10.55.21.65 with SMTP id f62mr71287218qkh.46.1448862550352; Sun, 29 Nov 2015 21:49:10 -0800 (PST) Sender: wlosh@bsdimp.com Received: by 10.140.27.181 with HTTP; Sun, 29 Nov 2015 21:49:10 -0800 (PST) X-Originating-IP: [50.253.99.174] In-Reply-To: <0A51B6D4-9EDD-4EFF-876F-C6B515DBB4F3@kientzle.com> References: <0A51B6D4-9EDD-4EFF-876F-C6B515DBB4F3@kientzle.com> Date: Sun, 29 Nov 2015 22:49:10 -0700 X-Google-Sender-Auth: z1V4o4IzxRtXMOMRukNS-TlZ6UU Message-ID: Subject: Re: mtree "language" enhancements From: Warner Losh To: Tim Kientzle Cc: "Simon J. Gerraty" , Michal Ratajsky , Brooks Davis , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2015 05:49:12 -0000 On Sun, Nov 29, 2015 at 9:28 PM, Tim Kientzle wrote: > > > On Nov 29, 2015, at 2:49 PM, Tim Kientzle wrote: > > > > Simon also asked: > >> Indeed I'd really like the ability to provide default uid/gid > >> for the case that a uname/gname cannot be looked up. > > > > I think 'tar' got this right: If uname and uid are both specified, then > look up uname and if that fails, use the specified uid. Ditto for > gname/gid. In particular, this lets a single specification be used to > rebuild a tree on another system with different UIDs or on a system that > does not (yet) have a full password file. An option could be provided for > the (rare) case that someone really wants to prefer UIDs to unames. > > On further reflection, preferring UIDs to unames would actually be pretty > common here. > > In particular, NanoBSD (and Crochet and other similar tools) should prefer > the UID when building images instead of looking up unames against the build > host's password file. I've implemented what we've talked about, except this. When doing the makefs, we should use the /etc/master_password that's inside the image in preference to either of these alternatives. That's the most correct thing to do: use as much of the data as you can, as late as you can. The thing I'm struggling with now is why would both be present? Would that indicate an error? Or someone changing the defaults? And if they are changing the defaults, why use a uid in preference to a uname? Is this to avoid contamination? To set something not in the password file, or just comfort level of the user? FreeBSD will write unames for install*. So I'm left thinking that maybe the rule should be 'last one wins' at least for the use case where we use the target's /etc/master_password. That's what I've actually implemented. Preliminary testing of http://people.freebsd.org/~imp/mtree-dedup.awk appears to be working. I haven't tried all the cases yet, but it is looking promising. I don't need append_from, so that's just a stub in this file. Since this is in awk, I don't use the host's /etc/password at all. That's one of the failures of mtree that I've seen when I tried to use it, and perhaps the source of your concern. I'd love to see any libmtree be able to manipulate mtree files absent the tree it describes and even any process of uname -> uid at all to avoid these issues. The silly awk thing I wrote is purely a path to set of key-value pair manipulation tool. Once I'm more confident about this after some testing and integration into NanoBSD, I'll post something to phabricator. But I'd welcome any comments on what I've implemented in the mean time. Warner