From owner-freebsd-hackers@FreeBSD.ORG Mon Jul 8 18:43:54 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B36F5A8F; Mon, 8 Jul 2013 18:43:54 +0000 (UTC) (envelope-from freebsd-list@nuos.org) Received: from cargobay.net (cargobay.net [174.136.100.98]) by mx1.freebsd.org (Postfix) with ESMTP id 8FC9011E3; Mon, 8 Jul 2013 18:43:54 +0000 (UTC) Received: from leonidas.ccsys.com (unknown [65.35.151.3]) by cargobay.net (Postfix) with ESMTPSA id DC7F7F42; Mon, 8 Jul 2013 18:42:49 +0000 (UTC) Message-ID: <51DB085A.9040701@nuos.org> Date: Mon, 08 Jul 2013 18:43:38 +0000 From: "Chad J. Milios" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130624 Thunderbird/17.0.6 MIME-Version: 1.0 To: Devin Teske Subject: Re: Announcing: nuOS 0.0.9.1b1 - a whole NEW FreeBSD distro, NOT a fork References: <51D9E499.103@nuos.org> <13CA24D6AB415D428143D44749F57D7201FB6177@ltcfiswmsgmb21> In-Reply-To: <13CA24D6AB415D428143D44749F57D7201FB6177@ltcfiswmsgmb21> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , "Teske, Devin" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2013 18:43:54 -0000 On 07/08/13 00:12, Teske, Devin wrote: > On Jul 7, 2013, at 2:58 PM, Chad J. Milios wrote: > [snip] > >> /etc is now a ZFS dataset of its own >> How did we do it? >> Decades of conventional wisdom says /etc must be on /. >> Check it out, discuss the whys and the trade-offs. > Well, I see in nu_install on GitHub how you're doing it: > > You added: > > init_script="/boot/init.sh" > > to /boot/loader.conf, wich -- among other things -- does these two interesting things (variable names changed to make things more clear): > > zfs rollback -r $zfs/swap/host@blank > NOTE: $zfs is equal to $( /bin/kenv vfs.root.mountfrom ) minus the leading "zfs:" > > and > > zfs mount $zpool/etc > NOTE: $zpool is equal to $zfs from above, leading up to (but not including) the first slash (/). > > Cute. Have to say I wasn't aware of the init_script feature of loader.conf(5). Not bad. We also had to put one file into the etc directory on the / "beneath" the /etc mount so that /sbin/init can read it before /etc is mounted. There were two or three ways we could do that and each has a tradeoff. What we did (mv /etc/login.conf.db /boot/etc; ln -s ../boot/etc/login.conf.db /etc/login.conf.db) has the undesirable effect that one must remember to (or be reminded/automated) run cap_mkdb anytime /etc is rolled to a different snapshot or a backup is restored to it (if that changes login.conf). With our customers at ccsys.com we have a proprietary management thing in userland (and you could lose out on that important event hook if you used anything other than our interface for zfs rollbacks and restoring backups, which we forbid). Since our goals at nuos.org are different, i'd like to implement that trigger somewhere better, ideally as-needed and immediate as possible. if anyone with more intimate knowledge of how and exactly when login.conf.db gets accessed has any thoughts... It could be a disaster for an admin to think their /etc is in a certain state and have that one file be out of sync. If better minds could chip in, I'm wondering if we're better off editing /sbin/init to run init_script _before_ loading the daemon class from login.conf.db (or explain why thats a bad idea) or if i should just add some sort of hook to run cap_mkdb right when needed using a DTrace script or auditd? Does anyone think this issue is moot? (Can't we just document this particular specific "gotcha" instance? I don't think so, I abhor any "gotcha" that deviates from behavior people expect from "upstream" fbsd.) Does anyone agree it's important we come as close to perfect a solution as we can? Is a separate /etc even worth it to people? Should i scrap that feature because of this issue? I think we can tighten this up so theres no twisted ankles and no one falling in this rare case but certainly potential manhole. (the manhole i'm talking about is login.conf and login.conf.db being out of sync because the later is a symlink to /boot/etc and someone might rollback to a more restrictive login.conf and think they're covered without running cap_mkdb again but their login.conf.db is actually out of sync and less restrictive in a way that burns them) Devin, thank you IMMENSELY for bsdinstall and especially bsdconfig. I use them both at work and they make life so much better. And thank you for the simplification using kenv. I was unaware of it