Date: Thu, 14 Oct 2010 13:45:25 -0700 From: Devin Teske <dteske@vicor.com> To: Christian Degen <bubulein@freenas.org> Cc: freebsd-jail@freebsd.org Subject: Re: jail_build(8) -- a dialog(1)-based utility for building jails from binary distributions Message-ID: <1287089125.15487.135.camel@localhost.localdomain> In-Reply-To: <AANLkTi=y7ZL%2Bxp81wxucYi0cwxSQL84wn-cjxXYMs8Hf@mail.gmail.com> References: <1287017429.26377.47.camel@localhost.localdomain> <AANLkTi=y7ZL%2Bxp81wxucYi0cwxSQL84wn-cjxXYMs8Hf@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 2010-10-14 at 14:16 +0200, Christian Degen wrote: > Hello Devin, > > you can take a look at http://www.chruetertee.ch/jailscript/ , this > script creates a jail from src Surely there may be _some_ similarities between your script and mine, though admittedly, I believe that the similaries end at our usage of dialog(1). My script builds jails from binary [pre-compiled] distributions, not from source. This difference alone means that our scripts will share very little in-common because the process of assembling a jail from source is completely different than assembling from binary. The reason that I wrote my script is because provisioning a jail from a binary distribution is several orders of magnitude _faster_ than building from source. When building a jail from source, one must: First prep: a. c[v]sup/portsnap the source code b. `buildworld' in /usr/src Then for each jail: c. `installworld' to jail root When building a jail from binary, one must: First prep: a. Download/copy the binary release from FTP or CD/DVD to /usr/repos Then for each jail: b. Unpack the binary release to the jail root The high-level difference doesn't look very substantial, but it _is_ because the `buildworld' step in the source-setup routine: 1. Takes hours (your mileage may vary -- based on hardware performance) 2. Takes up disk space (`/usr/obj' will occupy approx. 600MB depending what features you've enabled/disabled in your build) It's also worth noting... When you build from source, you need approx. 600MB of disk space for `/usr/src' (this is in addition to the approx. 600MB needed for `/usr/obj' -- the compiled source that you `installworld' from). Meanwhile, when you build a jail from the binary distribution, you only need approx. 300MB for the binary distribution release: [dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ ls ERRATA.HTM README.TXT cdrom.inf info proflibs ERRATA.TXT RELNOTES.HTM dict kernels src HARDWARE.HTM RELNOTES.TXT doc manpages HARDWARE.TXT base docbook.css packages README.HTM catpages games ports [dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ du -hs . 741M . [dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ du -hs packages 438M packages [dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ expr 741 - 438 303 Even less (241MB) if you consider that jails don't need kernels: [dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ du -hs kernels 62M kernels [dteske@ipm0 /repos/FreeBSD-8.1/8.1-RELEASE]$ expr 303 - 62 241 NOTE: My script will omit the kernels dist if it doesn't exist when building the jail. Last, but not least... $ time sr jail_build [...] Jail successfully created. real 0m40.593s user 0m8.383s sys 0m6.551s I wrote this script with speed in-mind, and I'd say that being able to provision a jail from a binary release in 40 seconds is pretty-damned efficient. > , perhaps you find some good ideas. It > is written with German dialogues, but code is code. You're right, code is code ^_^ And I'm pleasantly surprised that I haven't forgotten _all_ of my German ^_^ First, a few comments... ========================================= I've never fathomed putting a jail into an md(4) image. What advantages to using an md(4) based disk-image have you encountered? What disadvantages have you encountered? ========================================= Nice usage of truncate(1) -- I would have used dd(1) with /dev/zero But then again, I didn't know that truncate(1) went back so far (it's available on my FreeBSD-4.11 system ... amazing). Now, a few points where I noticed you could optimize your script... ========================================= I notice that you're script immediately prompts the user for two inputs... first the hostname of the new jail ("Name der Jail") and then immediately thereafter the domain name of the jail ("Domain Name der Jail"). Why not prompt the user for the fully-qualified host name (FQHN) and then split the result? For example: foo=fully.qualified.host.com name="${foo%%.*}" ## "fully" domain="${foo#*.}" ## "qualified.host.com" ========================================= I notice that you're sometimes trapping the return status of dialog(1), later comparing it against either 1 or 255, and exiting for both 1 and 255. In those cases where you exit for both 1 and 255, (in example:) case ${retval} in 1) exit;; 255) exit;; esac why not simply instead test for non-success (non-zero): # written in same if/then style preened from your source if [ $retval -ne 0 ] ; then exit fi Which (FYI) can be written shorter as: [ $retval -eq 0 ] || exit or [ $retval -ne 0 ] && exit ========================================= The following is not required: verz=`echo ${verz} | sed -e 's/\/\//\//g'` The path: /usr/bar//baz///abc Is understood to be the same as: /usr/bar/baz/abc Yes, the former looks ugly, but it is still usable. If you *really* want to clean up ugly paths, this is a better way: verz=`eval realpath ${verz}` NOTE: It may not be immediately obvious, but the "eval" portion is required to support tilde-expansion (this is because tilde-expansion is performed only-once _and before_ parameter-expansion; the `eval' causes re-evaluation after initial parameter-expansion, therefore allowing tilde-expansion to be performed on the expanded parameter). NOTE: The above does not account for paths containing spaces (though it should also be noted that the rest of your script also lacks support for paths with spaces, so this should be of little concern -- for example, the above echo to sed(1) for replacing '//' with '/', ... paths with multiple spaces will be munged due to lacking surrounding double- quotes... the echo will see a series of positional parameters after the parameter-expansion of ${verz} is performed, so if the path contains a series of whitespace, the whitespace will not be retained but rather replaced with a single space, or whatever the first character of $IFS happens to be). ========================================= Your script makes use of jexec(8) and jail(8). My script does not require any jail utilities to unpack the binary distribution to the desired jail root directory. The core dependencies of my script (jail_build(8)) are below: find(1) sort(1) awk(1) dialog(1) cat(1) rm(1) mkdir(1) tar(1) mtree(8) All versions of FreeBSD should be able to provide the above. And since jails first appeared in FreeBSD-4.0, that means that my script is still usable to people running FreeBSD-4.0 that want a better way to provision a jail. For example... [God Forbid] should you want to be able to provision a FreeBSD-1.0 jail within FreeBSD-4.8, -6.2, -7.1, -8.1, -whatever... my script will help make it happen. ========================================= The below can be optimized... if [ "${filedisk}" = "true" ] ; then Because `true' and `false' are both valid built-ins to the shell, you can do this (your style retained): if $filedisk ; then Parameter expansion is performed and then the command is executed, so what is executed is really: if true ; then Which is a valid statement. It's also more efficient, because rather than having the shell: 1. perform parameter expansion of filedisk, then... 2. `[' is executed with 3 arguments: "$true", "=", and "true" 3. `[' reads arguments 4. `[' performs string comparison between "true" and "true" 5. `[' returns zero (for success) 6. return status is evaluated for success 7. if-block is fired on success The following is performed instead: 1. parameter expansion of filedisk is performed 2. `true' is executed with zero arguments 3. `true' returns zero (for success) 4. return status is evaluated for success 5. if-block is fired on success ========================================= Last,... It should be noted that your script goes above and beyond what my script does. My script (aptly named "jail_build") only _builds_ the jail for you. It does not (and will never) be used for configuring a jail (I view the process of building a jail and setting up a jail to run as two separate routines that should be handled by two different programs -- if at all). This is further compounded by the fact that jails in FreeBSD-4.0 to pre-8.0 can be considered "jails 1.0", while FreeBSD-8.x can be considered to be "jails 2.0" and HEAD is working on "jails 3.0" (introduction of VIMAGE, etc.). Provisioning a jail has not changed in all that time, but meanwhile the way you go about setting up a jail has changed for each iteration. In "jails 1.0", you had to: a. manage your own aliases manually in rc.conf(5) b. Look at /proc/PID/status to determine which processes where running in association with a given jail... In example: $ uname -spr FreeBSD 4.11-STABLE i386 $ grep jail /usr/local/etc/rc.d/zzjail.sh jail /usr/vm/jail1 jail1.localdomain 192.168.1.127 /bin/sh /etc/rc jail /usr/vm/jail2 jail2.localdomain 192.168.1.128 /bin/sh /etc/rc jail /usr/vm/jail3 jail3.localdomain 192.168.1.129 /bin/sh /etc/rc $ /bin/sh -c 'grep -Ilrs jail2.localdomain /proc 2> /dev/null' /proc/503/status /proc/364/status /proc/360/status /proc/358/status /proc/355/status /proc/353/status /proc/346/status $ cat /proc/360/status sshd 360 1 360 360 -1,-1 sldr 1266020515,536071 1,28410 0,0 select 0 0 0,0 jail2.localdomain c. Roll your own killall like-so: /bin/sh -c 'kill -9 $(grep -Ilrs jailhostname /proc 2> /dev/null | awk -F/ "{print \$3}")' d. Manually deconfigure aliases via ifconfig(8) when bringing the jail down. Meanwhile, in "jails 2.0"... a. rc.conf(5) now has a bevy of setup options for jails b. aliases are automatically de/configured by /etc/rc.d/jail c. ps(1) now has a `jid' keyword (e.g.): ps axo pid,jid,command d. killall(1) now has a `-j JID' option e. There is now the jls(8) utility for listing running jails f. There is now the jexec(8) utility for executing commands within a running jail g. Many sysctl MIBs have been added h. Certain filesystems can be marked as ``jail-friendly'' (currently only nullfs [through simple patch] and zfs) i. jails can now be configured with multiple IP addresses j. jails can run within jails And of course... "jails 3.0" is in the works and some of the focal points could perhaps be: a. "Productionized" VIMAGE support b. ``jail-friendly'' NFS c. ``jail-friendly'' umount(8) c. additional utilities ... Because the landscape for managing, configuring, and maintaining jails has changed dramatically from each iterative revision of jails, I think it would be best if the process of building a jail were kept seperate from the process of configuring a jail. That is to say... That a script whose purpose is to build a jail can act universally for all versions of FreeBSD. Meanwhile, any script whose purpose is to configure a jail must perform differently depending on the faculties of the base host (since the only thing that FreeBSD-4.x shares in-common with FreeBSD-8.x and eventually 9.x is the jail(8) utility and nothing else). Of course... you could just call me an "over pedantic nut who for some unknown reason cares about legacy operating systems"... but then I'd remind you that FreeBSD-4.x was not officially EOL'd until about 3 years ago (keep in-mind that FreeBSD develops multiple releases with separate branch-tags simultaneoulsy and usually has three major releases in active-support at any given time: right now it's highest-7.x, highest-8.x, and uncut-release 9/HEAD, but it wasn't but a few years ago that it was 4.11, highest-5.x and highest-6.x ... pre-7.0). > Christian > > > > On Thu, Oct 14, 2010 at 2:50 AM, Devin Teske <dteske@vicor.com> wrote: > > > I'd like to share a script that I wrote for provisioning jails NOT from > > the standard `buildworld'/`installworld' mechanism, but rather from > > binary distributions. For example, those made available here: -- Cheers, Devin Teske -> CONTACT INFORMATION <- Business Solutions Consultant II FIS - fisglobal.com 510-735-5650 Mobile 510-621-2038 Office 510-621-2020 Office Fax 909-477-4578 Home/Fax devin.teske@fisglobal.com -> LEGAL DISCLAIMER <- This message contains confidential and proprietary information of the sender, and is intended only for the person(s) to whom it is addressed. Any use, distribution, copying or disclosure by any other person is strictly prohibited. If you have received this message in error, please notify the e-mail sender immediately, and delete the original message without making a copy. -> FUN STUFF <- -----BEGIN GEEK CODE BLOCK----- Version 3.1 GAT/CS d(+) s: a- C++(++++) UB++++$ P++(++++) L++(++++) !E--- W++ N? o? K- w O M+ V- PS+ PE Y+ PGP- t(+) 5? X+(++) R>++ tv(+) b+(++) DI+(++) D(+) G+>++ e>+ h r>++ y+ ------END GEEK CODE BLOCK------ http://www.geekcode.com/ -> END TRANSMISSION <-
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1287089125.15487.135.camel>