Date: Thu, 10 Oct 2013 17:21:30 +0000 From: "Teske, Devin" <Devin.Teske@fisglobal.com> To: Allan Jude <freebsd@allanjude.com> Cc: "<freebsd-current@freebsd.org>" <freebsd-current@freebsd.org>, "Teske, Devin" <Devin.Teske@fisglobal.com> Subject: Re: [CFT] Patch to bsdinstall to support root-on-ZFS and GELI Message-ID: <13CA24D6AB415D428143D44749F57D720FC51061@LTCFISWMSGMB21.FNFIS.com> In-Reply-To: <5256E163.80100@allanjude.com> References: <52531295.7090700@allanjude.com> <52546844.2010608@freebsd.org> <52549191.5010400@allanjude.com> <5254F582.1040406@freebsd.org> <13CA24D6AB415D428143D44749F57D720FC4B1F4@LTCFISWMSGMB21.FNFIS.com> <5256509E.3000604@freebsd.org> <52565555.8030906@allanjude.com> <525695BD.7060706@freebsd.org> <13CA24D6AB415D428143D44749F57D720FC508BE@LTCFISWMSGMB21.FNFIS.com> <5256E163.80100@allanjude.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Oct 10, 2013, at 10:18 AM, Allan Jude wrote: > On 2013-10-10 11:30, Teske, Devin wrote: >> On Oct 10, 2013, at 4:55 AM, Nathan Whitehorn wrote: >>=20 >>> On 10/10/13 09:20, Allan Jude wrote: >>>> On 2013-10-10 03:00, Nathan Whitehorn wrote: >>>>> On 10/09/13 18:55, Teske, Devin wrote: >>>>>> On Oct 8, 2013, at 11:19 PM, Nathan Whitehorn wrote: >>>>>>=20 >>>>>>> On 10/09/13 01:13, Allan Jude wrote: >>>>>>>> On 2013-10-08 16:17, Nathan Whitehorn wrote: >>>>>>>>> On 10/07/13 21:59, Allan Jude wrote: >>>>>>>>>> Devin Teske and I have been working on a big patch to bsdinstall= to >>>>>>>>>> implement installing on a ZFS pool. It supports both GPT and MBR= , the 4k >>>>>>>>>> sector gnop trick, and optional GELI encryption. We would like t= o commit >>>>>>>>>> this in time for 10.0-BETA1 so it needs some testing to work out= any >>>>>>>>>> obvious bugs before we send it off to re@ to get it committed. >>>>>>>>>>=20 >>>>>>>>>> It includes a single configuration menu that allows you to selec= t all of >>>>>>>>>> the required details, including which drives to use (gets detail= s from >>>>>>>>>> camcontrol, also includes an inspection utility that presents the >>>>>>>>>> detailed output of camcontrol inquiry/identify, and gpart show),= what >>>>>>>>>> ZFS RAID level to use (taking in to consideration the selected n= umber of >>>>>>>>>> drives), GPT/mbr, 4k YES/no, GELI yes/NO, pool name, etc. >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> Additional, it includes some other changes to bsdinstall: >>>>>>>>>> 1. Change the default to the 'non-standard keyboard mapping' pro= mpt to no >>>>>>>>>> 2. Replace the 3 separate dialogs to configure an ipv4 address w= ith just 1 >>>>>>>>>> 3. Remove the dialog asking if you wish to enable crash dumps, t= his >>>>>>>>>> feature has been combined into the regular 'services to enable' = dialog >>>>>>>>>> and enabled by default >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> You can browse the patches here: >>>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=3Dhttp://druidbsd.cvs= .sf.net/viewvc/druidbsd/bsdinstall_zfs/&k=3D%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3= D%0A&r=3DLTzUWWrRnz2iN3PtHDubWRSAh9itVJ%2BMUcNBCQ4tyeo%3D%0A&m=3DckXs2H5TEc= KdiN%2F9l%2FClInsMys9LmTwEbElzzrBchMU%3D%0A&s=3D6361ad5ccc04a6d8401158a3ee1= b3d8df9fcd2ef47ff6ad6365a57b907cceaec >>>>>>>>>>=20 >>>>>>>>>> I've built a bootonly.iso (10.0-ALPHA4) to make testing easier, >>>>>>>>>> available compressed (48 MB) or uncompressed (211 MB): >>>>>>>>>>=20 >>>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=3Dhttp://www.allanjud= e.com/bsd/zfsbootonly_2013-10-06.iso.xz&k=3D%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3= D%0A&r=3DLTzUWWrRnz2iN3PtHDubWRSAh9itVJ%2BMUcNBCQ4tyeo%3D%0A&m=3DckXs2H5TEc= KdiN%2F9l%2FClInsMys9LmTwEbElzzrBchMU%3D%0A&s=3D570daea25d9e629788c76c2b5a6= eae19d32050363986a1004a6434c3466965c2 >>>>>>>>>>=20 >>>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=3Dhttp://www.allanjud= e.com/bsd/zfsbootonly_2013-10-06.iso&k=3D%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0= A&r=3DLTzUWWrRnz2iN3PtHDubWRSAh9itVJ%2BMUcNBCQ4tyeo%3D%0A&m=3DckXs2H5TEcKdi= N%2F9l%2FClInsMys9LmTwEbElzzrBchMU%3D%0A&s=3Df52d604a2503a48b409f92f8376bb7= 73de4dbc469d1e74b16d051a20ad894820 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> We look forward to your feedback >>>>>>>>>>=20 >>>>>>>>> Thanks for doing this! I had a few comments: >>>>>>>>> 1. ZFS is not bootable on all architectures. Could you adjust tha= t menu >>>>>>>>> item to only display for i386, amd64, and (I think?) sparc64. Use= uname >>>>>>>>> -m, not -p, for this. >>>>>>>> I had not considered that, I'll make that change >>>>>>>>=20 >>>>>>>>> 1a. The script is broken on sparc64 in any case, which uses VTOC8 >>>>>>>>> instead of GPT. >>>>>>>> I'll disable sparc64 as well >>>>>>>>=20 >>>>>>>>> 2. Why are you using camcontrol? That is guaranteed not to work on >>>>>>>>> non-CAM systems. You should use the GEOM ident string if you need= an ID. >>>>>>>> The GEOM ident string doesn't do enough to help the user identify = which >>>>>>>> drive is which. >>>>>>>> More data is not exposed anywhere that I could find >>>>>>>>=20 >>>>>>>> What we really need, is dev.ada.0.desc% like we have for network >>>>>>>> interfaces and a slew of other devices. GEOM data is great, but it= is >>>>>>>> not exposed in a shell friendly way any place that I could find, o= ther >>>>>>>> than the sysctl with DOT and XML data. >>>>>>> This is one of the reasons the partition editor is written in C. Th= ere >>>>>>> are a few other odd corner-cases where C is much more powerful than= the >>>>>>> command-line for the GEOM operations that partedit needs to do. I'm= not >>>>>>> sure how to usefully get it just from the shell. You can see how to= do >>>>>>> it in C in the boot_disk() routine of partedit/auto_part.c. >>>>>>>=20 >>>>>>>>> 3. Any plans to integrate this into the regular partition editor?= ZFS >>>>>>>>> support is important enough that I will definitely not get in the= way, >>>>>>>>> even as a bolt-on, but it would be a shame for it to stay that wa= y. The >>>>>>>>> editor is also designed for ZFS to be added. >>>>>>>> I am a sysadmin, not a programmer. I can't write C. Most people >>>>>>>> deploying servers can't write C. I agree with Devin Teske, if ever= ything >>>>>>>> was in shell it would be a lot more usable for non-developers, who >>>>>>>> probably make up the majority of people who deploy FreeBSD. >>>>>>> There are some cases the other way too. Devin is probably the most >>>>>>> shell-proficient FreeBSD committer. >>>>>> Well, there's jilles ;D (he writes/maintains the sh(1) implementatio= n itself). >>>>>>=20 >>>>>>=20 >>>>>>> I certainly can't write shell >>>>>>> scripts at that level, which means that for me bsdconfig, for examp= le, >>>>>>> is effectively read-only (and quite hard to read as well). >>>>>> In the past few days of working on the bsdinstall_zfs patch-set with= Allan >>>>>> Jude, I learned something new actually. >>>>>>=20 >>>>>> It seems that sh(1) doesn't suffer so much from this "read only" con= cept. >>>>>>=20 >>>>>> Imagine you're starting a new C project on an operating system that = has >>>>>> all new syscalls and all new APIs that you've never seen. Would be p= retty >>>>>> hard, no doubt. Now imagine that you're thrown a life-line called PO= SIX >>>>>> and hey, now you're slinging code in MinGW on Windows when you don't >>>>>> know a lick of M$ syscalls or APIs. >>>>>>=20 >>>>>> In a similar manner, I've witnessed functionality be added that is t= ruly >>>>>> functional *without* using any of the API calls. Then I come in and = do >>>>>> a round of optimizations to leverage the existing API. >>>>>>=20 >>>>>> Shell is kinda like that... >>>>>>=20 >>>>>> I'm noticing Allan Jude doesn't know all the API calls yet (who coul= d? other >>>>>> than me of course -- and even I have trouble remembering them _all_)= yet >>>>>> this doesn't phase him (or others) from jumping in and lending a han= d. >>>>>>=20 >>>>>> No different than C, but the read-only aspect is lessened significan= tly I >>>>>> believe because there are so many people out there that know enough >>>>>> good shell syntax and are just a stone's throw away from *great* she= ll >>>>>> syntax (which I must admit jilles helped me cross that boundary). >>>>>>=20 >>>>>> I think in C, the read-only aspect is greater because its harder to = parcel-out >>>>>> the functions for a unit-test; harder to inject new code; and harder= to get >>>>>> to a functioning end-state. >>>>> Look, I have no doubt that in the right hands shell can do amazing >>>>> things and C can be badly written. That isn't the issue though. My >>>>> statement was purely that most FreeBSD developers (me included) are m= ore >>>>> comfortable with C than shell when used at the kind of level involved >>>>> here. This is not a value judgment but a statement of fact. Whether or >>>>> not, at some platonic level, shell or C or python or whatever are more >>>>> or less read-only is not the point here. The point is that I, and I >>>>> suspect many other developers, cannot write (or read) very advanced >>>>> levels of shell scripting but can read and write the equivalent progr= ams >>>>> when written in C in many cases. It's what the rest of the system is >>>>> written in and what we spend most of our time using. Whether this sho= uld >>>>> be the case or not is immaterial; the fact remains that shell scripti= ng >>>>> at any but a very basic level does introduce a very large barrier to >>>>> entry for probably a large majority of committers. >>>>>=20 >>>>> This is not always a problem -- especially if using something more >>>>> obscure allows very active development by the set of people working on >>>>> it -- but does reduce the set of people who can make modifications >>>>> substantially. I have no ability to change, or understand, most of >>>>> bsdconfig, for example. This isn't a problem since you are doing all = the >>>>> work and there is no reason I would need to or want to make changes to >>>>> it. But it could become a problem in a part of the system to which >>>>> multiple people needed to contribute. It's about other people's comfo= rt >>>>> zones and knowledge in the end. >>>>> -Nathan >>>>> _______________________________________________ >>>>> freebsd-current@freebsd.org mailing list >>>>> https://urldefense.proofpoint.com/v1/url?u=3Dhttp://lists.freebsd.org= /mailman/listinfo/freebsd-current&k=3D%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r= =3DLTzUWWrRnz2iN3PtHDubWRSAh9itVJ%2BMUcNBCQ4tyeo%3D%0A&m=3DckXs2H5TEcKdiN%2= F9l%2FClInsMys9LmTwEbElzzrBchMU%3D%0A&s=3D229c04e0b0ce43bcdc88819dc4a9cba8f= fa47ef43ec9ad5352019bd7c850feff >>>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd= .org" >>>> I definately see your point, but the code going in to the zfsboot part >>>> in particular, is not that advanced. I started this project two weeks >>>> ago with a well below average knowledge of shell script, a minor bit of >>>> experience in php, and no real programming experience to speak of. >>>>=20 >>>> I wrote most of the code that actually does things, Devin just cleaned >>>> it up (I didn't know anything about shstyle or the rules) and pointed = me >>>> at some handy subroutines he had written to avoid writing out 20 lines >>>> of code every time I wanted to display a dialog box >>>>=20 >>>> I wouldn't say any of what is in the proposed patches to bsdinstall is >>>> unmaintainable, and it definately makes it much more approachable for >>>> sysadmins who are the ones that actually need to be able to script >>>> bsdinstall, and generate various methods of automated deployment. This >>>> is a very important feature if we want FreeBSD to be taken seriously f= or >>>> deployment in mass virtualization environments (do not make my say >>>> cloud. we saw a great sign in Malta at some street festival "what is >>>> cloud?") >>> Sorry, I think you misunderstood me. I really appreciate your work, and >>> the ZFS stuff, as I said, is not that complex so there's no problem >>> there. Aside from a couple of issues I mentioned (with sparc, etc.), it >>> seems like a very good step forward. What I *was* trying to do was >>> discourage you from rewriting partedit as a shell script and to >>> encourage unifying the ZFS backend into partedit at some point in the >>> future significantly different from now. >>>=20 >>> I also wanted to mention that bsdinstall is completely scriptable >>> (although not the new ZFS code, I guess...) >> Oh, but it is. >>=20 >> The following variables have sensible defaults and are designed >> to be exported in a bsdinstall script to customize the actions >> performed by the zfsboot script: >>=20 >> dteske@scribe9.vicor.com scripts $ grep '^: ' zfsboot=20 >> : ${ZFSBOOT_POOL_NAME:=3Dzroot} >> : ${ZFSBOOT_BEROOT_NAME:=3Dbootenv} >> : ${ZFSBOOT_BOOTFS_NAME:=3Ddefault} >> : ${ZFSBOOT_VDEV_TYPE:=3Dstripe} >> : ${ZFSBOOT_GNOP_4K_FORCE_ALIGN:=3D1} >> : ${ZFSBOOT_GELI_ENCRYPTION:=3D} >> : ${ZFSBOOT_GELI_POOL_NAME:=3Dbootpool} >> : ${ZFSBOOT_GELI_BOOT_SIZE:=3D2g} >> : ${ZFSBOOT_GELI_KEY_file:///=3D/boot/encryption.key} >> : ${ZFSBOOT_DISKS:=3D} >> : ${ZFSBOOT_PARTITION_SCHEME:=3DGPT} >> : ${ZFSBOOT_SWAP_SIZE:=3D2g} >>=20 >> Notice the ZFSBOOT_ prefix matching the script name. >> These may not be the final names used as an overall >> effort is made (given time) to conflate the scripting variables >> and general namespace. >>=20 >>=20 >>=20 >>> and has been for some time. >> The type of scripting set forth by bsdinstall is not the same kind of >> scripting set forth by bsdconfig. >>=20 >> That being said, I've not yet made an effort to close the gap on >> bsdinstall scripting (to make it like bsdconfig scripting). >>=20 >> In bsdconfig, we can literally load a sysinstall "install.cfg" file and >> there is no need to for a complicated embedding system such as >> bsdinstall has (wherein the "script" is actually a multi-part file). >>=20 >> A lot of people have scoffed at backward compatibility because it >> would not be easy. However, I see a pretty big carrot dangling in >> front of the backward compatibility effort... >>=20 >> It's not the act of implementing the individual reswords that's going >> to bring the biggest value-add, but instead the bringing back of the >> process-flow to deployment that will see the greatest gain in >> functionality. By this, I mean that currently you invoke bsdinstall >> and it runs through a UX given a set of values set in variables. >> This is sub-optimal if you don't like the UX choices -- better to >> instead allow the script to take the reigns and do things by way >> of having the script drive (in sysinstall, this amounts to the fact that >> if "install.cfg" doesn't call diskPartitionWrite or doesn't call >> distExtractAll, then those steps are not performed). In bsdinstall, a >> change in focus to allow the script to drive *could* be done by >> requiring the user to fork-exec to a bunch of scripts -- and in-turn >> then requiring each/every custom setting to be *export*ed... but >> ultimately the best bang-for-your-buck is going to come from >> (I'll say it again) namespace conflation (writing the functionality >> as a series of includable functions that are run in the same >> namespace as the script that is driving). Naturally, a bunch of >> functions can be lost if you don't know where to look for them, and >> that comes back full-circle to `script.subr' and `variable.subr' as the >> "points of discovery" (besides a man-page; which is where sys- >> install documented its own reswords). >>=20 >>=20 >>=20 >>> Only the initial release that shipped with 9.0 was not. There's a >>> lengthy discussion of how this works in the man page. >> *nods* >>=20 >>=20 >>=20 >>> The scripting >>> support is extremely flexible (you can run an arbitrary shell script in >>> addition to the usual installation steps) and lets installations from >>> PXE or media operate completely unattended. >> Yes, but I've been taking to simply creating /etc/installerconf and >> completely bypassing the scripting that's documented in the man-page >> *because* I abhor multi-part file which requires a split-architecture >> approach to installation (no like). >>=20 >> Sysinstall often required me to do the same thing. Whereas in >> sysinstall the problem was that you couldn't speak native shell in the >> install.cfg file (requiring you to call out to a "system" resword to >> execute a script which generates a side "install.cfg" later brought >> back in with the "loadConfig" resword, bsdinstall instead requires me >> to have -- in the multi-part script it loads -- a preamble and a setup >> script and this causes a similar dichotomy in passing information >> between the stages). >>=20 >> In bsdconfig, I worked extremely hard to solve that issue. The end- >> result is that both problems are solved. The problem of sysinstall >> is gone because the "install.cfg" file becomes a shell script so you >> can then start speaking native shell in it. The multi-part problem is >> gone when you provide all the reswords in the native language >> (conflated namespace). > Devin, we should make a note to talk to Rick Miller (Verisign, host of > vBSDCon) while we are there. They still use sysinstall, and having > bsdinstall understand their sysinstall configs may be of great value to > them. That's the plan ;D Been working with ol' Rick for years now on that journey. We keep in- touch and I keep apprising him of how things are going. Also, I've been sharing with him recipes for /etc/installerconf that used the legacy reswords but branch out to the couple parts of bsdinstall that are needed to do the disk management and dist extraction. --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?13CA24D6AB415D428143D44749F57D720FC51061>