From owner-freebsd-stable@FreeBSD.ORG Sat Nov 15 11:03:35 2008 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B9781065672 for ; Sat, 15 Nov 2008 11:03:35 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id D16908FC1D for ; Sat, 15 Nov 2008 11:03:34 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1L1IHT-000PML-KG; Sat, 15 Nov 2008 12:21:31 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Luigi Rizzo In-reply-to: <20081114194534.GA64986@onelab2.iet.unipi.it> References: <20081114194534.GA64986@onelab2.iet.unipi.it> Comments: In-reply-to Luigi Rizzo message dated "Fri, 14 Nov 2008 20:45:34 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 15 Nov 2008 12:21:31 +0200 From: Danny Braniss Message-ID: Cc: stable@freebsd.org Subject: Re: diskless+pxe notes X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Nov 2008 11:03:35 -0000 > Hi, > i finally decided to try and use pxeboot to replace the etherboot > method I was using so far for diskless setups. > > The goal is to fully share the server's root and /usr directories, > as documented in diskless(8). I'd like to share the following > notes, hopefully to go in the manpage. > > cheers > luigi > Hi, With a slightly modified libstand/bootp.c - a PR was sent way back, but you can check ftp://ftp.cs.huji.ac.il/users/danny/freebsd/diskless-boot you can control the diskless boot options. by comenting out kernel= in /boot/defaults/loader.conf you can set in the dhcpd.conf. since most of the tags received via dhcp are placed in kenv, the crucial options are there! BTW, we use diskless servers/workstations for 90% of our hosts, the exception being: - the dhcp/tftp server - a 'lagged' server - the router/server get confused :-) - our mail servers, there is a bug somewhere, where some critical network resources get deadlocked. - our developement servers. the / of the diskless is almost identical to the server, but for many reasons, I like to keep it appart. The trick to overcome the read-only problem, is using unionfs for /etc: in rc.initdiskless: if [ -e /conf/union ]; then kldload unionfs mount_md 4096 /.etc mount_unionfs -o transparent /.etc /etc fi the /conf is nfs mounted from a central site, the location is passed via dhcp: confpath=`kenv conf-path` if [ -n "$confpath" ] ; then if [ "`expr $confpath : '\(.*\):'`" ] ; then echo Mounting $confpath on /conf mount_nfs $confpath /conf chkerr $? "mount_nfs $confpath /conf" to_umount="${to_umount} $confpath" fi fi the actual rc.conf is configured like this: eval `kenv | sed -n 's/^rc\.//p'` rm -f /etc/rc.conf /etc/rc.conf.local for fc in $conf0 $conf1 $conf2 $conf3 $conf4 $conf5 $conf6 $conf7 $conf8 $conf9 rc.conf.$hostname do ho=`expr $fc : '\(.*\):'` fl=`expr $fc : '.*/\(.*\)'` if [ "${ho}" != "" ]; then mp=`expr $fc : '\(.*\)/.*'` mount_nfs $mp /mnt > /dev/null 2>&1 if [ -f /mnt/$fl ]; then echo "# from $fc /mnt/$fl" >> /etc/rc.conf cat /mnt/$fl >> /etc/rc.conf fi umount /mnt > /dev/null 2>&1 elif [ -e /conf/$fc ] ; then echo "# from /conf/$fc" >> /etc/rc.conf cat /conf/$fc >> /etc/rc.conf fi done > -------------- root path configuration ----------------- > > There seems to be a well known problem in pxeloader, see > kern/106493 , where pxeloader defaults to using a root path of > /pxeroot when offered "/" . The patch suggested in > > http://www.freebsd.org/cgi/query-pr.cgi?pr=106493 > > is trivial and judging from it I believe this is addressing a > true bug and not a feature. Fortunately there is a workaround > (suggested in the PR) which is using "//" as a root path. > > ------------- sharing /boot with the server --------------- > > I believe it is quite useful to share the whole root > partition between the server and the diskless client. > This would require at a minimum some conditional code > in loader.conf (or loader.rc, etc) so that at least you > point to different kernels. > > A minimalistic approach can be adding this line to /boot/loader.conf > > bootfile="kernel\\${loaddev};kernel" > > The variable $loaddev contains the name of the load device, > which is "pxe0" in the case of pxeboot, and disk* in other > cases when loading from the local disk. > If you make sure that there is no 'kernel.disk*' on the > directory, and instead there is a kernel.pxe0 in the same > directory, then the diskless machines and the server will boot > from the proper file. > > Unfortunately i don't know how to implement a conditional > in /boot/loader.conf -- otherwise one could do much nicer things > such as differentiate which modules to load and so on. > > --------------- pxeloader bug in 7.x --------------------------- > Also worth mentioning is an annoying bug in pxeloader as compiled > on 7.x, see http://www.freebsd.org/cgi/query-pr.cgi?pr=118222 > i.e. the pxeloader in 7.x fails to proceed and prints a message > "can't figure out which disk we are booting from". > > The workaround is using a pxeloader from FreeBSD6 works. > I guess this is a compiler-related problem (given that 6.x uses gcc 3.4 > as a compiler, while 7.x uses gcc 4.2). > > ----------------------------------------------------------------- > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >