From owner-freebsd-doc@FreeBSD.ORG Sat Feb 14 04:50:22 2004 Return-Path: Delivered-To: freebsd-doc@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2FE1B16A4D0 for ; Sat, 14 Feb 2004 04:50:22 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 02D7E43D2F for ; Sat, 14 Feb 2004 04:50:15 -0800 (PST) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) i1ECoEbv067478 for ; Sat, 14 Feb 2004 04:50:14 -0800 (PST) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.10/8.12.10/Submit) id i1ECoELc067477; Sat, 14 Feb 2004 04:50:14 -0800 (PST) (envelope-from gnats) Resent-Date: Sat, 14 Feb 2004 04:50:14 -0800 (PST) Resent-Message-Id: <200402141250.i1ECoELc067477@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-doc@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Jean-Francois Dockes Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4567D16A4CE for ; Sat, 14 Feb 2004 04:40:14 -0800 (PST) Received: from relay-6v.club-internet.fr (relay-6v.club-internet.fr [194.158.96.111]) by mx1.FreeBSD.org (Postfix) with ESMTP id C6F2C43D1D for ; Sat, 14 Feb 2004 04:40:13 -0800 (PST) (envelope-from dockes@wanadoo.fr) Received: from hautmedoc.dockes.com (f08a-7-238.d1.club-internet.fr [212.194.174.238]) by relay-6v.club-internet.fr (Postfix) with ESMTP id 0ABD9256A8 for ; Sat, 14 Feb 2004 13:40:11 +0100 (CET) Received: from hautmedoc.dockes.com (localhost.dockes.com [127.0.0.1]) i1ECeATO034896 for ; Sat, 14 Feb 2004 13:40:10 +0100 (CET) (envelope-from dockes@hautmedoc.dockes.com) Received: (from dockes@localhost) by hautmedoc.dockes.com (8.12.9p2/8.12.9/Submit) id i1ECeAph034895; Sat, 14 Feb 2004 13:40:10 +0100 (CET) (envelope-from dockes) Message-Id: <200402141240.i1ECeAph034895@hautmedoc.dockes.com> Date: Sat, 14 Feb 2004 13:40:10 +0100 (CET) From: Jean-Francois Dockes To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Subject: docs/62834: Diskless operation handbook section update X-BeenThere: freebsd-doc@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Jean-Francois Dockes List-Id: Documentation project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Feb 2004 12:50:22 -0000 >Number: 62834 >Category: docs >Synopsis: Diskless operation handbook section update >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-doc >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Sat Feb 14 04:50:14 PST 2004 >Closed-Date: >Last-Modified: >Originator: Jean-Francois Dockes >Release: FreeBSD 4.9-RELEASE-p2 i386 >Organization: >Environment: System: FreeBSD hautmedoc 4.9-RELEASE-p2 FreeBSD 4.9-RELEASE-p2 #0: Tue Feb 10 07:47:58 CET 2004 dockes@hautmedoc:/usr/obj/usr/src/sys/GENEDOC i386 >Description: The diskless operation section needs to be updated, esp. because of 5.x changes >How-To-Repeat: >Fix: Index: chapter.sgml =================================================================== RCS file: /home/ncvs/doc/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml,v retrieving revision 1.271 diff -u -r1.271 chapter.sgml --- chapter.sgml 11 Feb 2004 19:28:25 -0000 1.271 +++ chapter.sgml 13 Feb 2004 08:54:01 -0000 @@ -2275,10 +2275,10 @@ diskless operation A FreeBSD machine can boot over the network and operate without a - local disk, using filesystems mounted from an NFS server. No system - modification is necessary, beyond standard configuration files. - Such a system is easy to set up because all the necessary elements - are readily available: + local disk, using filesystems mounted from an NFS server. No + system modification is necessary, beyond standard configuration + files. Such a system is relatively easy to set up because all + the necessary elements are readily available: There are at least two possible methods to load the kernel over @@ -2324,10 +2324,10 @@ There are many ways to set up diskless workstations. Many elements are involved, and most can be customized to suit local - taste. The following will describe the setup of a complete system, - emphasizing simplicity and compatibility with the - standard FreeBSD startup scripts. The system described has the - following characteristics: + taste. The following will describe variations on the setup of a + complete system, emphasizing simplicity and compatibility with + the standard FreeBSD startup scripts. The system described has + the following characteristics: @@ -2339,12 +2339,14 @@ configuration files overridden by ones specific to diskless operation or, possibly, to the workstation they belong to. The parts of the root which have to be - writable are overlaid with &man.mfs.8; filesystems. Any changes - will be lost when the system reboots. + writable are overlaid with &man.mfs.8; (FreeBSD 4.X) or + &man.md.4; (FreeBSD 5.X) filesystems. Any changes will be lost + when the system reboots. - The kernel is loaded by etherboot - , using DHCP (or BOOTP) and TFTP. + The kernel is transferred and loaded either with + etherboot, or PXE, + as some situations may mandate the use of either method. @@ -2353,6 +2355,111 @@ other hosts. + All the information in this section has been tested + using FreeBSD releases 4.9-RELEASE-p2 and 5.2-RELEASE. The text is + primarily structured for 4.X usage. Notes have been inserted where + appropriate to indicate 5.X changes. + + + + + Background information + + Setting up diskless workstations is both relatively + straightforward and prone to errors. These are sometimes + difficult to diagnose for a number of reasons. For example: + + + Compile time options may determine the possible values + or syntax of runtime parameters. + + + Error messages are often cryptic or totally + absent. + + + + In this context, having some knowledge of the background + mechanisms involved is very useful to solve the problems that + may arise. + + Several operations need to be performed for a successful + bootstrap: + + + + The machine needs to obtain initial + parameters such as its IP address, executable file name, + server name, root path. This is done using the DHCP or + BOOTP protocols. DHCP is a compatible extension of BOOTP, + and uses the same port numbers and basic packet + format. + It is possible to configure a system to use only BOOTP. + The &man.bootpd.8; server program is included in the + base FreeBSD system. + + However, DHCP has a number of advantages over BOOTP (nicer + configuration files, possibility of using PXE, plus many + others not directly related to diskless operation), and + we shall describe mainly a DHCP configuration, with + equivalent exemples using &man.bootpd.8; when possible. + The sample configuration will use the ISC DHCP software + package (release 3.0.1.r11 was installed on the test + server: ). + + + + The machine needs to transfer one or several + programs to local memory. Either TFTP or NFS are used. The + choice between TFTP and NFS is a compile time option in + several places. A common source of error is to specify file + names for the wrong protocol: TFTP typically transfers all + files from a single directory on the server, and would + expect simple file names relative to this directory. NFS + needs absolute file paths. + + + + The possible intermediate bootstrap programs and the + kernel need to be initialized and executed. There are + several important variations in this area: + + + When using PXE, the machine will load &man.pxeboot.8;, + which is a modified version of the FreeBSD third stage + loader. The loader will obtain most parameters necessary + to system startup, and leave them in the kernel + environment before transferring control. It is possible to + use a GENERIC kernel in this case. + + + + etherboot, will directly + load the kernel, with less preparation. A kernel with + specific options will need to be built. + + + + PXE and etherboot work + equally well with 4.X systems. Because 5.X kernels + normally let the loader do more work for them, PXE is + preferred for 5.X systems. + If your BIOS and network + cards support PXE, you should probably use it. However, + it is still possible to start a 5.2 system with + etherboot + + + + + Finally, the machine needs to access its filesystems. NFS + is used in all cases. + + + + See also &man.diskless.8; + + Setup Instructions @@ -2364,31 +2471,7 @@ booting - There are two protocols that are commonly used to boot a - workstation that retrieves its configuration over the network: BOOTP - and DHCP. They are used at several points in the workstation - bootstrap: - - etherboot uses - DHCP (by default) or BOOTP (needs a configuration option) to - find the kernel. (PXE uses DHCP). - - The kernel uses BOOTP to locate the NFS - root. - - - - It is possible to configure a system to use only BOOTP. - The &man.bootpd.8; server program is included in the - base FreeBSD system. - - However, DHCP has a number of advantages over BOOTP (nicer - configuration files, possibility of using PXE, plus many others - not directly related to diskless operation), and we shall describe - both a pure BOOTP, and a BOOTP+DHCP configuration, with an - emphasis on the latter, which will use the ISC DHCP software - package. - + Configuration Using ISC DHCP @@ -2399,7 +2482,7 @@ The isc-dhcp server can answer both BOOTP and DHCP requests. - As of release 4.4, isc-dhcp + As of release 4.9, isc-dhcp 3.0 is not part of the base system. You will first need to install the net/isc-dhcp3-server port or the @@ -2409,7 +2492,9 @@ Once isc-dhcp is installed, it needs a configuration file to run, (normally named /usr/local/etc/dhcpd.conf). Here follows - a commented example: + a commented example, where host margaux + uses etherboot and host + corbieres uses PXE. default-lease-time 600; @@ -2429,9 +2514,16 @@ hardware ethernet 01:23:45:67:89:ab; fixed-address margaux.example.com; next-server 192.168.4.4; - filename "/tftpboot/kernel.diskless"; + filename "/data/misc/kernel.diskless"; option root-path "192.168.4.4:/data/misc/diskless"; } + host corbieres { + hardware ethernet 00:02:b3:27:62:df; + fixed-address corbieres.example.com; + filename "pxeboot"; + next-server 192.168.4.4; + option root-path "192.168.4.4:/data/misc/diskless"; + } } @@ -2446,21 +2538,25 @@ The - next-server directive designates - the TFTP server (the default is to use the same host as the - DHCP server). + next-server directive designates the TFTP + or NFS server to use for loading loader or kernel file (the + default is to use the same host as the DHCP server). - The - filename directive defines the file that - etherboot will load as a - kernel. - PXE appears to prefer a relative file - name, and it loads pxeboot, not the - kernel (option filename - "pxeboot"). - - + The + filename directive defines the file that + etherboot or PXE will load for the + next execution step. It must be specified according to the + transfer method used. etherboot can + be compiled to use NFS or TFTP. The FreeBSD port + configures NFS by default. + + + PXE uses TFTP, + which is why a relative file name is used here (this may + depend on the TFTP + server configuration, but would be fairly typical). Also, PXE + loads pxeboot, not the kernel. The @@ -2478,7 +2574,7 @@ Here follows an equivalent bootpd - configuration. This would be found in + configuration (reduced to one client). This would be found in /etc/bootptab. Please note that etherboot @@ -2529,6 +2625,10 @@ of everything. Else refer to for information about ports and packages. + You can change the etherboot + configuration (ie: to use TFTP instead of NFS) by editing the + Config file in the etherboot source directory. + For our setup, we shall use a boot floppy. For other methods (PROM, or dos program), please refer to the etherboot documentation. @@ -2550,6 +2650,22 @@ + + Booting with PXE + + By default, the &man.pxeboot.8; loader loads the kernel via NFS. + It can be compiled to use TFTP instead by specifying the + LOADER_TFTP_SUPPORT option in + /etc/make.conf. See the comments in + /etc/defaults/make.conf for + instructions. + + To use PXE when the machine starts, you will usually + need to select the Boot from network option + in your BIOS setup, or type a function key during the PC + initialization. + + Configuring the TFTP and NFS Servers @@ -2563,8 +2679,10 @@ diskless operation - You need to enable tftpd on the TFTP - server: + If any of the boostrap programs (PXE, pxeboot, or etherboot) + are configured to use TFTP, you need to enable + tftpd on the file server: + Create a directory from which tftpd @@ -2575,7 +2693,7 @@ Add this line to your /etc/inetd.conf: - tftp dgram udp wait root /usr/libexec/tftpd tftpd -s /tftpboot + tftp dgram udp wait root /usr/libexec/tftpd tftpd -s /tftpboot It appears that at least some PXE versions want the TCP version of TFTP. In this case, add a second line, @@ -2595,7 +2713,7 @@ location is set in both inetd.conf and dhcpd.conf. - You also need to enable NFS and export the + In all cases, you also need to enable NFS and export the appropriate filesystem on the NFS server. @@ -2611,7 +2729,7 @@ point and replace margaux with the name of the diskless workstation): - /data/misc -alldirs -ro margaux + /data/misc -alldirs -ro margaux corbieres Tell mountd to reread its configuration @@ -2632,23 +2750,48 @@ kernel configuration - Create a kernel configuration file for the diskless client - with the following options (in addition to the usual - ones): + If using etherboot, you need to + create a kernel configuration file for the diskless client + with the following options (in addition to the usual ones): - options BOOTP # Use BOOTP to obtain IP address/hostname - options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info - options BOOTP_COMPAT # Workaround for broken bootp daemons. + options BOOTP # Use BOOTP to obtain IP address/hostname + options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info - You may also want to use BOOTP_NFSV3 and - BOOTP_WIRED_TO (refer to LINT). - + You may also want to use BOOTP_NFSV3, + BOOT_COMPAT and + BOOTP_WIRED_TO (refer to + LINT). + + These option names are historical and slightly + misleading as they actually enable indifferent use of DHCP or BOOTP + inside the kernel (it is also possible to force strict BOOTP or + DHCP use). + Build the kernel (See ), - and copy it to the tftp directory, under the name listed - in dhcpd.conf. + and copy it to the place specified in + dhcpd.conf. + When using PXE, building a kernel with the above + options is not strictly necessary. Enabling them will + cause more DHCP requests to be issued during kernel + startup, with a small risk of inconsistency between the + new values and those retrieved by pxeboot in some special + cases. The advantage of using them is that the host name + will be set as a side effect. Else you will need to set + the host name by another method, for example in a + client-specific rc.conf file. + + + In order to be loadable with + etherboot, a 5.x kernel needs + to have the device hints compiled in. You would typically + set the following option in the configuration file (see + the NOTES configuration comments file). + hints "GENERIC.hints" + + @@ -2671,6 +2814,13 @@ the place where the filesystem will be created (the DEST variable). + The clone_root script distributed + with FreeBSD 5.2 is out of date relative to the 4.x version, and + also needs a few specific 5.x adjustments. Please refer to PR + + misc/62417. + + Refer to the comments at the top of the script for instructions. They explain how the base filesystem is built, and how files may be selectively overridden by versions specific @@ -2689,35 +2839,64 @@ confusing. Use them for reference only, except if you prefer the method that they describe, in which case you will need customized rc scripts. + + When using PXE, it may happen that the loader will + get a wrong idea of the root filesystem to be used by the + kernel (this will be the case if the cloned root + /etc/fstab is a copy of the server's, + as the client-specific version will not be visible by the + loader, which looks for hints there). + This will cause + error messages at the end of the kernel initialization, + when the kernel tries to mount the wrong root and fails, + or even actual errors if it succeeds because a local disk + happens to be present. + + It is possible to prevent this problem from happening + by inserting the following line in + /boot/loader.conf: + + vfs.root.mountfrom="" + + + + Configuring Swap If needed, a swap file located on the server can be - accessed via NFS. The exact bootptab - or dhcpd.conf options are not clearly - documented at this time. The following configuration - suggestions have been reported to work in some installations - using isc-dhcp 3.0rc11. + accessed via NFS. One of the methods commonly used to do this + has been discontinued in release 5.X. + + + NFS swap with FreeBSD 4.x + + The swap file location and size can be specified with + BOOTP/DHCP FreeBSD-specific options 128 and 129. Examples of + configuration files for isc-dhcp 3 or bootpd follow. + Add the following lines to dhcpd.conf: - # Global section - option swap-path code 128 = string; - option swap-size code 129 = integer 32; - - host margaux { - ... # Standard lines, see above - option swap-path "192.168.4.4:/netswapvolume/netswap"; - option swap-size 64000; - } + # Global section + option swap-path code 128 = string; + option swap-size code 129 = integer 32; + + host margaux { + ... # Standard lines, see above + option swap-path "192.168.4.4:/netswapvolume/netswap"; + option swap-size 64000; + } - The idea is that, at least for a FreeBSD client, - DHCP/BOOTP option code 128 is the path to the NFS swap file, - and option code 129 is the swap size in kilobytes. Older - versions of dhcpd allowed a syntax of + + swap-path is the path to a directory + where swap files will be located. Each file will be named + swap.clientIp. + + Older versions of dhcpd used a syntax of option option-128 "..., which is no longer supported. /etc/bootptab would use the @@ -2733,12 +2912,12 @@ On the NFS swap file server, create the swap file(s) - - &prompt.root; mkdir /netswapvolume/netswap - &prompt.root; cd /netswapvolume/netswap - &prompt.root; dd if=/dev/zero bs=1024 count=64000 of=swap.192.168.4.6 - &prompt.root; chmod 0600 swap.192.168.4.6 - + + &prompt.root; mkdir /netswapvolume/netswap + &prompt.root; cd /netswapvolume/netswap + &prompt.root; dd if=/dev/zero bs=1024 count=64000 of=swap.192.168.4.6 + &prompt.root; chmod 0600 swap.192.168.4.6 + 192.168.4.6 is the IP address for the diskless client. @@ -2747,13 +2926,31 @@ On the NFS swap file server, add the following line to /etc/exports: - /netswapvolume -maproot=0:10 -alldirs margaux +/netswapvolume -maproot=0:10 -alldirs margaux Then tell mountd to reread the exports file, as above. + + + + NFS swap with FreeBSD 5.x + + The kernel does not support enabling NFS swap at boot + time. Swap must be enabled by the startup scripts, by + mounting a writeable file system and creating and enabling a + swap file. This can be done, for example, in + /etc/rc.local. Example: + + + swapfile=/writableVolume/swap/swap.MyIP + dd if=/dev/zero of=$swapfile bs=1k count=1 oseek=100000 + swapon $swapfile + rm -rf $swapfile + + >Release-Note: >Audit-Trail: >Unformatted: