Date: Sat, 14 Feb 2004 13:40:10 +0100 (CET) From: Jean-Francois Dockes <dockes@wanadoo.fr> To: FreeBSD-gnats-submit@FreeBSD.org Subject: docs/62834: Diskless operation handbook section update Message-ID: <200402141240.i1ECeAph034895@hautmedoc.dockes.com> Resent-Message-ID: <200402141250.i1ECoELc067477@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 62834 >Category: docs >Synopsis: Diskless operation handbook section update >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-doc >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Sat Feb 14 04:50:14 PST 2004 >Closed-Date: >Last-Modified: >Originator: Jean-Francois Dockes >Release: FreeBSD 4.9-RELEASE-p2 i386 >Organization: >Environment: System: FreeBSD hautmedoc 4.9-RELEASE-p2 FreeBSD 4.9-RELEASE-p2 #0: Tue Feb 10 07:47:58 CET 2004 dockes@hautmedoc:/usr/obj/usr/src/sys/GENEDOC i386 >Description: The diskless operation section needs to be updated, esp. because of 5.x changes >How-To-Repeat: >Fix: Index: chapter.sgml =================================================================== RCS file: /home/ncvs/doc/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml,v retrieving revision 1.271 diff -u -r1.271 chapter.sgml --- chapter.sgml 11 Feb 2004 19:28:25 -0000 1.271 +++ chapter.sgml 13 Feb 2004 08:54:01 -0000 @@ -2275,10 +2275,10 @@ <indexterm><primary>diskless operation</primary></indexterm> <para>A FreeBSD machine can boot over the network and operate without a - local disk, using filesystems mounted from an NFS server. No system - modification is necessary, beyond standard configuration files. - Such a system is easy to set up because all the necessary elements - are readily available:</para> + local disk, using filesystems mounted from an NFS server. No + system modification is necessary, beyond standard configuration + files. Such a system is relatively easy to set up because all + the necessary elements are readily available:</para> <itemizedlist> <listitem> <para>There are at least two possible methods to load the kernel over @@ -2324,10 +2324,10 @@ <para>There are many ways to set up diskless workstations. Many elements are involved, and most can be customized to suit local - taste. The following will describe the setup of a complete system, - emphasizing simplicity and compatibility with the - standard FreeBSD startup scripts. The system described has the - following characteristics:</para> + taste. The following will describe variations on the setup of a + complete system, emphasizing simplicity and compatibility with + the standard FreeBSD startup scripts. The system described has + the following characteristics:</para> <itemizedlist> <listitem> @@ -2339,12 +2339,14 @@ configuration files overridden by ones specific to diskless operation or, possibly, to the workstation they belong to.</para> <para>The parts of the <filename>root</filename> which have to be - writable are overlaid with &man.mfs.8; filesystems. Any changes - will be lost when the system reboots.</para> + writable are overlaid with &man.mfs.8; (FreeBSD 4.X) or + &man.md.4; (FreeBSD 5.X) filesystems. Any changes will be lost + when the system reboots.</para> </listitem> <listitem> - <para>The kernel is loaded by <application>etherboot - </application>, using DHCP (or BOOTP) and TFTP.</para> + <para>The kernel is transferred and loaded either with + <application>etherboot</application>, or <emphasis>PXE</emphasis>, + as some situations may mandate the use of either method.</para> </listitem> </itemizedlist> @@ -2353,6 +2355,111 @@ other hosts.</para> </caution> + <para>All the information in this section has been tested + using FreeBSD releases 4.9-RELEASE-p2 and 5.2-RELEASE. The text is + primarily structured for 4.X usage. Notes have been inserted where + appropriate to indicate 5.X changes.</para> + + + + <sect2> + <title>Background information</title> + + <para>Setting up diskless workstations is both relatively + straightforward and prone to errors. These are sometimes + difficult to diagnose for a number of reasons. For example: + <itemizedlist> + <listitem> + <para>Compile time options may determine the possible values + or syntax of runtime parameters.</para> + </listitem> + <listitem> + <para>Error messages are often cryptic or totally + absent.</para> + </listitem> + </itemizedlist> + + <para>In this context, having some knowledge of the background + mechanisms involved is very useful to solve the problems that + may arise.</para> + + <para>Several operations need to be performed for a successful + bootstrap:</para> + + <itemizedlist> + <listitem> + <para>The machine needs to obtain initial + parameters such as its IP address, executable file name, + server name, root path. This is done using the DHCP or + BOOTP protocols. DHCP is a compatible extension of BOOTP, + and uses the same port numbers and basic packet + format.</para> + <para>It is possible to configure a system to use only BOOTP. + The &man.bootpd.8; server program is included in the + base FreeBSD system.</para> + + <para>However, DHCP has a number of advantages over BOOTP (nicer + configuration files, possibility of using PXE, plus many + others not directly related to diskless operation), and + we shall describe mainly a DHCP configuration, with + equivalent exemples using &man.bootpd.8; when possible. + The sample configuration will use the ISC DHCP software + package (release 3.0.1.r11 was installed on the test + server: ).</para> + </listitem> + + <listitem> + <para>The machine needs to transfer one or several + programs to local memory. Either TFTP or NFS are used. The + choice between TFTP and NFS is a compile time option in + several places. A common source of error is to specify file + names for the wrong protocol: TFTP typically transfers all + files from a single directory on the server, and would + expect simple file names relative to this directory. NFS + needs absolute file paths.</para> + </listitem> + + <listitem> + <para>The possible intermediate bootstrap programs and the + kernel need to be initialized and executed. There are + several important variations in this area:</para> + <itemizedlist> + <listitem> + <para>When using PXE, the machine will load &man.pxeboot.8;, + which is a modified version of the FreeBSD third stage + loader. The loader will obtain most parameters necessary + to system startup, and leave them in the kernel + environment before transferring control. It is possible to + use a GENERIC kernel in this case.</para> + </listitem> + + <listitem> + <para><application>etherboot</application>, will directly + load the kernel, with less preparation. A kernel with + specific options will need to be built.</para> + </listitem> + </itemizedlist> + + <para>PXE and <application>etherboot</application> work + equally well with 4.X systems. Because 5.X kernels + normally let the loader do more work for them, PXE is + preferred for 5.X systems.</para> + <para>If your BIOS and network + cards support PXE, you should probably use it. However, + it is still possible to start a 5.2 system with + <application>etherboot</application></para> + + </listitem> + + <listitem> + <para>Finally, the machine needs to access its filesystems. NFS + is used in all cases.</para> + </listitem> + </itemizedlist> + + <para>See also &man.diskless.8;</para> + + </sect2> <sect2> <title>Setup Instructions</title> @@ -2364,31 +2471,7 @@ <secondary>booting</secondary> </indexterm> - <para>There are two protocols that are commonly used to boot a - workstation that retrieves its configuration over the network: BOOTP - and DHCP. They are used at several points in the workstation - bootstrap:</para> - <itemizedlist> - <listitem><para><application>etherboot</application> uses - DHCP (by default) or BOOTP (needs a configuration option) to - find the kernel. (PXE uses DHCP).</para> - </listitem> - <listitem><para>The kernel uses BOOTP to locate the NFS - root.</para> - </listitem> - </itemizedlist> - - <para>It is possible to configure a system to use only BOOTP. - The &man.bootpd.8; server program is included in the - base FreeBSD system.</para> - - <para>However, DHCP has a number of advantages over BOOTP (nicer - configuration files, possibility of using PXE, plus many others - not directly related to diskless operation), and we shall describe - both a pure BOOTP, and a BOOTP+DHCP configuration, with an - emphasis on the latter, which will use the ISC DHCP software - package.</para> - + <sect4> <title>Configuration Using ISC DHCP</title> <indexterm> @@ -2399,7 +2482,7 @@ <para>The <application>isc-dhcp</application> server can answer both BOOTP and DHCP requests.</para> - <para>As of release 4.4, <application>isc-dhcp + <para>As of release 4.9, <application>isc-dhcp 3.0</application> is not part of the base system. You will first need to install the <filename role="package">net/isc-dhcp3-server</filename> port or the @@ -2409,7 +2492,9 @@ <para>Once <application>isc-dhcp</application> is installed, it needs a configuration file to run, (normally named <filename>/usr/local/etc/dhcpd.conf</filename>). Here follows - a commented example:</para> + a commented example, where host <literal>margaux</literal> + uses <application>etherboot</application> and host + <literal>corbieres</literal> uses PXE.</para> <programlisting> default-lease-time 600; @@ -2429,9 +2514,16 @@ hardware ethernet 01:23:45:67:89:ab; fixed-address margaux.example.com; next-server 192.168.4.4;<co id="co-dhcp-next-server"> - filename "/tftpboot/kernel.diskless";<co id="co-dhcp-filename"> + filename "/data/misc/kernel.diskless";<co id="co-dhcp-filename"> option root-path "192.168.4.4:/data/misc/diskless";<co id="co-dhcp-root-path"> } + host corbieres { + hardware ethernet 00:02:b3:27:62:df; + fixed-address corbieres.example.com; + filename "pxeboot";<co id="co-dhcp-filenamePxe"> + next-server 192.168.4.4; + option root-path "192.168.4.4:/data/misc/diskless"; + } } </programlisting> @@ -2446,21 +2538,25 @@ </callout> <callout arearefs="co-dhcp-next-server"><para>The - <literal>next-server</literal> directive designates - the TFTP server (the default is to use the same host as the - DHCP server).</para> + <literal>next-server</literal> directive designates the TFTP + or NFS server to use for loading loader or kernel file (the + default is to use the same host as the DHCP server).</para> </callout> - <callout arearefs="co-dhcp-filename"><para>The - <literal>filename</literal> directive defines the file that - <application>etherboot</application> will load as a - kernel. - <note><para>PXE appears to prefer a relative file - name, and it loads <command>pxeboot</command>, not the - kernel (<literal>option filename - "pxeboot"</literal>).</para> - </note> - </para> + <callout arearefs="co-dhcp-filename"><para>The + <literal>filename</literal> directive defines the file that + <application>etherboot</application> or PXE will load for the + next execution step. It must be specified according to the + transfer method used. <application>etherboot</application> can + be compiled to use NFS or TFTP. The FreeBSD port + configures NFS by default.</para> + </callout> + + <callout arearefs="co-dhcp-filenamePxe"><para>PXE uses TFTP, + which is why a relative file name is used here (this may + depend on the TFTP + server configuration, but would be fairly typical). Also, PXE + loads <filename>pxeboot</filename>, not the kernel.</para> </callout> <callout arearefs="co-dhcp-root-path"><para>The @@ -2478,7 +2574,7 @@ </indexterm> <para>Here follows an equivalent <command>bootpd</command> - configuration. This would be found in + configuration (reduced to one client). This would be found in <filename>/etc/bootptab</filename>.</para> <para>Please note that <application>etherboot</application> @@ -2529,6 +2625,10 @@ of everything. Else refer to <xref linkend="ports"> for information about ports and packages.</para> + <para>You can change the <application>etherboot</application> + configuration (ie: to use TFTP instead of NFS) by editing the + Config file in the etherboot source directory.</para> + <para>For our setup, we shall use a boot floppy. For other methods (PROM, or dos program), please refer to the <application>etherboot</application> documentation.</para> @@ -2550,6 +2650,22 @@ </sect3> + <sect3> + <title>Booting with PXE</title> + + <para>By default, the &man.pxeboot.8; loader loads the kernel via NFS. + It can be compiled to use TFTP instead by specifying the + LOADER_TFTP_SUPPORT option in + <filename>/etc/make.conf</filename>. See the comments in + <filename>/etc/defaults/make.conf</filename> for + instructions.</para> + + <para>To use PXE when the machine starts, you will usually + need to select the <literal>Boot from network</literal> option + in your BIOS setup, or type a function key during the PC + initialization.</para> + + </sect3> <sect3> <title>Configuring the TFTP and NFS Servers</title> @@ -2563,8 +2679,10 @@ <secondary>diskless operation</secondary> </indexterm> - <para>You need to enable <command>tftpd</command> on the TFTP - server:</para> + <para>If any of the boostrap programs (PXE, pxeboot, or etherboot) + are configured to use TFTP, you need to enable + <command>tftpd</command> on the file server:</para> + <procedure> <step> <para>Create a directory from which <command>tftpd</command> @@ -2575,7 +2693,7 @@ <para>Add this line to your <filename>/etc/inetd.conf</filename>:</para> - <programlisting>tftp dgram udp wait root /usr/libexec/tftpd tftpd -s /tftpboot</programlisting> + <programlisting>tftp dgram udp wait root /usr/libexec/tftpd tftpd -s /tftpboot</programlisting> <note><para>It appears that at least some PXE versions want the TCP version of TFTP. In this case, add a second line, @@ -2595,7 +2713,7 @@ location is set in both <filename>inetd.conf</filename> and <filename>dhcpd.conf</filename>.</para> - <para>You also need to enable NFS and export the + <para>In all cases, you also need to enable NFS and export the appropriate filesystem on the NFS server.</para> <procedure> @@ -2611,7 +2729,7 @@ point and replace <replaceable>margaux</replaceable> with the name of the diskless workstation):</para> - <programlisting><replaceable>/data/misc</replaceable> -alldirs -ro <replaceable>margaux</replaceable></programlisting> + <programlisting><replaceable>/data/misc</replaceable> -alldirs -ro <replaceable>margaux corbieres</replaceable></programlisting> </step> <step> <para>Tell <command>mountd</command> to reread its configuration @@ -2632,23 +2750,48 @@ <secondary>kernel configuration</secondary> </indexterm> - <para>Create a kernel configuration file for the diskless client - with the following options (in addition to the usual - ones):</para> + <para>If using <application>etherboot</application>, you need to + create a kernel configuration file for the diskless client + with the following options (in addition to the usual ones):</para> <programlisting> - options BOOTP # Use BOOTP to obtain IP address/hostname - options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info - options BOOTP_COMPAT # Workaround for broken bootp daemons. + options BOOTP # Use BOOTP to obtain IP address/hostname + options BOOTP_NFSROOT # NFS mount root filesystem using BOOTP info </programlisting> - <para>You may also want to use <literal>BOOTP_NFSV3</literal> and - <literal>BOOTP_WIRED_TO</literal> (refer to <filename>LINT</filename>).</para> - + <para>You may also want to use <literal>BOOTP_NFSV3</literal>, + <literal>BOOT_COMPAT</literal> and + <literal>BOOTP_WIRED_TO</literal> (refer to + <filename>LINT</filename>).</para> + + <para>These option names are historical and slightly + misleading as they actually enable indifferent use of DHCP or BOOTP + inside the kernel (it is also possible to force strict BOOTP or + DHCP use).</para> + <para>Build the kernel (See <xref linkend="kernelconfig">), - and copy it to the tftp directory, under the name listed - in <filename>dhcpd.conf</filename>.</para> + and copy it to the place specified in + <filename>dhcpd.conf</filename>.</para> + <note><para>When using PXE, building a kernel with the above + options is not strictly necessary. Enabling them will + cause more DHCP requests to be issued during kernel + startup, with a small risk of inconsistency between the + new values and those retrieved by pxeboot in some special + cases. The advantage of using them is that the host name + will be set as a side effect. Else you will need to set + the host name by another method, for example in a + client-specific <filename>rc.conf</filename> file.</para> + </note> + + <note><para>In order to be loadable with + <application>etherboot</application>, a 5.x kernel needs + to have the device hints compiled in. You would typically + set the following option in the configuration file (see + the NOTES configuration comments file). + <programlisting>hints "GENERIC.hints" </programlisting> + </para> + </note> </sect3> @@ -2671,6 +2814,13 @@ the place where the filesystem will be created (the <literal>DEST</literal> variable).</para> + <note><para>The <filename>clone_root</filename> script distributed + with FreeBSD 5.2 is out of date relative to the 4.x version, and + also needs a few specific 5.x adjustments. Please refer to PR + <ulink url="http://www.freebsd.org/cgi/query-pr.cgi?pr=misc/62417"> + misc/62417</ulink>.</para> + </note> + <para>Refer to the comments at the top of the script for instructions. They explain how the base filesystem is built, and how files may be selectively overridden by versions specific @@ -2689,35 +2839,64 @@ confusing. Use them for reference only, except if you prefer the method that they describe, in which case you will need customized <filename>rc</filename> scripts.</para> + + <note><para>When using PXE, it may happen that the loader will + get a wrong idea of the root filesystem to be used by the + kernel (this will be the case if the cloned root + <filename>/etc/fstab</filename> is a copy of the server's, + as the client-specific version will not be visible by the + loader, which looks for hints there).</para> + <para>This will cause + error messages at the end of the kernel initialization, + when the kernel tries to mount the wrong root and fails, + or even actual errors if it succeeds because a local disk + happens to be present. </para> + + <para>It is possible to prevent this problem from happening + by inserting the following line in + <filename>/boot/loader.conf</filename>:</para> + <programlisting> + vfs.root.mountfrom="" + </programlisting> + </note> + + </sect3> <sect3> <title>Configuring Swap</title> <para>If needed, a swap file located on the server can be - accessed via NFS. The exact <filename>bootptab</filename> - or <filename>dhcpd.conf</filename> options are not clearly - documented at this time. The following configuration - suggestions have been reported to work in some installations - using isc-dhcp 3.0rc11.</para> + accessed via NFS. One of the methods commonly used to do this + has been discontinued in release 5.X.</para> + + <sect4> + <title>NFS swap with FreeBSD 4.x</title> + + <para>The swap file location and size can be specified with + BOOTP/DHCP FreeBSD-specific options 128 and 129. Examples of + configuration files for isc-dhcp 3 or bootpd follow.</para> + <procedure> <step><para>Add the following lines to <filename>dhcpd.conf</filename>:</para> <programlisting> - # Global section - option swap-path code 128 = string; - option swap-size code 129 = integer 32; - - host margaux { - ... # Standard lines, see above - option swap-path <replaceable>"192.168.4.4:/netswapvolume/netswap"</replaceable>; - option swap-size <replaceable>64000</replaceable>; - } + # Global section + option swap-path code 128 = string; + option swap-size code 129 = integer 32; + + host margaux { + ... # Standard lines, see above + option swap-path <replaceable>"192.168.4.4:/netswapvolume/netswap"</replaceable>; + option swap-size <replaceable>64000</replaceable>; + } </programlisting> - <para>The idea is that, at least for a FreeBSD client, - DHCP/BOOTP option code 128 is the path to the NFS swap file, - and option code 129 is the swap size in kilobytes. Older - versions of <command>dhcpd</command> allowed a syntax of + + <para><literal>swap-path</literal> is the path to a directory + where swap files will be located. Each file will be named + <filename>swap.<replaceable>clientIp</replaceable></filename>.</para> + + <para>Older versions of <command>dhcpd</command> used a syntax of <literal>option option-128 "...</literal>, which is no longer supported.</para> <para><filename>/etc/bootptab</filename> would use the @@ -2733,12 +2912,12 @@ <step> <para>On the NFS swap file server, create the swap file(s)</para> - <screen> - &prompt.root; <userinput>mkdir <replaceable>/netswapvolume/netswap</replaceable></userinput> - &prompt.root; <userinput>cd <replaceable>/netswapvolume/netswap</replaceable></userinput> - &prompt.root; <userinput>dd if=/dev/zero bs=1024 count=<replaceable>64000</replaceable> of=swap.<replaceable>192.168.4.6</replaceable></userinput> - &prompt.root; <userinput>chmod 0600 swap.<replaceable>192.168.4.6</replaceable></userinput> - </screen> + <screen> + &prompt.root; <userinput>mkdir <replaceable>/netswapvolume/netswap</replaceable></userinput> + &prompt.root; <userinput>cd <replaceable>/netswapvolume/netswap</replaceable></userinput> + &prompt.root; <userinput>dd if=/dev/zero bs=1024 count=<replaceable>64000</replaceable> of=swap.<replaceable>192.168.4.6</replaceable></userinput> + &prompt.root; <userinput>chmod 0600 swap.<replaceable>192.168.4.6</replaceable></userinput> + </screen> <para><replaceable>192.168.4.6</replaceable> is the IP address for the diskless client.</para> </step> @@ -2747,13 +2926,31 @@ <para>On the NFS swap file server, add the following line to <filename>/etc/exports</filename>:</para> <programlisting> - <replaceable>/netswapvolume</replaceable> -maproot=0:10 -alldirs <replaceable>margaux</replaceable> +<replaceable>/netswapvolume</replaceable> -maproot=0:10 -alldirs <replaceable>margaux</replaceable> </programlisting> <para>Then tell <application>mountd</application> to reread the exports file, as above.</para> </step> </procedure> + </sect4> + + <sect4> + <title>NFS swap with FreeBSD 5.x</title> + + <para>The kernel does not support enabling NFS swap at boot + time. Swap must be enabled by the startup scripts, by + mounting a writeable file system and creating and enabling a + swap file. This can be done, for example, in + <filename>/etc/rc.local</filename>. Example:</para> + + <screen> + swapfile=/writableVolume/swap/swap.<replaceable>MyIP</replaceable> + dd if=/dev/zero of=$swapfile bs=1k count=1 oseek=100000 + swapon $swapfile + rm -rf $swapfile + </screen> + </sect4> </sect3> <sect3> >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200402141240.i1ECeAph034895>