Date: Fri, 2 Sep 2011 20:08:15 -0400 From: Glen Barber <gjb@FreeBSD.org> To: Craig Rodrigues <rodrigc@crodrigues.org> Cc: kan@freebsd.org, rmacklem@freebsd.org, freebsd-doc@freebsd.org, alfred@freebsd.org Subject: Re: PXE Booting with an NFS root file system Message-ID: <20110903000815.GA72769@glenbarber.us> In-Reply-To: <CAG=rPVdZVDo9Vhdj9u1UcopmEcVS3G9PyfFLs%2By-vY5Nn98-mg@mail.gmail.com> References: <CAG=rPVenPP0n5uu8JGAksyTbPW3zMfxz%2BEJNvsPZ=o4P_pex3A@mail.gmail.com> <alpine.BSF.2.00.1108300808040.46886@wonkity.com> <CAG=rPVe4MxScm0B58o1w25hyBrt8-Psh3sobiL90dZm7PFCHpw@mail.gmail.com> <CAG=rPVdZVDo9Vhdj9u1UcopmEcVS3G9PyfFLs%2By-vY5Nn98-mg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Craig Rodrigues wrote:
> Hi,
>
> I have incorporated more feedback from Glen Barber and Ryusuke Suzuki.
>
Hi Craig,
Comments are in-line (not many):
Index: en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml
===================================================================
RCS file: /home/dcvs/doc/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml,v
retrieving revision 1.436
diff -u -r1.436 chapter.sgml
--- en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml 18 Feb 2011 18:10:36 -0000 1.436
+++ en_US.ISO8859-1/books/handbook/advanced-networking/chapter.sgml 2 Sep 2011 21:19:42 -0000
@@ -33,6 +33,10 @@
</listitem>
<listitem>
+ <para>How to set up network PXE booting with an NFS root file system.</para>
+ </listitem>
+
+ <listitem>
<para>How to set up network address translation.</para>
</listitem>
@@ -4171,6 +4175,304 @@
</sect2>
</sect1>
+ <sect1 id="network-pxe-nfs">
+ <sect1info>
+ <authorgroup>
+ <author>
+ <firstname>Craig</firstname>
+ <surname>Rodrigues</surname>
+ <affiliation>
+ <address>rodrigc@FreeBSD.org</address>
+ </affiliation>
+ <contrib>Written by </contrib>
+ </author>
+ </authorgroup>
+ </sect1info>
+ <title>PXE Booting with an NFS root file system</title>
+
+ <para>The &intel; Preboot eXecution Environment (<acronym>PXE</acronym>)
+ allows booting the operating system over the network.
+ <acronym>PXE</acronym> support is usually provided in the
+ <acronym>BIOS</acronym> of modern motherboards, where
+ it can be enabled in the <acronym>BIOS</acronym> settings
+ which enable booting from the network. A fully functioning
+ <acronym>PXE</acronym> setup also requires properly configured
+ <acronym>DHCP</acronym> and <acronym>TFTP</acronym> servers.</para>
+
+ <para>When the host computer boots, it receives information over
+ <acronym>DHCP</acronym> about where to obtain the initial boot
+ loader via TFTP. After the host computer receives this information,
+ it downloads the boot loader via <acronym>TFTP</acronym>, and then
+ executes the boot loader. This is documented section 2.2.1 of the
+ <ulink url="http://download.intel.com/design/archives/wfm/downloads/pxespec.pdf">Preboot Execution Environment (PXE) Specification</ulink>.
+ In &os;, the boot loader retrieved during the <acronym>PXE</acronym>
+ process is <filename>/boot/pxeboot</filename>. After
+ <filename>/boot/pxeboot</filename> executes, the &os; kernel is
+ loaded, and the rest of the &os; bootup sequence proceeds.
+ Refer to <link linkend="boot">The FreeBSD Booting Process</link>
I think FreeBSD can be replaced by &os; here...
+ of the &os; Handbook.</para>
+
+ <para>The <literal>pxeboot</literal> implementation in &os; was
+ written by &a.ps; and &a.jhb;. It is documented in the
+ &man.pxeboot.8; man page written by &a.dwhite;, and in the
+ <ulink url="&url.articles.pxe;/article.html">FreeBSD Jumpstart Guide</ulink>
... and here.
+ written by &a.alfred;.</para>
+
+ <sect2>
+ <title>Setting Up the <command>chroot</command> Environment for the NFS Root File system</title>
+
+ <procedure>
+ <step>
+ <para>Choose a directory which will have a &os; installation
+ which will be NFS mountable. For example, a directory such
+ as <filename>/b/tftpboot/FreeBSD/install</filename> can be used.</para>
+
+ <screen>&prompt.root; <userinput>export NFSROOTDIR=/b/tftpboot/FreeBSD/install</userinput>
+&prompt.root; <userinput>mkdir -p ${NFSROOTDIR}</userinput></screen>
+ </step>
+
+ <step>
+ <para>Enable the NFS server by following the instructions in
+ the <link linkend="network-configuring-nfs">Configuring
+ NFS</link> section of the &os; Handbook.</para>
+ </step>
+
+ <step>
+ <para>Export the directory via NFS by adding the following to
+ <filename>/etc/exports</filename>:</para>
+
+ <programlisting>/b -ro -alldirs</programlisting>
+ </step>
+
+ <step>
+ <para>Restart the NFS server:</para>
+
+ <screen>&prompt.root; <userinput>/etc/rc.d/nfsd restart</userinput></screen>
+ </step>
+
+ <step>
+ <para>Enable &man.inetd.8; by following the steps outlined in
+ <xref linkend="network-inetd-settings">.</para>
+ </step>
+
+ <step>
+ <para>Add the following line to
+ <filename>/etc/inetd.conf</filename>:</para>
+
+ <programlisting>tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /b/tftpboot</programlisting>
+ </step>
+
+ <step>
+ <para>Restart inetd:</para>
+
+ <screen>&prompt.root; <userinput>/etc/rc.d/inetd restart</userinput></screen>
+ </step>
+
+ <step>
+ <para>Rebuild the &os; kernel and userland (see
+ <link linkend="makeworld">Rebuilding "world"</link>
+ in the &os; Handbook.)</para>
+
+ <screen>&prompt.root; <userinput>cd /usr/src</userinput>
+&prompt.root; <userinput>make buildworld</userinput>
+&prompt.root; <userinput>make buildkernel</userinput></screen>
+ </step>
+
+ <step>
+ <para>Install &os; into the directory mounted over
+ <acronym>NFS</acronym>.</para>
+
+ <screen>
+&prompt.root; <userinput>make installworld DESTDIR=${NFSROOTDIR}</userinput>
+&prompt.root; <userinput>make installkernel DESTDIR=${NFSROOTDIR}</userinput>
+&prompt.root; <userinput>make distribution DESTDIR=${NFSROOTDIR}</userinput>
+ </screen>
+ </step>
+
+ <step>
+ <para>Test that the <acronym>TFTP</acronym> server works and
+ can download the boot loader which will be obtained via PXE:</para>
+
+ <screen>
+&prompt.root; <userinput>tftp localhost</userinput>
+tftp> <userinput>get FreeBSD/install/boot/pxeboot</userinput>
+Received 264951 bytes in 0.1 seconds
+ </screen>
+ </step>
+
+ <step>
+ <para>Edit <filename>${NFSROOTDIR}/etc/fstab</filename> and create an entry
+ to mount the root file system over NFS:</para>
+
+ <programlisting>
+# Device Mountpoint FSType Options Dump Pass
+myhost.example.com:/b/tftpboot/FreeBSD/install / nfs ro 0 0
+ </programlisting>
+
+ <para>Replace <replaceable>myhost.example.com</replaceable>
+ with the hostname or IP address of your <acronym>NFS</acronym>
+ server. In this example, the root file system is mounted
+ "read-only" in order to prevent <acronym>NFS</acronym>
+ clients from potentially deleting the contents of the root
+ file system.</para>
+ </step>
+
+ <step>
+ <screen>&prompt.root; <userinput>chroot ${NFSROOTDIR}</userinput>
+&prompt.root; <userinput>passwd</userinput></screen>
+ </step>
+
+ <step>
+ <para>Enable ssh root logins in
+ <filename>${NFSROOTDIR}/etc/ssh/sshd_config</filename></para>
There is a missing "." after </filename>.
+ </step>
+
+ <step>
+ <para>Customize the ${NFSROOTDIR} by using &man.chroot.8;. In the
+ <command>chroot</command> environment, the ${NFSROOTDIR} can be
+ customized by doing things like adding packages with
+ <command>pkg_add</command>, editing the password file with
+ &man.vipw.8;, editing &man.amd.conf.5; maps for automounting,
+ etc. For example:</para>
+
+ <screen>
+&prompt.root; <userinput>chroot ${NFSROOTDIR}</userinput>
+&prompt.root; <userinput>pkg_add -r bash</userinput></screen>
+ </step>
+ </procedure>
+ </sect2>
+
+ <sect2>
+ <title>Configuring Memory File Systems used by <filename>/etc/rc.initdiskless</filename></title>
+
+ <para>If you boot from an NFS root volume,
+ <filename>/etc/rc</filename>
+ detects that you booted over NFS and runs
+ <filename>/etc/rc.initdiskless</filename>.
+ Read this script to understand what is going on. We need to make
+ <filename>/etc</filename> and <filename>/var</filename> memory backed
+ file systems because these directories need to be writable, but
+ the NFS root directory is read-only.</para>
+
+ <screen>
+&prompt.root; <userinput>chroot ${NFSROOTDIR}</userinput>
+&prompt.root; <userinput>mkdir -p conf/base</userinput>
+&prompt.root; <userinput>tar -c -v -f conf/base/etc.cpio.gz --format cpio --gzip etc</userinput>
+&prompt.root; <userinput>tar -c -v -f conf/base/var.cpio.gz --format cpio --gzip var</userinput></screen>
+
+ <para>When system boots, memory file systems for
+ <filename>/etc</filename> and <filename>/var</filename>
+ will be created and mounted, and the contents of the
+ <filename>cpio.gz</filename> files will be copied into them.</para>
+ </sect2>
+
+ <sect2>
+ <title>Setting up the DHCP Server</title>
+
+ <para>PXE requires a <acronym>TFTP</acronym> server and a
+ <acronym>DHCP</acronym> server to be set up. The
+ <acronym>DHCP</acronym> server does not necessarily need
+ to be the same machine as the <acronym>TFTP</acronym> server,
+ but it needs to be accessible in your network.</para>
+
+ <procedure>
+ <step>
+ <para>Install the <acronym>DHCP</acronym> server by following
+ the instructions documented at
+ <link linkend="network-dhcp-server">Installing and Configuring a DHCP Server</link>
+ in the &os; Handbook. Make sure that <filename>/etc/rc.conf</filename>
+ and <filename>/usr/local/etc/dhcpd.conf</filename>
+ are correctly configured.</para>
+ </step>
+
+ <step>
+ <para>In <filename>/usr/local/etc/dhcpd.conf</filename>, configure
+ the <literal>next-server</literal>, <literal>filename</literal>,
+ and <literal>option root-path</literal> settings,
+ to specify your <acronym>TFTP</acronym> server IP address,
+ the path to <filename>/boot/pxeboot</filename> in
+ <acronym>TFTP</acronym>, and the path to <acronym>NFS</acronym>
+ root file system. Here is a sample <filename>dhcpd.conf</filename>
+ setup:</para>
+
+ <programlisting>
+subnet 192.168.0.0 netmask 255.255.255.0 {
+ range 192.168.0.2 192.168.0.3 ;
+ option subnet-mask 255.255.255.0 ;
+ option routers 192.168.0.1 ;
+ option broadcast-address 192.168.0.255 ;
+ option domain-name-server 192.168.35.35, 192.168.35.36 ;
+ option domain-name "example.com";
+
+ # IP address of TFTP server
+ next-server 192.168.0.1 ;
+
+ # path of boot loader obtained
+ # via tftp
+ filename "FreeBSD/install/boot/pxeboot" ;
+
+ # pxeboot boot loader will try to NFS mount this directory for root FS
+ option root-path "192.168.0.1:/b/tftpboot/FreeBSD/intall/" ;
+
+}
+ </programlisting>
+ </step>
+ </procedure>
+
+ <sect2><title>Configuring the PXE client and Debugging Connection Problems</title>
+ <procedure>
+ <step>
+ <para>When the client machine boots up, enter the
+ <acronym>BIOS</acronym> configuration menu. Configure the
+ <acronym>BIOS</acronym> to boot from the network. If all your
+ previous configuration steps are correct, then everything should
+ "just work".</para>
+ </step>
+
+ <step>
+ <para>Use the <filename role="package">net/wireshark</filename>
+ port to debug the <acronym>DHCP</acronym> and <acronym>TFTP</acronym>
+ network traffic to look for any problems.
There is a missing </para> after "problems."
+ </step>
+
+ <step>
+ <para>Make sure that the <filename>pxeboot</filename> file can
+ be retrieved by <acronym>TFTP</acronym>. On your
+ <acronym>TFTP</acronym> server, look in
+ <filename>/var/log/xferlog</filename> to ensure that the
+ <filename>pxeboot</filename> file is being retrieved from
+ the correct location. To test the configuration from
+ <filename>dhcpd.conf</filename> above:</para>
+
+ <screen>&prompt.root; <userinput>tftp 192.168.0.1</userinput>
+tftp> <userinput>get FreeBSD/install/boot/pxeboot</userinput>
+Received 264951 bytes in 0.1 seconds</screen>
+ </step>
+
+ <step>
+ <para>Make sure that the root file system can be mounted
+ via <acronym>NFS</acronym>. To test configuration from
+ <filename>dhcpd.conf</filename> above:</para>
+
+ <screen>&prompt.root; <userinput>mount -t nfs 192.168.0.1:/b/tftpboot/FreeBSD/install /mnt</userinput></screen>
+ </step>
+
+ <step>
+ <para>Read the code in <filename>src/sys/boot/i386/libi386/pxe.c</filename>
+ to understand how the <filename>pxeboot</filename> loader sets
+ variables like <literal>boot.nfsroot.server</literal> and
+ <literal>boot.nfsroot.path</literal>. These variables are then
+ used in the NFS diskless root mount code in
+ <filename>src/sys/nfsclient/nfs_diskless.c</filename>.</para>
+ </step>
+
+ <step>
+ <para>Read &man.pxeboot.8; and &man.loader.8;.</para>
+ </step>
+ </procedure>
+ </sect2>
+ </sect1>
+
<sect1 id="network-isdn">
<title>ISDN</title>
I think the rest looks good. Nice work!
Glen
--
Glen Barber | gjb@FreeBSD.org
FreeBSD Documentation Project
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110903000815.GA72769>