Date: Fri, 3 Mar 2017 10:56:35 -0500 From: Mike Tancsa <mike@sentex.net> To: freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: speeding up pxe netboot Message-ID: <6aec04eb-c751-4b76-2c73-cfb42ae75a29@sentex.net> In-Reply-To: <e82cc46b-a2c6-8167-0fcc-c7ecb7cf0edd@sentex.net> References: <e82cc46b-a2c6-8167-0fcc-c7ecb7cf0edd@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
OK, just for the archives, this seems to be an issue with a new class of
SuperMicro motherboard we started using. For whatever reason, the
initial pxe boot takes forever. Using different motherboards shows the
normal expected speeds.
Supermicro X11SSL-F, BIOS Revision: 5.11 1.0b Release Date: 12/29/2015
---Mike
On 3/2/2017 3:13 PM, Mike Tancsa wrote:
> I am not sure if something broke in my environment or not, but it seems
> something is causing pxe boots to take forever now.
>
> The nfs server is RELENG11 and I have pretty much the defaults for
> serving up nfs access.
>
> The slow part seems to be on the boot up from
> /boot/kernel/kernel text=0x1532060 \
>
> On a directly connected gigabit network, its only serving up traffic at
> 0.6Mb/s
>
> Once the box fully boots however, the nfs mount works at full gigabit
> speeds for read and write. From another box that is not netbooted, if I
> do a nfs mount and force UDP, its not quite gigabit speeds, but still
> greater than 500Mb
>
>
> # dd if=/mnt/test of=/dev/null bs=1m
> 1205+0 records in
> 1205+0 records out
> 1263534080 bytes transferred in 18.621882 secs (67852115 bytes/sec)
> or 542Mb/s
>
> Any idea how / where to troubleshoot this ? I am booting a RELENG_11
> client from a RELENG_11 server.
>
>
> config bits below
> next-server 10.0.0.1;
>
> host s-microx11 {
> hardware ethernet 0c:c4:7a:zz:xx:xx;
> fixed-address 10.0.0.42;
> filename "/11/boot/pxeboot";
> option root-path "/pxe/11";
> option host-name "s-microx11.sentex.ca";
> }
>
>
>
> nfs_client_enable="YES"
> nfs_server_enable="YES"
> nfs_server_flags="-u -t"
> rpcbind_enable="YES"
> mountd_enable="YES"
> mountd_flags="-l -r -S"
> rpc_lockd_enable="YES"
> rpc_statd_enable="YES"
>
>
> Nothing in the tcpdumpd jumps out
>
> 12:08:23.719761 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 738
> 104 lookup fh
> Unknown/888D10CADE13D1BC0A0018680400000059813E000000000000000000
> "kernel"
> 12:08:23.719924 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 738
> reply ok 236 lookup fh
> Unknown/888D10CADE13D1BC0A00018004000000568B3E000000000000000000
> 12:08:23.748818 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 739
> 104 lookup fh
> Unknown/888D10CADE13D1BC0A00018004000000568B3E000000000000000000
> "kernel"
> 12:08:23.749020 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 739
> reply ok 236 lookup fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000
> 12:08:23.762367 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 740
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 0
> 12:08:23.762577 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 740
> reply ok 1152 read
> 12:08:23.775816 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 741
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 1024
> 12:08:23.775998 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 741
> reply ok 1152 read
> 12:08:23.789392 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 742
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 2048
> 12:08:23.789598 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 742
> reply ok 1152 read
> 12:08:23.802893 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 743
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 3072
> 12:08:23.803089 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 743
> reply ok 1152 read
> 12:08:23.819173 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 744
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 4096
> 12:08:23.819382 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 744
> reply ok 1152 read
> 12:08:23.848305 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 745
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 5120
> 12:08:23.848455 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 745
> reply ok 1152 read
> 12:08:23.861720 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 746
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 6144
> 12:08:23.861857 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 746
> reply ok 1152 read
> 12:08:23.875188 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 747
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 7168
> 12:08:23.875395 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 747
> reply ok 1152 read
> 12:08:23.888799 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 748
> 104 read fh
> Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024
> bytes @ 8192
>
>
>
> ---Mike
>
>
>
>
>
--
-------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada http://www.tancsa.com/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6aec04eb-c751-4b76-2c73-cfb42ae75a29>