From owner-freebsd-questions@freebsd.org Fri Mar 3 15:56:38 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D8FDCF696D for ; Fri, 3 Mar 2017 15:56:38 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 232261CF1 for ; Fri, 3 Mar 2017 15:56:37 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id v23FuanB093483 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 3 Mar 2017 10:56:37 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c] ([IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id v23FuYXL091985 for ; Fri, 3 Mar 2017 10:56:34 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: speeding up pxe netboot To: freebsd-questions References: From: Mike Tancsa Organization: Sentex Communications Message-ID: <6aec04eb-c751-4b76-2c73-cfb42ae75a29@sentex.net> Date: Fri, 3 Mar 2017 10:56:35 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Mar 2017 15:56:38 -0000 OK, just for the archives, this seems to be an issue with a new class of SuperMicro motherboard we started using. For whatever reason, the initial pxe boot takes forever. Using different motherboards shows the normal expected speeds. Supermicro X11SSL-F, BIOS Revision: 5.11 1.0b Release Date: 12/29/2015 ---Mike On 3/2/2017 3:13 PM, Mike Tancsa wrote: > I am not sure if something broke in my environment or not, but it seems > something is causing pxe boots to take forever now. > > The nfs server is RELENG11 and I have pretty much the defaults for > serving up nfs access. > > The slow part seems to be on the boot up from > /boot/kernel/kernel text=0x1532060 \ > > On a directly connected gigabit network, its only serving up traffic at > 0.6Mb/s > > Once the box fully boots however, the nfs mount works at full gigabit > speeds for read and write. From another box that is not netbooted, if I > do a nfs mount and force UDP, its not quite gigabit speeds, but still > greater than 500Mb > > > # dd if=/mnt/test of=/dev/null bs=1m > 1205+0 records in > 1205+0 records out > 1263534080 bytes transferred in 18.621882 secs (67852115 bytes/sec) > or 542Mb/s > > Any idea how / where to troubleshoot this ? I am booting a RELENG_11 > client from a RELENG_11 server. > > > config bits below > next-server 10.0.0.1; > > host s-microx11 { > hardware ethernet 0c:c4:7a:zz:xx:xx; > fixed-address 10.0.0.42; > filename "/11/boot/pxeboot"; > option root-path "/pxe/11"; > option host-name "s-microx11.sentex.ca"; > } > > > > nfs_client_enable="YES" > nfs_server_enable="YES" > nfs_server_flags="-u -t" > rpcbind_enable="YES" > mountd_enable="YES" > mountd_flags="-l -r -S" > rpc_lockd_enable="YES" > rpc_statd_enable="YES" > > > Nothing in the tcpdumpd jumps out > > 12:08:23.719761 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 738 > 104 lookup fh > Unknown/888D10CADE13D1BC0A0018680400000059813E000000000000000000 > "kernel" > 12:08:23.719924 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 738 > reply ok 236 lookup fh > Unknown/888D10CADE13D1BC0A00018004000000568B3E000000000000000000 > 12:08:23.748818 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 739 > 104 lookup fh > Unknown/888D10CADE13D1BC0A00018004000000568B3E000000000000000000 > "kernel" > 12:08:23.749020 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 739 > reply ok 236 lookup fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 > 12:08:23.762367 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 740 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 0 > 12:08:23.762577 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 740 > reply ok 1152 read > 12:08:23.775816 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 741 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 1024 > 12:08:23.775998 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 741 > reply ok 1152 read > 12:08:23.789392 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 742 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 2048 > 12:08:23.789598 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 742 > reply ok 1152 read > 12:08:23.802893 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 743 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 3072 > 12:08:23.803089 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 743 > reply ok 1152 read > 12:08:23.819173 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 744 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 4096 > 12:08:23.819382 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 744 > reply ok 1152 read > 12:08:23.848305 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 745 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 5120 > 12:08:23.848455 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 745 > reply ok 1152 read > 12:08:23.861720 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 746 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 6144 > 12:08:23.861857 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 746 > reply ok 1152 read > 12:08:23.875188 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 747 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 7168 > 12:08:23.875395 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 747 > reply ok 1152 read > 12:08:23.888799 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 748 > 104 read fh > Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 > bytes @ 8192 > > > > ---Mike > > > > > -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/