From owner-freebsd-questions@freebsd.org Thu Mar 2 20:13:22 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50E5ACF593E for ; Thu, 2 Mar 2017 20:13:22 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 311E3E26 for ; Thu, 2 Mar 2017 20:13:22 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id v22KDKPg008431 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 2 Mar 2017 15:13:20 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:b5c7:64f5:af7c:6d7b] ([IPv6:2607:f3e0:0:4:b5c7:64f5:af7c:6d7b]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id v22KDIgg088098 for ; Thu, 2 Mar 2017 15:13:18 -0500 (EST) (envelope-from mike@sentex.net) To: freebsd-questions From: Mike Tancsa Subject: speeding up pxe netboot Organization: Sentex Communications Message-ID: Date: Thu, 2 Mar 2017 15:13:17 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Mar 2017 20:13:22 -0000 I am not sure if something broke in my environment or not, but it seems something is causing pxe boots to take forever now. The nfs server is RELENG11 and I have pretty much the defaults for serving up nfs access. The slow part seems to be on the boot up from /boot/kernel/kernel text=0x1532060 \ On a directly connected gigabit network, its only serving up traffic at 0.6Mb/s Once the box fully boots however, the nfs mount works at full gigabit speeds for read and write. From another box that is not netbooted, if I do a nfs mount and force UDP, its not quite gigabit speeds, but still greater than 500Mb # dd if=/mnt/test of=/dev/null bs=1m 1205+0 records in 1205+0 records out 1263534080 bytes transferred in 18.621882 secs (67852115 bytes/sec) or 542Mb/s Any idea how / where to troubleshoot this ? I am booting a RELENG_11 client from a RELENG_11 server. config bits below next-server 10.0.0.1; host s-microx11 { hardware ethernet 0c:c4:7a:zz:xx:xx; fixed-address 10.0.0.42; filename "/11/boot/pxeboot"; option root-path "/pxe/11"; option host-name "s-microx11.sentex.ca"; } nfs_client_enable="YES" nfs_server_enable="YES" nfs_server_flags="-u -t" rpcbind_enable="YES" mountd_enable="YES" mountd_flags="-l -r -S" rpc_lockd_enable="YES" rpc_statd_enable="YES" Nothing in the tcpdumpd jumps out 12:08:23.719761 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 738 104 lookup fh Unknown/888D10CADE13D1BC0A0018680400000059813E000000000000000000 "kernel" 12:08:23.719924 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 738 reply ok 236 lookup fh Unknown/888D10CADE13D1BC0A00018004000000568B3E000000000000000000 12:08:23.748818 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 739 104 lookup fh Unknown/888D10CADE13D1BC0A00018004000000568B3E000000000000000000 "kernel" 12:08:23.749020 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 739 reply ok 236 lookup fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 12:08:23.762367 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 740 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 0 12:08:23.762577 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 740 reply ok 1152 read 12:08:23.775816 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 741 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 1024 12:08:23.775998 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 741 reply ok 1152 read 12:08:23.789392 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 742 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 2048 12:08:23.789598 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 742 reply ok 1152 read 12:08:23.802893 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 743 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 3072 12:08:23.803089 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 743 reply ok 1152 read 12:08:23.819173 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 744 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 4096 12:08:23.819382 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 744 reply ok 1152 read 12:08:23.848305 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 745 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 5120 12:08:23.848455 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 745 reply ok 1152 read 12:08:23.861720 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 746 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 6144 12:08:23.861857 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 746 reply ok 1152 read 12:08:23.875188 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 747 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 7168 12:08:23.875395 IP 10.0.0.1.2049 > 10.0.0.42.987: NFS reply xid 747 reply ok 1152 read 12:08:23.888799 IP 10.0.0.42.987 > 10.0.0.1.2049: NFS request xid 748 104 read fh Unknown/888D10CADE13D1BC0A00028004000000568B3E000000000000000000 1024 bytes @ 8192 ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/