From owner-freebsd-net@freebsd.org Wed Sep 19 02:58:11 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C15BC108B810 for ; Wed, 19 Sep 2018 02:58:11 +0000 (UTC) (envelope-from kiri@kx.openedu.org) Received: from kx.openedu.org (flets-sg1027.kamome.or.jp [202.216.24.27]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1215E8E38D for ; Wed, 19 Sep 2018 02:58:09 +0000 (UTC) (envelope-from kiri@kx.openedu.org) Received: from kx.openedu.org (kx.openedu.org [202.216.24.27]) by kx.openedu.org (8.14.5/8.14.5) with ESMTP id w8J2w72D053986; Wed, 19 Sep 2018 11:58:07 +0900 (JST) (envelope-from kiri@kx.openedu.org) Message-Id: <201809190258.w8J2w72D053986@kx.openedu.org> Date: Wed, 19 Sep 2018 11:58:07 +0900 From: KIRIYAMA Kazuhiko To: "Andrey V. Elsukov" Cc: KIRIYAMA Kazuhiko , freebsd-net@freebsd.org Subject: Re: NFS poor performance in ipfw_nat In-Reply-To: <201809190033.w8J0X0J5051781@kx.openedu.org> References: <201809172253.w8HMrXSS025987@kx.openedu.org> <8315728b-afe9-7631-d2ad-2d9b06c3d72d@yandex.ru> <201809190033.w8J0X0J5051781@kx.openedu.org> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.6 MULE XEmacs/21.4 (patch 22) (Instant Classic) (amd64--freebsd) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=ISO-2022-JP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2018 02:58:12 -0000 At Wed, 19 Sep 2018 09:33:00 +0900, 私 wrote: > > At Tue, 18 Sep 2018 14:02:18 +0300, > Andrey V. Elsukov wrote: > > > > [1 Re: NFS poor performance in ipfw_nat ] > > [1.1 ] > > On 18.09.2018 01:53, KIRIYAMA Kazuhiko wrote: > > > Hi, all > > > > > > I'm working on ipfw_nat box with port redirect for sunrpc > > > (111) and nfsd (2049): > > > > > > # ifconfig > > > em0: flags=8843 metric 0 mtu 1500 > > > options=85259b > > > igb0: flags=8843 metric 0 mtu 1500 > > > options=e505bb > > > # ipfw nat show config > > > ipfw nat 123 config if em0 log deny_in same_ports unreg_only reset redirect_port tcp 192.168.1.253:22 22253 redirect_port tcp 192.168.1.252:22 22252 redirect_port tcp 192.168.1.251:22 22251 redirect_port tcp 192.168.1.250:22 22250 redirect_port tcp 192.168.1.249:22 22249 redirect_port tcp 192.168.1.248:22 22248 redirect_port tcp 192.168.1.247:22 22247 redirect_port tcp 192.168.1.246:22 22246 redirect_port tcp 192.168.1.245:22 22245 redirect_port tcp 192.168.1.244:22 22244 redirect_port tcp 192.168.1.243:22 22243 > > > > Is there any suggestions ? > > > > > > > Hi, > > > > try to disable TSO on your NICs. > > Thanks for your advice. Add '-lro' and '-tso' to ifconfig, > transfer rate up to almost native NIC speed: > > # dd if=/dev/zero of=/.dake/tmp/foo.img bs=1k count=1m > 1048576+0 records in > 1048576+0 records out > 1073741824 bytes transferred in 10.688162 secs (100460852 bytes/sec) > # > > BTW in VM on behyve, transfer rate to NFS mount of VM server > (bhyve) is appreciably low level: > > # dd if=/dev/zero of=/.dake/tmp/foo.img bs=1k count=1m > 1048576+0 records in > 1048576+0 records out > 1073741824 bytes transferred in 32.094448 secs (33455687 bytes/sec) This was limited by disk transfer speed: # dd if=/dev/zero of=/var/tmp/foo.img bs=1k count=1m 1048576+0 records in 1048576+0 records out 1073741824 bytes transferred in 21.692358 secs (49498623 bytes/sec) # > # ifconfig > vtnet0: flags=8943 metric 0 mtu 1500 > options=80028 > ether 58:9c:fc:0c:10:58 > inet 192.168.1.193 netmask 0xffffff00 broadcast 192.168.1.255 > media: Ethernet 10Gbase-T > status: active > nd6 options=29 > lo0: flags=8049 metric 0 mtu 16384 > options=680003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > groups: lo > nd6 options=21 > # df > Filesystem 1K-blocks Used Avail Capacity Mounted on > /dev/vtbd0p2 5060988 4244412 411700 91% / > devfs 1 1 0 100% /dev > vm.pis:/.dake 14773545597 51624777 14721920820 0% /.dake > vm.pis:/ds/src/current/12.0/r338547 14749697607 27776787 14721920820 0% /usr/src > vm.pis:/ds/obj/current/12.0/r338547 14772357057 50436237 14721920820 0% /usr/obj > # cat /etc/rc.conf > hostname="tbedfc" > ifconfig_vtnet0="inet 192.168.1.193 netmask 255.255.255.0" > defaultrouter="192.168.1.254" > sshd_enable="YES" > nfs_client_enable="YES" > rpc_lockd_enable="YES" > rpc_statd_enable="YES" > moused_enable="YES" > ntpd_enable="YES" > powerd_enable="YES" > dumpdev="AUTO" > # uname -a > FreeBSD tbedfc 12.0-ALPHA5 FreeBSD 12.0-ALPHA5 #0: Mon Sep 10 22:19:36 JST 2018 root@tbedfc:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > # > > It seems to be unconcerned to TSO or LRO ... > > > > > -- > > WBR, Andrey V. Elsukov > > > > [2 OpenPGP digital signature ] > > > > --- > KIRIYAMA Kazuhiko > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >