From owner-freebsd-net@FreeBSD.ORG Wed Apr 11 10:02:41 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C041E1065672; Wed, 11 Apr 2012 10:02:41 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 622F38FC08; Wed, 11 Apr 2012 10:02:41 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 3A5C57300A; Wed, 11 Apr 2012 12:21:58 +0200 (CEST) Date: Wed, 11 Apr 2012 12:21:58 +0200 From: Luigi Rizzo To: Jason Hellenthal Message-ID: <20120411102158.GA59821@onelab2.iet.unipi.it> References: <20120410225257.GB53350@onelab2.iet.unipi.it> <20120411040948.GA24775@DataIX.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120411040948.GA24775@DataIX.net> User-Agent: Mutt/1.4.2.3i Cc: current@freebsd.org, net@freebsd.org Subject: Re: strange ping response times... X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2012 10:02:41 -0000 On Wed, Apr 11, 2012 at 12:09:48AM -0400, Jason Hellenthal wrote: > ... > > Apparently a ping -f has a much lower RTT than one with non-zero > > delay between transmissions. Part of the story could be that > > the flood version invokes a non-blocking select. > > On the other hand, pinging on the loopback should make > > the response available right away, so what could be the reason > > for the additional 3..10us in the ping response time ? > > > > The following are numbers on an i7-2600k at 3400 MHz + turboboost, > > running stable/9 amd64. Note how the min ping time significantly > > increases moving from flood to 10ms to 1s. > > On an Intel 10G interface i am seeing a min of 14-16us with > > Sorry but I am having trouble wrapping my head around this as the > results below are only for the loopback address 127.0.0.1 that does not > travel on the wire at all ? Did you assign this to the 10ge interface ? > if so which result is which. ? yes i have set net.inet.icmp.icmplim=0 the 127.0.0.1 are purely through the software path. The 10ge numbers are between two freebsd machines connected back-to-back with an SFP+ twinax cable. In this case i get a min RTT of 12-13us with a ping -f, and 22-25us with ping -i 0.001 For the 10ge experiments, there is a clear dependency on the ixgbe interrupt mitigation delay (setting to 10Kintr/s gives ~120us RTT) but once you get down to 30-35us the mitigation has no more effect and you need other tricks (such as disabling AIM, ipfw, ipv6) to shave the extra 3-4us. In both cases there are several microseconds wasted between the -f and normal case. cheers luigi > In order to get any good results on my machine I had to tune the > following. > > net.inet.icmp.reply_from_interface=1 > net.inet.icmp.icmplim_output=0 > net.inet.icmp.icmplim=0 > net.inet.icmp.log_redirect=1 > > You may have other options if this is greater than stable/8 @ 234095 > > But yes the results seem skewed to some extent and being loopback I > would think there should be a very near zero rx/tx time. > > > > a ping flood, and up to 33-35us with the standard 1s interval > > (using -q probably trims another 2..5us) > > > > > sudo ping -c 1000 -q -f 127.0.0.1 > > round-trip min/avg/max/stddev = 0.002/0.003/0.012/0.001 ms > > > sudo ping -c 1000 -q -f 127.0.0.1 > > round-trip min/avg/max/stddev = 0.002/0.002/0.013/0.001 ms > > > sudo ping -c 1000 -q -f 127.0.0.1 > > round-trip min/avg/max/stddev = 0.002/0.002/0.013/0.001 ms > > > sudo ping -c 10000 -q -f 127.0.0.1 > > round-trip min/avg/max/stddev = 0.002/0.002/0.011/0.001 ms > > > > > sudo ping -c 1000 -q -i 0.01 127.0.0.1 > > round-trip min/avg/max/stddev = 0.005/0.012/0.017/0.001 ms > > > sudo ping -c 1000 -q -i 0.01 127.0.0.1 > > round-trip min/avg/max/stddev = 0.004/0.012/0.016/0.001 ms > > > sudo ping -c 200 -q -i 0.01 127.0.0.1 > > round-trip min/avg/max/stddev = 0.007/0.012/0.017/0.002 ms > > > sudo ping -c 200 -q -i 0.01 127.0.0.1 > > round-trip min/avg/max/stddev = 0.005/0.012/0.018/0.002 ms > > > sudo ping -c 200 -q -i 0.01 127.0.0.1 > > round-trip min/avg/max/stddev = 0.009/0.012/0.020/0.002 ms > > > sudo ping -c 200 -q -i 0.01 127.0.0.1 > > round-trip min/avg/max/stddev = 0.006/0.012/0.016/0.001 ms > > > > > sudo ping -c 200 -q -i 0.1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.007/0.014/0.020/0.001 ms > > > sudo ping -c 200 -q -i 0.1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.006/0.014/0.019/0.002 ms > > > sudo ping -c 200 -q -i 0.1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.007/0.014/0.021/0.001 ms > > > sudo ping -c 200 -q -i 0.1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.007/0.014/0.020/0.001 ms > > > sudo ping -c 200 -q -i 0.1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.006/0.014/0.021/0.002 ms > > > sudo ping -c 200 -q -i 0.1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.010/0.014/0.022/0.001 ms > > > > > sudo ping -c 20 -q -i 1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.013/0.018/0.022/0.002 ms > > > sudo ping -c 20 -q -i 1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.012/0.018/0.021/0.002 ms > > > sudo ping -c 20 -q -i 1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.009/0.017/0.018/0.002 ms > > > sudo ping -c 20 -q -i 1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.011/0.017/0.021/0.002 ms > > > sudo ping -c 20 -q -i 1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.010/0.017/0.020/0.002 ms > > > sudo ping -c 20 -q -i 1 127.0.0.1 > > round-trip min/avg/max/stddev = 0.009/0.017/0.028/0.004 ms > > > > cheers > > luigi > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > -- > ;s =;