From owner-freebsd-net@FreeBSD.ORG  Tue Apr 10 23:12:53 2012
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id B95E71065675;
	Tue, 10 Apr 2012 23:12:53 +0000 (UTC)
	(envelope-from luigi@onelab2.iet.unipi.it)
Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238])
	by mx1.freebsd.org (Postfix) with ESMTP id 77EE78FC08;
	Tue, 10 Apr 2012 23:12:53 +0000 (UTC)
Received: by onelab2.iet.unipi.it (Postfix, from userid 275)
	id 534117300A; Wed, 11 Apr 2012 01:32:11 +0200 (CEST)
Date: Wed, 11 Apr 2012 01:32:11 +0200
From: Luigi Rizzo <rizzo@iet.unipi.it>
To: Barney Wolff <barney@databus.com>
Message-ID: <20120410233211.GA53829@onelab2.iet.unipi.it>
References: <20120410225257.GB53350@onelab2.iet.unipi.it>
	<4F84B6DB.5040904@freebsd.org>
	<20120410230500.GA22829@pit.databus.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20120410230500.GA22829@pit.databus.com>
User-Agent: Mutt/1.4.2.3i
Cc: current@freebsd.org, net@freebsd.org
Subject: Re: strange ping response times...
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Apr 2012 23:12:53 -0000

On Tue, Apr 10, 2012 at 07:05:00PM -0400, Barney Wolff wrote:
> CPU cache?
> Cx states?
> powerd?

powerd is disabled, and i am going down to C1 at most
	> sysctl -a | grep cx
	hw.acpi.cpu.cx_lowest: C1
	dev.cpu.0.cx_supported: C1/1 C2/80 C3/104

which shouldn't take so much. Sure, cache matters, but the
fact is, icmp processing on loopback should occur inline.

unless there is a forced descheduling on a select with timeout > 0
which would explain the extra few microseconds (and makes me worry
on how expensive is a scheduling decision...)

cheers
luigi

> On Tue, Apr 10, 2012 at 03:40:27PM -0700, Julian Elischer wrote:
> > On 4/10/12 3:52 PM, Luigi Rizzo wrote:
> > > I noticed this first on a 10G interface, but now there seems
> > > to be a similar issue on the loopback.
> > >
> > > Apparently a ping -f has a much lower RTT than one with non-zero
> > > delay between transmissions. Part of the story could be that
> > > the flood version invokes a non-blocking select.
> > > On the other hand, pinging on the loopback should make
> > > the response available right away, so what could be the reason
> > > for the additional 3..10us in the ping response time ?
> > >
> > > The following are numbers on an i7-2600k at 3400 MHz + turboboost,
> > > running stable/9 amd64. Note how the min ping time significantly
> > > increases moving from flood to 10ms to 1s.
> > > On an Intel 10G interface i am seeing a min of 14-16us with
> > > a ping flood, and up to 33-35us with the standard 1s interval
> > > (using -q probably trims another 2..5us)
> > 
> > I'd suggest some ktr points around the loopback path..