From owner-freebsd-current@FreeBSD.ORG Mon Nov 5 18:28:27 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1B4E99F2 for ; Mon, 5 Nov 2012 18:28:27 +0000 (UTC) (envelope-from lists@rewt.org.uk) Received: from abby.lhr1.as41113.net (abby.lhr1.as41113.net [91.208.177.20]) by mx1.freebsd.org (Postfix) with ESMTP id C7E188FC12 for ; Mon, 5 Nov 2012 18:28:26 +0000 (UTC) Received: from [172.16.11.21] (bella.stf.rewt.org.uk [91.208.177.62]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by abby.lhr1.as41113.net (Postfix) with ESMTPS id 3XwMpK0xZmz13P1; Mon, 5 Nov 2012 18:28:25 +0000 (GMT) Message-ID: <50980544.4000507@rewt.org.uk> Date: Mon, 05 Nov 2012 18:28:20 +0000 From: Joe Holden User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: Dynamic Ticks/HZ References: <509758B8.1000409@rewt.org.uk> <50975F6F.6010907@rewt.org.uk> <5097898C.9080109@rewt.org.uk> <20121105163654.GA12870@onelab2.iet.unipi.it> <5097E880.8010001@rewt.org.uk> <20121105165748.GA13098@onelab2.iet.unipi.it> <509803A2.7090302@rewt.org.uk> In-Reply-To: <509803A2.7090302@rewt.org.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2012 18:28:27 -0000 Joe Holden wrote: > Luigi Rizzo wrote: >> On Mon, Nov 05, 2012 at 04:25:36PM +0000, Joe Holden wrote: >>> Luigi Rizzo wrote: >>>> On Mon, Nov 05, 2012 at 08:11:41AM -0500, Ryan Stone wrote: >>>>> On Mon, Nov 5, 2012 at 4:40 AM, Joe Holden wrote: >>>>> >>>>>> doh, running kernel wasn't as GENERIC as I thought it was, looks like >>>>>> device polling not only breaks dynamic ticks but also reduces rx >>>>>> ability >>>>>> significantly, exactly 150,000 pps per 1000hz on igb versus >>>>>> 650,000 without >>>>>> >>>>>> Is this a known issue? (and if device polling isn't as useful as >>>>>> it once >>>>>> was, should it be removed?) >>>>>> >>>>> Device polling on modern multiqueue NICs isn't very useful because >>>>> you're >>>>> limited to a single thread for handling packets. I have a patch >>>>> that fixes >>>>> this that I've let fall by the wayside. >>>> the 150,000 is result of the combination of the default value of >>>> sysctl kern.polling.burst_max and kern.polling.idle_poll=0 >>>> (i think this is the default value for the latter). >>>> >>>> The 150 was sized for the peak pps on a 100Mbit/s interface, >>>> back in 2001. You should at least be able to raise the number >>>> and see what kind of throughput you can achieve. >>>> >>>> This said, modern nics also have interrupt moderation so you >>>> don't really need polling. >>>> >>>> cheers >>>> luigi >>> Hi Luigi, >>> >>> This makes sense, am I likely to achieve better throughput (in the >>> forwarding path at this point) with netisr rather than polling, >>> especially as mentioned above the igb does indeed have multiple >>> queues for rx? >> >> at 1Gbit/s you probably don't need multiqueue (I am actually surpised >> you can only do 650kpps, but perhaps because you are using ipfw and >> not just doing plain forwarding ?) >> >>> On another note, is netmap usable in the forwarding context at all as >>> it is rather awesome >> >> It depends on what you need to do. If you have a v4/v6 router you >> won't see any advantage (at the moment; there is some work in the >> pipeline but probably it won't be available before spring). >> >> If you just need to implement a firewall to protect the internal >> network then it is another story and you can use the ipfw on netmap >> that I posted in august. >> >> cheers >> luigi > Hi, > > I have a setup where box1 is connected directly to box2 (igb0), with > pkt-gen however box2 doesn't seem to actually forward any packets, > sysctl -ip 1 shows 1.48Mpps unreachables generated (matching the input), > is there something specific I need to do to test forwarding? > > Thanks, > Joe Nevermind, didn't set destination MAC addr, apologies for the noise!