From owner-freebsd-net@FreeBSD.ORG Tue Jul 1 03:10:25 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 872901065672 for ; Tue, 1 Jul 2008 03:10:25 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from atlas.gtcomm.net (atlas.gtcomm.net [67.215.15.242]) by mx1.freebsd.org (Postfix) with ESMTP id 409B48FC1C for ; Tue, 1 Jul 2008 03:10:25 +0000 (UTC) (envelope-from paul@gtcomm.net) Received: from c-76-108-179-28.hsd1.fl.comcast.net ([76.108.179.28] helo=[192.168.1.6]) by atlas.gtcomm.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.67) (envelope-from ) id 1KDWCl-0000mb-7x; Mon, 30 Jun 2008 23:06:55 -0400 Message-ID: <4869A099.5070206@gtcomm.net> Date: Mon, 30 Jun 2008 23:12:25 -0400 From: Paul User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: Sepherosa Ziehau References: <4867420D.7090406@gtcomm.net> <200806301944.m5UJifJD081781@lava.sentex.ca> <20080701004346.GA3898@stlux503.dsto.defence.gov.au> <20080701010716.GF3898@stlux503.dsto.defence.gov.au> <486986D9.3000607@monkeybrains.net> <48699960.9070100@gtcomm.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Ingo Flaschberger , "Wilkinson, Alex" , "Support \(Rudy\)" Subject: Re: Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2008 03:10:25 -0000 I have been unable to even come close to livelocking the machine with the em driver interrupt moderation. So that to me throws polling out the window. I tried 8000hz with polling modified to allow 10000 burst and it makes no difference in the amount of pps I can jam through.. It' seems to be limited by the routing path in the kernel more than anything else. If a driver/hardware didn't support interrupt mitigation then it would definitely lock the machine. Sepherosa Ziehau wrote: > On 7/1/08, Paul wrote: > >> All the NIC drivers in 7 pretty much use interrupt moderation so it can >> > > I am not quite sure whether em(4)'s RX interrupt moderation works as > expected or not. But, AFAIK, nfe(4) and re(4) does not have RX > interrupt moderation. Their TX interrupt moderation could be mimiced > by using their hardware timer and disabling their TX interrupt. > > The lacking of RX im is difficult to handle, I could imagine following way: > - During init, enable RX intr > - When RX intr comes, disable RX intr and set up hardware timer intr > - When timer intr comes and no RX happens, disable timer intr and enable RX intr > > Properly configured #RX desc and timer intr interval will be required > to make sure that the RX desc collection could keep up with the > hardware speed. I used pure timer intr (8000Hz) on nfe(4) in dfly w/ > good result, i.e. TX/RX @linespeed without livelocking the system. > The drawback of pure timer intr is that you waste extra cpu power, > when there is nothing to process. > > >> never lock the machine anyway.. This effectively kills polling and it really >> no longer has any use except to be able to have a fraction of the cpu set >> > > Oh? Really? :] > > Best Regards, > sephe > >