From owner-freebsd-net@FreeBSD.ORG Sat Sep 8 15:30:01 2007 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5530016A417 for ; Sat, 8 Sep 2007 15:30:01 +0000 (UTC) (envelope-from daffy@xview.net) Received: from mail.oav.net (mail.oav.net [193.218.105.18]) by mx1.freebsd.org (Postfix) with ESMTP id 00EFF13C468 for ; Sat, 8 Sep 2007 15:30:00 +0000 (UTC) (envelope-from daffy@xview.net) Received: from localhost (localhost [127.0.0.1]) by mail02.oav.net (Postfix) with ESMTP id 08C563F441; Sat, 8 Sep 2007 17:19:44 +0200 (CEST) (envelope-from daffy@xview.net) X-Virus-Scanned: by amavisd-new at mail02.oav.net Received: from mail02.oav.net ([127.0.0.1]) by localhost (mail02.oav.net [127.0.0.1]) (amavisd-new, port 10026) with LMTP id HxLssib72-MY; Sat, 8 Sep 2007 17:19:43 +0200 (CEST) Received: from [192.168.0.10] (ble59-1-82-66-131-119.fbx.proxad.net [82.66.131.119]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail02.oav.net (Postfix) with ESMTP id 4320D3F443; Sat, 8 Sep 2007 17:19:43 +0200 (CEST) (envelope-from daffy@xview.net) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=ISO-8859-1; delsp=yes; format=flowed Message-Id: <89A69741-1961-431D-A102-4AC89411A40B@xview.net> Content-Transfer-Encoding: quoted-printable From: Olivier Warin Date: Sat, 8 Sep 2007 17:17:56 +0200 To: Fabien THOMAS X-Mailer: Apple Mail (2.752.3) Cc: freebsd-net@freebsd.org, Suphathorn RAMABUTR , Luigi Rizzo Subject: Re: new version of polling for FreeBSD 6.x X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2007 15:30:01 -0000 Le 6 sept. 07 =E0 15:12, Fabien THOMAS a =E9crit : > Hi, > > After many years of good services we will stop using FreeBSD 4.x :) > During my performance regression tests under FreeBSD 6.2 i've found =20= > that polling has lower performance than interrupt. > To solve that issue i've rewritten the core of polling to be more =20 > SMP ready. > > You can find a summary of all my tests and the source code at the =20 > following address: > http://www.netasq.com/opensource/pollng-rev1-freebsd.tgz > > Feel free to ask more detailed information if necessary and report =20 > any bugs / comments. Hello Fabien, 1- I have noticed you are not using GENERIC config file, can you =20 provide us more information on how your KERNCONF differs from =20 GENERIC ? I am pretty sure you have removed all the debug OPTIONs =20 from the kernel, isn't it ? 2- Did you get a chance to try Jumbo Frames and evaluate jumbo mbufs =20 (I never success to make them work for me, did someone had more =20 chance ?). In any cases, PPS values are important for such tests. Andre is right, with Fast Forward you get the best perfs for such test. 3- Did you monitor the DUT to see where CPU cycles were spend in each =20= test ? 4- Have you considered measuring the time it takes for an interrupt =20 to be handled and processed by the kernel Bottom/Top Half and ISR ? [1] 5- When I have performed some test using a Spirent SmartBits =20 (SmartFlow) last summer I got the following results [2]. (For =20 comparison purposes) 6- In the test with Spirent Avalanche, you are using Lighttpd as =20 webserver, did you enable kqueue ? how many workers ? You are using HTTP 1.0 wo Keep-Alive, what was your net.inet.tcp.msl =20 MIB's value ? 7- Polling is known to introduce higher latency, I would expect its =20 benefits to be less in 7-CURRENT compared to 6.x since (Scott ?) a =20 FAST INTR Handler has been introduced around a year ago. Nonetheless, =20= what you report sounds like a perf regression... Have you filled a =20 PR ? Luigi might have a good explaination here. :-D 8- Lock profiling information were obtanied through KTR ? 9- I was wondering if you have explored Intr CPU affinity [3] and IRQ =20= Swizzling [4] ? Thanks for your efforts and your valuable contribution, best regards, /Olivier [1] http://xview.net/papers/interrupt_latency_scheduler/ (WARNING: =20 Never find the time to finish that doc and publish it) && http://xview.net/research/RTBench/ [2] http://xview.net/papers/jumbo_mbufs_freebsd/ [3] http://www.intel.com/cd/ids/developer/asmo-na/eng/188935.htm?prn=3DY [4] http://download.intel.com/design/chipsets/applnots/31433702.pdf -- Olivier Warin