From owner-freebsd-performance@FreeBSD.ORG Fri Aug 21 08:15:55 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF862106568E for ; Fri, 21 Aug 2009 08:15:55 +0000 (UTC) (envelope-from fabrizio.invernizzi@telecomitalia.it) Received: from GRFEDG702BA020.telecomitalia.it (grfedg702ba020.telecomitalia.it [156.54.233.201]) by mx1.freebsd.org (Postfix) with ESMTP id 47BFB8FC62 for ; Fri, 21 Aug 2009 08:15:55 +0000 (UTC) Received: from GRFHUB702BA020.griffon.local (10.188.101.112) by GRFEDG702BA020.telecomitalia.it (10.188.45.101) with Microsoft SMTP Server (TLS) id 8.1.340.0; Fri, 21 Aug 2009 10:15:52 +0200 Received: from GRFMBX702BA020.griffon.local ([10.188.101.11]) by GRFHUB702BA020.griffon.local ([10.188.101.112]) with mapi; Fri, 21 Aug 2009 10:15:51 +0200 From: Invernizzi Fabrizio To: Ivan Voras , "freebsd-performance@freebsd.org" Date: Fri, 21 Aug 2009 10:15:46 +0200 Thread-Topic: Strange CPU distributionat very high level bandwidth Thread-Index: Acoh7wlzPrst/QZzTueUdgkiEjVfgAASDhUQ Message-ID: <36A93B31228D3B49B691AD31652BCAE9A45770696C@GRFMBX702BA020.griffon.local> References: <36A93B31228D3B49B691AD31652BCAE9A456967AF4@GRFMBX702BA020.griffon.local> In-Reply-To: Accept-Language: it-IT Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: it-IT Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Subject: RE: Strange CPU distributionat very high level bandwidth X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2009 08:15:55 -0000 Thanks for your suggestion. I hope to have time to do some tests on 8.0 and send some result on the ML= next week. ------------------------------------------------------------------ Telecom Italia Fabrizio INVERNIZZI Technology - TILAB Accesso Fisso e Trasporto Via Reiss Romoli, 274 10148 Torino Tel. +39 011 2285497 Mob. +39 3316001344 Fax +39 06 41867287 > -----Original Message----- > From: owner-freebsd-performance@freebsd.org > [mailto:owner-freebsd-performance@freebsd.org] On Behalf Of Ivan Voras > Sent: venerd=EC 21 agosto 2009 1.14 > To: freebsd-performance@freebsd.org > Subject: Re: Strange CPU distributionat very high level bandwidth > > Invernizzi Fabrizio wrote: > > Hi all > > > > i am going on with some performance tests on a 10gbe > network card with FreeBSD. > > > > I am doing this test: I send UDP traffic to be forwarded to > the other port of the card on both the card ports. > > Using 1492-long packets i am uppering the number of > packets per second i sent In order to see wich is the maximum > bandwidth (or pps) the system can support without losses. > > > > The limit seems to be about 1890Mbps per port (3870 Mbps total). > > Looking more in deep the CPU behaviour i see this : > > - uppering the sent pps results in uppering the > intterrupt time (about 90%) > > - when i am very strict to the limit, interrupt time > falls to about > > 10% and CPU is always (85%) in system (rx/tx driver procedure) > > > > Questions: > > - Is not the AIM intended to contrast this behaviour to limit > > interrupts sent to CPU? (nothing changes if i disable it) > > - Why does the system start loosing pkts in that condition? > > - Why does the system seem to perform better when it is > managing more context switches? > > > > > - FreeBSD 7.2-RELEASE (64 bit) > > One idea for you, not directly tied to forwarding as is but > to the recent development of multithreaded packet acceptance > code, is to use 8.x (currently in BETA so usual precautions > about debugging being enabled apply) and then play with > netisr and worker thread settings. > > See the source here: > > http://svn.freebsd.org/viewvc/base/head/sys/net/netisr.c?view=3D markup&pathrev=3D195078 > > and the comments starting at "Three direct dispatch policies > are supported". > > The code is experimental and thus disabled in 8.0 unless a > combination of the following loader tunables are set: > > net.isr.direct_force > net.isr.direct > net.isr.maxthreads > net.isr.bindthreads > > I think you can start simply by turning off > net.isr.direct_force and then start increasing > net.isr.maxthreads until the benefits (if any) go away. Since > it is experimental code, your benchmarks would be nice to have. > > > > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dall= a conoscenza di queste informazioni sono rigorosamente vietate. Qualora abb= iate ricevuto questo documento per errore siete cortesemente pregati di dar= ne immediata comunicazione al mittente e di provvedere alla sua distruzione= , Grazie. This e-mail and any attachments is confidential and may contain privileged = information intended for the addressee(s) only. Dissemination, copying, pri= nting or use by anybody else is unauthorised. If you are not the intended r= ecipient, please delete this message and any attachments and advise the sen= der by return e-mail, Thanks.