From owner-freebsd-performance@FreeBSD.ORG Wed Aug 19 11:24:14 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 566311065690 for ; Wed, 19 Aug 2009 11:24:14 +0000 (UTC) (envelope-from fabrizio.invernizzi@telecomitalia.it) Received: from GRFEDG702BA020.telecomitalia.it (grfedg702ba020.telecomitalia.it [156.54.233.201]) by mx1.freebsd.org (Postfix) with ESMTP id D81988FC6D for ; Wed, 19 Aug 2009 11:24:13 +0000 (UTC) Received: from GRFHUB702BA020.griffon.local (10.188.101.112) by GRFEDG702BA020.telecomitalia.it (10.188.45.101) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 19 Aug 2009 13:24:12 +0200 Received: from GRFMBX702BA020.griffon.local ([10.188.101.12]) by GRFHUB702BA020.griffon.local ([10.188.101.112]) with mapi; Wed, 19 Aug 2009 13:24:12 +0200 From: Invernizzi Fabrizio To: Gergely CZUCZY Date: Wed, 19 Aug 2009 13:24:09 +0200 Thread-Topic: Strange CPU distributionat very high level bandwidth Thread-Index: Acogu9Nt7FwB6np+SGWf+gFU3I3n+wAA2Rnw Message-ID: <36A93B31228D3B49B691AD31652BCAE9A456967AFF@GRFMBX702BA020.griffon.local> References: <36A93B31228D3B49B691AD31652BCAE9A456967AF4@GRFMBX702BA020.griffon.local> <20090819125707.0000396e@unknown> In-Reply-To: <20090819125707.0000396e@unknown> Accept-Language: it-IT Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: it-IT Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-performance@freebsd.org" Subject: RE: Strange CPU distributionat very high level bandwidth X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2009 11:24:14 -0000 There is an error in my previous email: the upper limit I see is 5722 Mbps = (total). I am sending 239700 pps per interface, 1492-bytes long. Fabrizio > -----Original Message----- > From: Gergely CZUCZY [mailto:phoemix@harmless.hu] > Sent: mercoled=EC 19 agosto 2009 12.57 > To: Invernizzi Fabrizio > Cc: freebsd-performance@freebsd.org > Subject: Re: Strange CPU distributionat very high level bandwidth > > Hello, > > Just a question. May I ask how many pps is this traffic > (packet per second). Forward performance actually depends on > the pps rate and not on the bandwidth usage as far as my > experience goes. As I've calculated by your given data, it > might be around 166Kpps, but i might be wrong there. > > On Wed, 19 Aug 2009 12:13:37 +0200 > Invernizzi Fabrizio wrote: > > > Hi all > > > > i am going on with some performance tests on a 10gbe > network card with > > FreeBSD. > > > > I am doing this test: I send UDP traffic to be forwarded to > the other > > port of the card on both the card ports. Using 1492-long > packets i am > > uppering the number of packets per second i sent In order > to see wich > > is the maximum bandwidth (or pps) the system can support without > > losses. > > > > The limit seems to be about 1890Mbps per port (3870 Mbps total). > > Looking more in deep the CPU behaviour i see this : > > - uppering the sent pps results in uppering the intterrupt time > > (about 90%) > > - when i am very strict to the limit, interrupt time > falls to about > > 10% and CPU is always (85%) in system (rx/tx driver procedure) > > > > Questions: > > - Is not the AIM intended to contrast this behaviour to limit > > interrupts sent to CPU? (nothing changes if i disable it) > > - Why does the system start loosing pkts in that condition? > > - Why does the system seem to perform better when it is > managing more > > context switches? > > > > > > > > These are my system details: > > > > - HP 380 G5 (XEON X5420, CPU speed: 2.50GHz, BUS speed: > 1333 MHz, L2 > > cache size: 12 MB, L2 cache speed: 2,5 GHz) with 1 quad-core > > installed. > > > > - Network card: Silicom PE10G2i-LR - Dual Port Fiber (LR) > 10 Gigabit > > Ethernet PCI Express Server Adapter Intel(r) based (chip 82598EB). > > > > - FreeBSD 7.2-RELEASE (64 bit) > > > > Driver ixgbe-1.8.6 > > > > hw.intr_storm_threshold:2000000 > > > > dev.ix.0.low_latency: 128 > > dev.ix.0.ave_latency: 400 > > dev.ix.0.bulk_latency: 1200 > > dev.ix.1.low_latency: 128 > > dev.ix.1.ave_latency: 400 > > dev.ix.1.bulk_latency: 1200 > > > > ------------------------------------------------------------------ > > Telecom Italia > > Fabrizio INVERNIZZI > > Technology - TILAB > > Accesso Fisso e Trasporto > > Via Reiss Romoli, 274 10148 Torino > > Tel. +39 011 2285497 > > Mob. +39 3316001344 > > Fax +39 06 41867287 > > > > Questo messaggio e i suoi allegati sono indirizzati esclusivamente > > alle persone indicate. La diffusione, copia o qualsiasi > altra azione > > derivante dalla conoscenza di queste informazioni sono > rigorosamente > > vietate. Qualora abbiate ricevuto questo documento per errore siete > > cortesemente pregati di darne immediata comunicazione al > mittente e di > > provvedere alla sua distruzione, Grazie. > > > > This e-mail and any attachments is confidential and may contain > > privileged information intended for the addressee(s) only. > > Dissemination, copying, printing or use by anybody else is > > unauthorised. If you are not the intended recipient, please delete > > this message and any attachments and advise the sender by return > > e-mail, Thanks. > > > > [cid:00000000000000000000000000000001@TI.Disclaimer]Rispetta > > l'ambiente. Non stampare questa mail se non ? necessario. > > > > > > > > -- > Sincerely, > Gergely CZUCZY > Harmless Digital Bt > > +36-30-9702963 > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dall= a conoscenza di queste informazioni sono rigorosamente vietate. Qualora abb= iate ricevuto questo documento per errore siete cortesemente pregati di dar= ne immediata comunicazione al mittente e di provvedere alla sua distruzione= , Grazie. This e-mail and any attachments is confidential and may contain privileged = information intended for the addressee(s) only. Dissemination, copying, pri= nting or use by anybody else is unauthorised. If you are not the intended r= ecipient, please delete this message and any attachments and advise the sen= der by return e-mail, Thanks.