From owner-freebsd-performance@FreeBSD.ORG Mon Aug 3 10:27:00 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CD3F1065674 for ; Mon, 3 Aug 2009 10:27:00 +0000 (UTC) (envelope-from fabrizio.invernizzi@telecomitalia.it) Received: from GRFEDG702BA020.telecomitalia.it (grfedg702ba020.telecomitalia.it [156.54.233.201]) by mx1.freebsd.org (Postfix) with ESMTP id E80DF8FC12 for ; Mon, 3 Aug 2009 10:26:59 +0000 (UTC) (envelope-from fabrizio.invernizzi@telecomitalia.it) Received: from GRFHUB701BA020.griffon.local (10.188.101.111) by GRFEDG702BA020.telecomitalia.it (10.188.45.101) with Microsoft SMTP Server (TLS) id 8.1.340.0; Mon, 3 Aug 2009 12:26:58 +0200 Received: from GRFMBX702BA020.griffon.local ([10.188.101.11]) by grfhub701ba020.griffon.local ([10.188.101.111]) with mapi; Mon, 3 Aug 2009 12:26:57 +0200 From: Invernizzi Fabrizio To: Stefan Lambrev Date: Mon, 3 Aug 2009 12:26:55 +0200 Thread-Topic: Test on 10GBE Intel based network card Thread-Index: AcoUI0uqyXnc4/suT/+J7T7yVslx/gAAKPbA Message-ID: <36A93B31228D3B49B691AD31652BCAE9A4560DF96A@GRFMBX702BA020.griffon.local> References: <36A93B31228D3B49B691AD31652BCAE9A4560DF911@GRFMBX702BA020.griffon.local> <0E567C7E-4EAA-4B89-9A8D-FD0450D32ED7@moneybookers.com> <36A93B31228D3B49B691AD31652BCAE9A4560DF947@GRFMBX702BA020.griffon.local> <18AAC16B-3CC0-4C70-A009-00A325AB5932@moneybookers.com> In-Reply-To: <18AAC16B-3CC0-4C70-A009-00A325AB5932@moneybookers.com> Accept-Language: it-IT, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: it-IT, en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-performance@freebsd.org" Subject: RE: Test on 10GBE Intel based network card X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2009 10:27:00 -0000 > > If you are meaning in term of Packet per second, you are right. > > These are the packet per second measured during tests: > > > > 64 byte: 610119 Pps > > 512 byte: 516917 Pps > > 1492 byte: 464962 Pps > > > > > >> Am I correct that the maximum you can reach is around 639,000 packets > >> per second? > > > > Yes, as you can see the maximum is 610119 Pps. > > Where does this limit come from? > > I duno - the tests I did before were with SYN packets (random source) > which was my worst scenario, > and the server CPU were busy generating MD5 check sums for > "syncache" (around 35% of the time). > > If I have to compare my results with your, you beat me with factor > 2.5, may be because you use ICMP for the test > and your processor is better then my test stations :) > Also my experience is only with gigabit cards (em driver) and FreeBSD > 7.something_before_1 where the em thread was eating 100% cpu. > If you are lucky LOCK_PROFILING(9) will help you to see where the CPUs > spend their time, if not you will see kernel panic :) I will check, thanks for the hint. > Once problematic locks identified they can be reworked, but I think > the first part is already done > and work on the second already started. > > In my experience increasing hw.em.rxd and hw.em.txd yelled better > results, but I think ixgb already comes tuned by default > as it still doesn't have to support such a large number of different > cards. I did some tuning in the code of the driver e recompiled the kernel in orde= r to reduce context switching (interrupt mitigation) since the driver does = not support POLLING. > Also at the time of my tests there were not support for multi queues > in the OS even if they were supported by the HW, which is changed in > 7.2 (?) It looks like multi queue is working since I can see the load distributed o= ver the 4 cores. Fabrizio Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle per= sone indicate. La diffusione, copia o qualsiasi altra azione derivante dall= a conoscenza di queste informazioni sono rigorosamente vietate. Qualora abb= iate ricevuto questo documento per errore siete cortesemente pregati di dar= ne immediata comunicazione al mittente e di provvedere alla sua distruzione= , Grazie. This e-mail and any attachments is confidential and may contain privileged = information intended for the addressee(s) only. Dissemination, copying, pri= nting or use by anybody else is unauthorised. If you are not the intended r= ecipient, please delete this message and any attachments and advise the sen= der by return e-mail, Thanks.