From owner-freebsd-performance@FreeBSD.ORG Mon Aug 3 16:36:51 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3B36106572F for ; Mon, 3 Aug 2009 16:36:51 +0000 (UTC) (envelope-from raykinsella78@gmail.com) Received: from mail-bw0-f206.google.com (mail-bw0-f206.google.com [209.85.218.206]) by mx1.freebsd.org (Postfix) with ESMTP id 118CD8FC1B for ; Mon, 3 Aug 2009 16:36:50 +0000 (UTC) (envelope-from raykinsella78@gmail.com) Received: by bwz2 with SMTP id 2so2412299bwz.43 for ; Mon, 03 Aug 2009 09:36:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=s3+cfOPR3IJVvXf9lJLws/ILduooUKdAL3far+Tt8nA=; b=paOxsXvpc7MCPS4tVG27EBTE2cUzOtFS+a8pjJ673uVPDUxrA6Gm3i22F7WRvFSncn dMrdLoNO1MAPhKEcUNGdjbxhVaGN+Ugkx82u5/abq1VZgVNXL5g9ous06wtnwwAN8WUO 0SLYtZKFD93JTZ9o/2FN+q6MP0Y32V0XxhHbU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=R+jkAdgow0L2qe3LxIUDEsFeJbWR9wevV8Iw/+h1xCVli2SSf6ooyfjJ05H3kX2TDU wYBaZV3Fr9P8AOSYs5eLi5RSUQ/kHRa+B/hkapWQi0WG5PJTsFdXsIaQgrOG+mwbI1xO 2+hg/Z+4CsVBy7eOp3ilUhBphECbi32IcOWw0= MIME-Version: 1.0 Received: by 10.239.162.205 with SMTP id m13mr655111hbd.87.1249315633619; Mon, 03 Aug 2009 09:07:13 -0700 (PDT) In-Reply-To: <2a41acea0908030820o40438f6erda78927733529a9@mail.gmail.com> References: <36A93B31228D3B49B691AD31652BCAE9A4560DF911@GRFMBX702BA020.griffon.local> <0E567C7E-4EAA-4B89-9A8D-FD0450D32ED7@moneybookers.com> <36A93B31228D3B49B691AD31652BCAE9A4560DF947@GRFMBX702BA020.griffon.local> <18AAC16B-3CC0-4C70-A009-00A325AB5932@moneybookers.com> <36A93B31228D3B49B691AD31652BCAE9A4560DF96A@GRFMBX702BA020.griffon.local> <2a41acea0908030820o40438f6erda78927733529a9@mail.gmail.com> Date: Mon, 3 Aug 2009 17:07:13 +0100 Message-ID: <584ec6bb0908030907i4371d2d1y63fc23bb889ae06d@mail.gmail.com> From: Ray Kinsella To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "freebsd-performance@freebsd.org" , Invernizzi Fabrizio , Stefan Lambrev Subject: Re: Test on 10GBE Intel based network card X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2009 16:36:52 -0000 Hi all, cpuset is the command to set a cpu affinity, there are details @ http://bramp.net/blog/post vmstat -z is the command you need to determine whether there is contention for mbufs. although the cpu usage does not suggest the system is memory constrained. Regards Ray Kinsella On Mon, Aug 3, 2009 at 4:20 PM, Jack Vogel wrote: > If you go to FreeBSD 8 you will get the improved stack code, and the RX/TX > queue pairs > will be pinned to cpus. It should improve performance. > > Make sure you have enough mbuf memory allocated, try increasing the > descriptors. > > Jack > > > On Mon, Aug 3, 2009 at 3:26 AM, Invernizzi Fabrizio < > fabrizio.invernizzi@telecomitalia.it> wrote: > > > > > If you are meaning in term of Packet per second, you are right. > > > > These are the packet per second measured during tests: > > > > > > > > 64 byte: 610119 Pps > > > > 512 byte: 516917 Pps > > > > 1492 byte: 464962 Pps > > > > > > > > > > > >> Am I correct that the maximum you can reach is around 639,000 > packets > > > >> per second? > > > > > > > > Yes, as you can see the maximum is 610119 Pps. > > > > Where does this limit come from? > > > > > > I duno - the tests I did before were with SYN packets (random source) > > > which was my worst scenario, > > > and the server CPU were busy generating MD5 check sums for > > > "syncache" (around 35% of the time). > > > > > > If I have to compare my results with your, you beat me with factor > > > 2.5, may be because you use ICMP for the test > > > and your processor is better then my test stations :) > > > Also my experience is only with gigabit cards (em driver) and FreeBSD > > > 7.something_before_1 where the em thread was eating 100% cpu. > > > If you are lucky LOCK_PROFILING(9) will help you to see where the CPUs > > > spend their time, if not you will see kernel panic :) > > > > > > I will check, thanks for the hint. > > > > > Once problematic locks identified they can be reworked, but I think > > > the first part is already done > > > and work on the second already started. > > > > > > In my experience increasing hw.em.rxd and hw.em.txd yelled better > > > results, but I think ixgb already comes tuned by default > > > as it still doesn't have to support such a large number of different > > > cards. > > > > I did some tuning in the code of the driver e recompiled the kernel in > > order to reduce context switching (interrupt mitigation) since the driver > > does not support POLLING. > > > > > Also at the time of my tests there were not support for multi queues > > > in the OS even if they were supported by the HW, which is changed in > > > 7.2 (?) > > > > It looks like multi queue is working since I can see the load distributed > > over the 4 cores. > > > > > > Fabrizio > > > > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle > > persone indicate. La diffusione, copia o qualsiasi altra azione derivante > > dalla conoscenza di queste informazioni sono rigorosamente vietate. > Qualora > > abbiate ricevuto questo documento per errore siete cortesemente pregati > di > > darne immediata comunicazione al mittente e di provvedere alla sua > > distruzione, Grazie. > > > > This e-mail and any attachments is confidential and may contain > privileged > > information intended for the addressee(s) only. Dissemination, copying, > > printing or use by anybody else is unauthorised. If you are not the > intended > > recipient, please delete this message and any attachments and advise the > > sender by return e-mail, Thanks. > > > > _______________________________________________ > > freebsd-performance@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > > To unsubscribe, send any mail to " > > freebsd-performance-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to " > freebsd-performance-unsubscribe@freebsd.org" >