From owner-freebsd-performance@FreeBSD.ORG Sat Mar 1 01:03:19 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF269106567D for ; Sat, 1 Mar 2008 01:03:19 +0000 (UTC) (envelope-from if@xip.at) Received: from chile.gbit.at (ns1.xip.at [193.239.188.99]) by mx1.freebsd.org (Postfix) with ESMTP id 457BC8FC19 for ; Sat, 1 Mar 2008 01:03:18 +0000 (UTC) (envelope-from if@xip.at) Received: (qmail 21564 invoked from network); 1 Mar 2008 02:03:16 +0100 Received: from unknown (HELO filebunker.xip.at) (86.59.10.180) by chile.gbit.at with (DHE-RSA-AES256-SHA encrypted) SMTP; 1 Mar 2008 02:03:16 +0100 Date: Sat, 1 Mar 2008 02:03:16 +0100 (CET) From: Ingo Flaschberger To: Willem Jan Withagen In-Reply-To: <47C8964C.9080309@digiware.nl> Message-ID: References: <20080226003107.54CD94500E@ptavv.es.net> <47C8964C.9080309@digiware.nl> User-Agent: Alpine 1.00 (LFD 882 2007-12-20) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Mailman-Approved-At: Sat, 01 Mar 2008 01:10:50 +0000 Cc: alves , freebsd-net@freebsd.org, freebsd-performance@freebsd.org, =?ISO-8859-1?Q?Daniel_Dias_Gon=E7?=@FreeBSD.ORG, Kevin Oberman Subject: Re: FBSD 1GBit router? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Mar 2008 01:03:20 -0000 >> I have a 1.2Ghz Pentium-M appliance, with 4x 32bit, 33MHz pci intel e1000 >> cards. >> With maximum tuning I can "route" ~400mbps with big packets and ~80mbps >> with 64byte packets. >> around 100kpps, whats not bad for a pci architecture. >> >> To reach higher bandwiths, better busses are needed. >> pci-express cards are currently the best choice. >> one dedicated pci-express lane (1.25gbps) has more bandwith than a whole >> 32bit, 33mhz pci-bus. > > Like you say routing 400 Mb/s is close to the max of the PCI bus, which > has a theoretical max of 33*4*8 ~ 1Gbps. Now routing is 500Mb/s in, 500Mb/s > out. So you are within 80% of the bus-max, not counting memory-access and > others. yes. > PCI express will give you a bus per PCI-E device into a central hub, thus > upping the limit to the speed of the FrontSideBus in Intel architectures. > Which at the moment is a lot higher than what a single PCI bus does. Thats why my next router will be based at this box: http://www.axiomtek.com/products/ViewProduct.asp?view=429 Hopefully there will be direct memory bus connected nic's in future. (HyperTransport connected nic's) > What it does not explain is why you can only get 80Mb/s with 64byte packets, > which would suggest other bottlenecks than just the bus. Perhaps something with interrupts: http://books.google.at/books?id=pr4fspaQqZkC&pg=PA144&lpg=PA144&dq=pci+interrupt+delay&source=web&ots=zbvVU2CgVx&sig=APe9YjdtK35ccnow7BDI2hzie7s&hl=de#PPA144,M1 MSI (Message-signalled Interrupts) are not very common on PCI architekture; PCI-E use only MSI. The kpps keept always around 100, equally if I used fast-forwarding, fast-interrupts, or higher HZ values than 1000HZ. But 100kpps is great for a router hardware of about 600eur. Kind regards, Ingo Flaschberger