From owner-freebsd-current@FreeBSD.ORG Thu Nov 18 12:33:54 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2B28216A4CE; Thu, 18 Nov 2004 12:33:54 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B736243D31; Thu, 18 Nov 2004 12:33:53 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id iAICWMup069220; Thu, 18 Nov 2004 07:32:22 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)iAICWLPl069217; Thu, 18 Nov 2004 12:32:22 GMT (envelope-from robert@fledge.watson.org) Date: Thu, 18 Nov 2004 12:32:21 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Pawel Jakub Dawidek In-Reply-To: <20041118094257.GB8120@darkness.comp.waw.pl> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Emanuel Strobl cc: freebsd-current@FreeBSD.org cc: freebsd-stable@FreeBSD.org Subject: Re: serious networking (em) performance (ggate and NFS) problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 12:33:54 -0000 On Thu, 18 Nov 2004, Pawel Jakub Dawidek wrote: > On Wed, Nov 17, 2004 at 11:57:41PM +0100, Emanuel Strobl wrote: > +> Dear best guys, > +> > +> I really love 5.3 in many ways but here're some unbelievable transfer rates, > +> after I went out and bought a pair of Intel GigaBit Ethernet Cards to solve > +> my performance problem (*laugh*): > [...] > > I done some test in the past with ggate and PCI64/GBit NICs and I get > ~38MB/s AFAIR. > > Remember that when using 32bit PCIs you can get transfer about > 500Mbit/s. > > Please run those test with netperf (/usr/ports/benchmarks/netperf) and > send results. Be aware, btw, that while netperf is a pretty decent tool, it performs a lot of socket select and timing operations, so isn't always a good measure of maximum capabilities of a system. I.e., it is not unusual to see that a netperf send test will only see send() as one in three or one in four system calls -- as a result, it uses a measurable amount of CPU on things other than sending. In an environment with CPU constraints (slower CPU or faster network), this can impact the performance results substantially. For example, when measuring maximum packet send performance using minimal packet sizes from user space, several of my test boxes are constrained with 64-bit gig-e PCI cards based on CPU. In particular, the combined cost of the additional system calls and operations cuts into available CPU for send. By eliminating the misc. overheads of netperf using netsend, I can improve performance by 20%-30%. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research