From owner-freebsd-current@FreeBSD.ORG Thu Nov 18 13:02:13 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C5FE16A4CE; Thu, 18 Nov 2004 13:02:13 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6821F43D58; Thu, 18 Nov 2004 13:02:12 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id iAID0dGC069640; Thu, 18 Nov 2004 08:00:39 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)iAID0cLp069637; Thu, 18 Nov 2004 13:00:38 GMT (envelope-from robert@fledge.watson.org) Date: Thu, 18 Nov 2004 13:00:38 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Wilko Bulte In-Reply-To: <20041118124902.GC75559@freebie.xs4all.nl> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Emanuel Strobl cc: freebsd-current@freebsd.org cc: freebsd-stable@freebsd.org Subject: Re: serious networking (em) performance (ggate and NFS) problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Nov 2004 13:02:13 -0000 On Thu, 18 Nov 2004, Wilko Bulte wrote: > On Thu, Nov 18, 2004 at 12:27:44PM +0000, Robert Watson wrote.. > > > > On Wed, 17 Nov 2004, Emanuel Strobl wrote: > > > > > I really love 5.3 in many ways but here're some unbelievable transfer > > > rates, after I went out and bought a pair of Intel GigaBit Ethernet > > > Cards to solve my performance problem (*laugh*): > > > > I think the first thing you want to do is to try and determine whether the > > problem is a link layer problem, network layer problem, or application > > (file sharing) layer problem. Here's where I'd start looking: > > And you definitely want to look at polling(4) He did, but he set the HZ at 256, which is sufficiently low as to guarantee a substantial increase in latency, and likely guarantee interface and socket queue overruns (although I haven't done the math to verify that is the case). Between the very finite sizes of ifnet send queues, socket buffers, and if_em descriptors, and on-board buffers on the card, high latency polling can result in lots of packet loss and delay under load. Hence the recommendation of a relatively high value of HZ so that the queues in the driver are drained regularly, and sends acknowledged so that the sent mbufs can be reclaimed and reused. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research