From owner-freebsd-performance@FreeBSD.ORG Wed Jun 11 18:03:50 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 91B7F37B401 for ; Wed, 11 Jun 2003 18:03:50 -0700 (PDT) Received: from perrin.int.nxad.com (internal.ext.nxad.com [69.1.70.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id 024A543FD7 for ; Wed, 11 Jun 2003 18:03:50 -0700 (PDT) (envelope-from hmp@nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1072) id 6EFA520F01; Wed, 11 Jun 2003 18:03:49 -0700 (PDT) Date: Wed, 11 Jun 2003 18:03:49 -0700 From: Hiten Pandya To: Terry Lambert Message-ID: <20030612010349.GA23018@perrin.int.nxad.com> References: <20030609211526.58641.qmail@web14908.mail.yahoo.com> <3EE4FAED.6090603@centtech.com> <3EE595D2.B223CA19@mindspring.com> <3EE5F8DE.30001@centtech.com> <3EE7021E.F2928B7@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3EE7021E.F2928B7@mindspring.com> X-Operating-System: FreeBSD FreeBSD 4.7-STABLE User-Agent: Mutt/1.5.4i cc: freebsd-performance@freebsd.org cc: Eric Anderson Subject: Re: Slow disk write speeds over network X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2003 01:03:50 -0000 On Wed, Jun 11, 2003 at 03:19:10AM -0700, Terry Lambert wrote: > > Ok - done.. some where defaults, and I couldn't find net.isr.enable.. > > Did I need to config something on my kernel for it to show up? > > You have to set a compile option; look in /usr/src/sys/net; grep > for "netisr_dispatch" or just "dispatch". Terry, ``NetISR Dispatch'' and friends are not available in FreeBSD 4.x versions. I am sure that he is using 4.x and that's why he can't find it in his source. > The net.isr.enable=1 will save you about 100ms per packet, > minimum, and more if you have a high interrupt overhead that > livelocks you from running NETISR. What it does is turns on > direct processing by IP and TCP of packets as they come in > the interface and you get the interrupt. Combined with soft > interrupt coelescing and polling, they should give you > another 1/3 of the receiver livelock fixup. The final third > isn't available, unless you are willing to hack network stack > code and scheduler code, since FreeBSD doesn't include LRP or > Weighted Fair Share Queuing. Does it help if queueing discipline is changed? > > which should be good enough cards to support it.. I've also put 100Mbit > > cards in place of the gige's for now to make sure I wasn't hitting a > > GigE problem or negotiation problem.. > > You should grep for DEVICE_POLLING in the network device > drivers you are interested in using to see if they have the > support. Also you can get up to 15% by adding soft interrupt > coelescing code, if the driver doesn't already support it (I > added it for a couple of drivers, and it was committed after > the benchmarks showed it was good, but it's not everywhere); > the basic idea is you take the interrupt, run rx_eof(), and > call ether_input(). Then repeat the process until you hit > some count limit, or until there's no more data. The direct > dispatch (net.isr.enable) combined with that will process most > packet trains to completion at interrupt, saving you 10ms up > and 10ms back down per packet exchange (NETISR only runs on > exit from spl or at the HZ time, which is default every 10ms). Hmm, can you point these drivers to me? Cheers. -- Hiten (hmp@FreeBSD.ORG)