From owner-freebsd-net@FreeBSD.ORG Sat Nov 17 07:08:28 2007 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2185616A46C for ; Sat, 17 Nov 2007 07:08:28 +0000 (UTC) (envelope-from is@rambler-co.ru) Received: from relay0.rambler.ru (relay0.rambler.ru [81.19.66.187]) by mx1.freebsd.org (Postfix) with ESMTP id CFD0713C45D for ; Sat, 17 Nov 2007 07:08:26 +0000 (UTC) (envelope-from is@rambler-co.ru) Received: from relay0.rambler.ru (localhost [127.0.0.1]) by relay0.rambler.ru (Postfix) with ESMTP id 56E976FDF; Sat, 17 Nov 2007 10:08:19 +0300 (MSK) Received: from localhost (is1.park.rambler.ru [81.19.64.121]) by relay0.rambler.ru (Postfix) with ESMTP id 3447B6FDC; Sat, 17 Nov 2007 10:08:19 +0300 (MSK) Date: Sat, 17 Nov 2007 10:10:53 +0300 From: Igor Sysoev To: Bruce Evans Message-ID: <20071117071053.GA18091@rambler-co.ru> References: <20071116154019.GE93422@rambler-co.ru> <20071117065908.T65479@delplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20071117065908.T65479@delplex.bde.org> User-Agent: Mutt/1.5.13 (2006-08-11) X-Virus-Scanned: No virus found Cc: freebsd-net@FreeBSD.org Subject: Re: bge loader tunables X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Nov 2007 07:08:28 -0000 On Sat, Nov 17, 2007 at 08:30:58AM +1100, Bruce Evans wrote: > On Fri, 16 Nov 2007, Igor Sysoev wrote: > > >The attached patch creates the following bge loader tunables: > > I plan to commit old work to do this using sysctls. Tunables are > harder to use and aren't needed since changes to the defaults aren't > needed for booting. I also implemented dynamic tuning for rx coal > parameters so that the sysctls are mostly not needed. Ask for patches > if you want to test this extensively. Yes, I can test your patches on 6.2 and 7.0. Now bge set the coalescing parameters at attach time. Do the sysctl's allow to change them on-the-fly ? How does rx dynamic tuning work ? Could it be turned off ? > >hw.bge.rxd=512 > > > >Number of standard receive descriptors allocated by the driver. > >The default value is 256. The maximum value is 512. > > I always use 512 for this. The corresponding value for jumbo buffers > is hard-coded (JSLOTS exists to tune the value at config time, like > SSLOTS does for this, but is no longer used). Only machines with a > small amount of memory should care about the wastage from always > allocating the max number of descriptors. I agree: the default jumbo rx ring takes 256*9216=2.3M, while maximum standard rx ring takes 512*2048=1M, nevertheless it is limited to 256*2048=512K. > >hw.bge.rx_int_delay=500 > > > >This value delays the generation of receive interrupts in microseconds. > >The default value is 150 microseconds. > > This is a good default. I normally use 100 (goes with dynamic tuning to > limit the rx interrupt rate to 10 kHz). > > >hw.bge.tx_int_delay=500 > > > >This value delays the generation of transmit interrupts in microseconds. > >The default value is 150 microseconds. > > I use 1 second. Infinity works right, except it wastes mbufs when the > tx is idle for a long time. It seems 1 second is good for me: I use sendfile() and lot of mbufs clusters: kern.ipc.nmbclusters=196608 > >hw.bge.rx_coal_desc=64 > > > >This value delays the generation of receive interrupts until specified > >number of packets will be received. The default value is 10. > > 64 is a good default. 10 is a bad default (it optimizes too much for > latency at a cost of efficiency to be good). I use 1 when optimizing > for latency. Dynamic tuning sets this to a value suitable for limiting > the rx interrupt rate to a specified frequency (10 kHz is a good limit). > > >hw.bge.tx_coal_desc=128 > > > >This value delays the generation of transmit interrupts until specified > >number of packets will be transmited. The default value is 10. > > 128 is a good default. I use 384. There are few latency issues here, so > the default of 10 mainly costs efficiency. Does 384 not delay tx if there is shortage of free tx descriptors ? -- Igor Sysoev http://sysoev.ru/en/