From owner-freebsd-current@FreeBSD.ORG Tue Mar 9 13:32:16 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B62DC1065670 for ; Tue, 9 Mar 2010 13:32:16 +0000 (UTC) (envelope-from ianf@clue.co.za) Received: from inbound01.jnb1.gp-online.net (inbound01.jnb1.gp-online.net [41.161.16.135]) by mx1.freebsd.org (Postfix) with ESMTP id 0FD1C8FC24 for ; Tue, 9 Mar 2010 13:32:11 +0000 (UTC) Received: from [41.154.88.19] (helo=clue.co.za) by inbound01.jnb1.gp-online.net with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.63) (envelope-from ) id 1NozXR-00074z-AL; Tue, 09 Mar 2010 15:31:57 +0200 Received: from localhost ([127.0.0.1] helo=clue.co.za) by clue.co.za with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NozXP-000GGH-T9; Tue, 09 Mar 2010 15:31:55 +0200 To: pyunyh@gmail.com From: Ian FREISLICH In-Reply-To: <20100308174949.GA1311@michelle.cdnetworks.com> References: <20100308174949.GA1311@michelle.cdnetworks.com> <20100305215539.GG14818@michelle.cdnetworks.com> <20100305210435.GF14818@michelle.cdnetworks.com> <20100305184046.GD14818@michelle.cdnetworks.com> <20100305175639.GB14818@michelle.cdnetworks.com> X-Attribution: BOFH Date: Tue, 09 Mar 2010 15:31:55 +0200 Message-Id: Cc: current@freebsd.org Subject: Re: dev.bce.X.com_no_buffers increasing and packet loss X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 13:32:16 -0000 Pyun YongHyeon wrote: > On Mon, Mar 08, 2010 at 04:45:20PM +0200, Ian FREISLICH wrote: > > Pyun YongHyeon wrote: > > > On Fri, Mar 05, 2010 at 11:16:41PM +0200, Ian FREISLICH wrote: > > > > Pyun YongHyeon wrote: > > > > > Thanks for the info. Frankly, I have no idea how to explain the > > > > > issue given that you have no heavy load. > > > > > > > > How many cores would be involved in handling the traffic and runnig > > > > PF rules on this machine? There are 4x > > > > CPU: Quad-Core AMD Opteron(tm) Processor 8354 (2194.51-MHz K8-class CPU ) > > > > In this server. I'm also using carp extensively. > > > > > > > > > > pf(4) uses a single lock for processing, number of core would have > > > no much benefit. > > > > What's interesting is the effect on CPU utilisation and interrupt > > generation that net.inet.ip.fastforwarding has: > > > > net.inet.ip.fastforwarding=1 > > interrupt rate is around 10000/s per bce interface > > cpu 8.0% interrupt > > > > Yes, this is one of intentional change of the patch. Stock bce(4) > seems to generate too much interrupts on BCM5709 so I rewrote > interrupt handling with the help of David. sysctl nodes are also > exported to control interrupt moderation so you can change them if > you want. Default value was tuned to generate interrupts less than > 10k per second and try to minimize latencies. Can you explain the tunables please - I'm guessing it's these: dev.bce.$i.tx_quick_cons_trip_int dev.bce.$i.tx_quick_cons_trip dev.bce.$i.tx_ticks_int dev.bce.$i.tx_ticks dev.bce.$i.rx_quick_cons_trip_int dev.bce.$i.rx_quick_cons_trip dev.bce.$i.rx_ticks_int dev.bce.$i.rx_ticks -- Ian Freislich