From owner-freebsd-net@FreeBSD.ORG Fri Jan 30 10:45:07 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1835216A4CF for ; Fri, 30 Jan 2004 10:45:07 -0800 (PST) Received: from csmail.commserv.ucsb.edu (cspdc.commserv.ucsb.edu [128.111.251.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D94F43D2F for ; Fri, 30 Jan 2004 10:45:05 -0800 (PST) (envelope-from steve@expertcity.com) Received: from expertcity.com ([68.111.37.3]) by csmail.commserv.ucsb.edu (Netscape Messaging Server 3.62) with ESMTP id 379; Fri, 30 Jan 2004 10:45:04 -0800 Message-ID: <401AA66E.3090708@expertcity.com> Date: Fri, 30 Jan 2004 10:46:06 -0800 From: Steve Francis User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5b) Gecko/20030827 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Luigi Rizzo References: <401AA3A0.7080208@expertcity.com> <20040130103855.A7798@xorpc.icir.org> In-Reply-To: <20040130103855.A7798@xorpc.icir.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-net@freebsd.org Subject: Re: Device polling, kern.polling.burst_max and gig-e X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jan 2004 18:45:07 -0000 Luigi Rizzo wrote: >i would probably increase HZ to 2000 and burst_max to 300-400, >not much more though otherwise you are going to spend too much >time in the timer handler. >In any case, i don't think the card is able to go above 6-700kpps. > > OK, thanks. Each card is only being asked to do (at most ) 250kpps. Two cards with that load in the system. No tuning of kern.polling.each_burst recommended? Thanks >If you are having a lot of load, it is natural that you are >going to get losses, the 2sec period is probably how often the >nic updates the stats. > > cheers > luigi > >On Fri, Jan 30, 2004 at 10:34:08AM -0800, Steve Francis wrote: > > >>We have a 4.9-RELEASE-p1 box dedicated to some traffic analysis. It >>monitors on two em interfaces: about 200,000 pps on one interface, and >>180,000 pps on the other. >>It's been dealing with that OK, but our traffic levels are increasing - >>we reached over 240,000 pps on one interface last week. This made CPU >>reach 100%, and some packets not get processed. >>So, last night we enabled polling on the nics. >>Initially, great result - CPU dropped from 82% load (45% system load due >>to interupts) yesterday to 55% load today (12% in system), for same pps >>load (about 300,000 pps total) at the time. >> >>However, input errors went from 0 to about 1200 (oddly, it was 1200 >>every other second, and 0 for the seconds in-between.) >> >>A bit of digging around led me to increase kern.polling.burst_max. >>According to http://info.iet.unipi.it/~luigi/polling/, "The default >>value is enough for a 100Mbit ethernet". I increased it gradually to >>900, whcih has almost (but not entirely) eliminated the errors. Now the >>errors are zero for most intervals, but every 10 or so intervals there >>are between 10 and 100 input errors. >> >>So: >>- does it make sense to leave the default at 150, in this day of gigabit >>nics? >>- is there a danger in increasing the burt_max? (My burst size goes >>straight to the max of 900.) >>- can it be increased more ? >>- are there other variables that make sense to increase for gigabit? >>(like kern.polling.each_burst:?) >> >>Since I increased the burst max, I now have slowly incrementing >>kern.polling.lost_polls - about 1 every 2 seconds. Anything to worry about? >> >>Thanks >>Steve Francis >> >> >> >> >>_______________________________________________ >>freebsd-net@freebsd.org mailing list >>http://lists.freebsd.org/mailman/listinfo/freebsd-net >>To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >>