From owner-freebsd-net@FreeBSD.ORG Fri Jan 30 10:33:09 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3378716A4CE for ; Fri, 30 Jan 2004 10:33:09 -0800 (PST) Received: from csmail.commserv.ucsb.edu (cspdc.commserv.ucsb.edu [128.111.251.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7659C43D48 for ; Fri, 30 Jan 2004 10:33:07 -0800 (PST) (envelope-from steve@expertcity.com) Received: from expertcity.com ([68.111.37.3]) by csmail.commserv.ucsb.edu (Netscape Messaging Server 3.62) with ESMTP id 384; Fri, 30 Jan 2004 10:33:06 -0800 Message-ID: <401AA3A0.7080208@expertcity.com> Date: Fri, 30 Jan 2004 10:34:08 -0800 From: Steve Francis User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5b) Gecko/20030827 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Device polling, kern.polling.burst_max and gig-e X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Jan 2004 18:33:09 -0000 We have a 4.9-RELEASE-p1 box dedicated to some traffic analysis. It monitors on two em interfaces: about 200,000 pps on one interface, and 180,000 pps on the other. It's been dealing with that OK, but our traffic levels are increasing - we reached over 240,000 pps on one interface last week. This made CPU reach 100%, and some packets not get processed. So, last night we enabled polling on the nics. Initially, great result - CPU dropped from 82% load (45% system load due to interupts) yesterday to 55% load today (12% in system), for same pps load (about 300,000 pps total) at the time. However, input errors went from 0 to about 1200 (oddly, it was 1200 every other second, and 0 for the seconds in-between.) A bit of digging around led me to increase kern.polling.burst_max. According to http://info.iet.unipi.it/~luigi/polling/, "The default value is enough for a 100Mbit ethernet". I increased it gradually to 900, whcih has almost (but not entirely) eliminated the errors. Now the errors are zero for most intervals, but every 10 or so intervals there are between 10 and 100 input errors. So: - does it make sense to leave the default at 150, in this day of gigabit nics? - is there a danger in increasing the burt_max? (My burst size goes straight to the max of 900.) - can it be increased more ? - are there other variables that make sense to increase for gigabit? (like kern.polling.each_burst:?) Since I increased the burst max, I now have slowly incrementing kern.polling.lost_polls - about 1 every 2 seconds. Anything to worry about? Thanks Steve Francis