From owner-freebsd-hackers@FreeBSD.ORG Sun Feb 29 18:38:33 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D44B816A4CE for ; Sun, 29 Feb 2004 18:38:33 -0800 (PST) Received: from mail.sandvine.com (sandvine.com [199.243.201.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3AA6D43D2D for ; Sun, 29 Feb 2004 18:38:33 -0800 (PST) (envelope-from don@sandvine.com) Received: by mail.sandvine.com with Internet Mail Service (5.5.2657.72) id <15P450LP>; Sun, 29 Feb 2004 21:38:32 -0500 Message-ID: From: Don Bowman To: 'Mike Tancsa' , Don Bowman Date: Sun, 29 Feb 2004 21:38:20 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain; charset="iso-8859-1" cc: freebsd-hackers@freebsd.org Subject: RE: em0, polling performance, P4 2.8ghz FSB 800mhz X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Mar 2004 02:38:34 -0000 From: Mike Tancsa [mailto:mike@sentex.net] > At 08:44 PM 29/02/2004, Don Bowman wrote: > >From: Mike Tancsa [mailto:mike@sentex.net] > > > > > > On Sat, 28 Feb 2004 23:17:44 -0500, in > sentex.lists.freebsd.hackers > > > > >If you want to spend more time in kernel, perhaps change > > > > > > > >I might have HZ @ 2500 as well. > > > > > > Hi, > > > Just curious as to the reasoning behind that ? > > > >@ high packet rates, you don't have enough DMA > >queues available to the em driver, and will drop. > >increasing the number of dma buffers will cause > >problems with cache occupancy. Increasing the HZ > >doesn't have a huge cost. > > But why that value ? Did you determine it by trial and error > or deduce it > based on some other factors ? Also, is this value optimal > for fxp based boxes. I picked 2500 as the best for my system. Its higher than allowed by rfc1323 and PAWS [kern/61404], but not by so much that i anticipate a problem. For my target packets per second rate, it means that i can use a reasonable number of dma descriptors. I found that bridging performance in particular needs the higher hz to avoid dropping packets, to improve its performance. I'm not sure what affect on fxp. fxp is inherently limited by something internal to it, which prevents achieving high packet rates. bge is the best chip, but doesn't have the best bsd support. The value of HZ needs to be based on your target packet rate, the maximum latency in your system, and the size of your buffers for all steps. more buffers == better ability to handle latency bursts, but worse for cache occupancy. Freebsd is not the best system for trying to guarantee latency through, you can find things like ahd, syncache, arp freeing that will suddenly wake up and munch all kinds of cpu time with spl? taken. freebsd-current is both better and worse: its better with the fine grained locking, but worse since those locks can end up costing you more than you would have spent just taking giant and being done with it: semaphores are expensive, particularly on SMP systems.