From owner-freebsd-net@FreeBSD.ORG Mon Oct 3 10:32:06 2005 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3791E16A41F for ; Mon, 3 Oct 2005 10:32:06 +0000 (GMT) (envelope-from ferdinand.goldmann@jku.at) Received: from mail2.edvz.uni-linz.ac.at (mail2.edvz.uni-linz.ac.at [140.78.3.69]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8BBA543D45 for ; Mon, 3 Oct 2005 10:32:04 +0000 (GMT) (envelope-from ferdinand.goldmann@jku.at) Received: from emailsecure.uni-linz.ac.at (emailsecure.uni-linz.ac.at [140.78.3.66]) by mail2.edvz.uni-linz.ac.at (8.13.3/8.13.3) with ESMTP id j93AVuIB044821 for ; Mon, 3 Oct 2005 12:31:57 +0200 (CEST) (envelope-from ferdinand.goldmann@jku.at) Received: from [140.78.164.13] (jku006048.edvz.uni-linz.ac.at [140.78.6.48]) by emailsecure.uni-linz.ac.at (Postfix) with ESMTP id D86E5228019 for ; Mon, 3 Oct 2005 12:31:56 +0200 (CEST) Message-ID: <4341089F.7010504@jku.at> Date: Mon, 03 Oct 2005 12:31:59 +0200 From: Ferdinand Goldmann Organization: Johannes Kepler University User-Agent: Thunderbird 1.4 (Macintosh/20050908) MIME-Version: 1.0 To: net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.44 Cc: Subject: dummynet, em driver, device polling issues :-(( X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ferdinand.goldmann@jku.at List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Oct 2005 10:32:06 -0000 Hello All! We are using a 1.2GHz xSeries machine to do poor man's traffic shaping for the student hostels connected to our campus. We are using dummynet to do shaping, there is no stateful filtering, there are however several filter rules to lock out people and to do MAC based filtering. Number of users is ranging from several hundred up to 2-3000. We are running a recent 5.4-STABLE. The machine is fitted with two onboard Intel 100Mbit interfaces. One of the two was running on very high load (the internal interface acting as gateway for all of the hostels, where no shaping is done), so we put in a spare Intel Gbit card: em0@pci1:5:0: class=0x020000 card=0x10028086 chip=0x10268086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = '82545GM Gigabit Ethernet Controller' Initial observations showed that the system had a very high interrupt load, most of the times > 50%, and lagged a little bit. Plus, performance-testing with tcpspray yielded bad results at around 10MB/s, or even worse. So we tried enabling device polling on the em0 interface. After some testing, we set HZ=1000 and kern.polling.burst_max: 300. This made things slightly better, our tcpspray tests now reaching a ~13MB/s throughput. However, now the input errors on the em0 interface were rapidly rising: Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll em0 1500 00:0e:0c:6d:4c:f9 3738461290 8605788 3613603979 0 0 So for now, the polling is disabled again. On the mailing list, I have read about performance problems with the em driver, and problems with device polling as well. * are there any magic performance tips for use with the em driver? Any way to eliminate the input errors while using device polling? Or is all of this code too broken at the moment? * is there a driver by Intel which will work well? Has anyone tested such a driver with a network-performance hungry application? For now, the machine seems to run stable, no more errors with device polling disabled. But performance is far lower than it should be. :-( Kind regards and thanks for any input! -- >> Ferdinand Goldmann //// | | >> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at |--00 | UNIX | >> Tel. : +43/732/2468/9398 Fax. : +43/732/2468/9397 C ^ | | >> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at \ ~/ ~~~|~~~~~~~~ >> PGP D4CF 8AA4 4B2A 7B88 65CA 5EDC 0A9B FA9A 13EA B993| |-----3