From owner-freebsd-net@FreeBSD.ORG Wed Feb 8 01:06:21 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 59C7016A420 for ; Wed, 8 Feb 2006 01:06:21 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from smtp.openaccess.org (smtp.openaccess.org [66.165.52.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id DBC8143D45 for ; Wed, 8 Feb 2006 01:06:20 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from [216.57.214.90] (unknown [216.57.214.90]) by smtp.openaccess.org (Postfix) with ESMTP id 81FAC6D4456; Tue, 7 Feb 2006 17:06:19 -0800 (PST) In-Reply-To: <20060206050301.AC895DCAA3F@mail.npubs.com> References: <20060205011931.E5F08DCA99E@mail.npubs.com> <43E63FC3.2030409@errno.com> <20060206050301.AC895DCAA3F@mail.npubs.com> Mime-Version: 1.0 (Apple Message framework v746.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <8D39FF7E-646E-4770-9ABF-337F69344EA7@staff.openaccess.org> Content-Transfer-Encoding: 7bit From: Michael DeMan Date: Tue, 7 Feb 2006 17:06:29 -0800 To: nielsen@memberwebs.com X-Mailer: Apple Mail (2.746.2) Cc: "freebsd-net@FreeBSD.org" Subject: Re: Polling for ath driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2006 01:06:21 -0000 Hi, Just my 2-cents, but we've found polling to be extremely valuable on low-end hardware as described here. We use it only on fxp drivers, but it moved throughput on 133Mhz hardware from something around 8Mbps to 20Mbps on regular layer-3 packet forwarding and also bumped VPN performance up significantly when used with hardware VPN accelerators. Also, since you can force the system to reserve a certain percent for non-polling tasks, you still have SSH access even when it is running hard, although I suppose at the risk of perhaps packet loss if your userland applications use a lot of CPU. I have no idea on FBSD 5.4 or 6.0 though and would be curious. We're running 5.4 on our normal servers now, but I've been nervous moving the low end equipment off of 4.x because of performance issues. - mike Michael F. DeMan Director of Technology OpenAccess Network Services Bellingham, WA 98225 michael@staff.openaccess.org 360-647-0785 On Feb 5, 2006, at 9:03 PM, Nate Nielsen wrote: > Sam Leffler wrote: >> You might try explaining why you think polling helps your >> performance. >> Unless you've significantly restructured the interrupt handling in >> the >> driver most work is deferred to a non-interrupt context. > > Yes, I saw that. However the interrupts themselves when they are fired > at upwards of a thousand per second do greatly affect performance and > scheduling on a slow box. > > I understand that it may not seem intuitive on faster systems which > are > completely capable of handling such a small interrupt load. But these > embedded systems are slow little puppies, running between 133 and 266 > Mhz (586 class chip). > > Adding polling to this driver does increase performance on embedded > systems. With my current patch (on a 233Mhz system), the throughput > (in > this case a simple TCP stream) goes up by ~6Mbits, from 18Mbits to > 24Mbits. > > But that's not the main benefit. The main thing is the scheduling. > Without having the thousands of interrupts, the box is better able to > balance the RX/TX with the other aspects, such as encapsulation, > packet > filtering and other activities. > > When the entire box is polling driven, it's total throughput > (ethernet, > encapsulation, hardware encryption, wireless) increases greatly and > does > not exhibit livelock symptoms. > > Without polling, these slow systems easily exhibit livelock. This is > where incoming traffic can cause so many interrupts that outgoing > traffic is completely halted and the throughput drops to zero or near > zero. Under these conditions userland processes also run barely or not > at all. The scheduler has no say in what's going on in the system, as > interrupts overwhelm all other activity. > >> Also the >> driver in 6.0 and later does tx interrupt mitigation and rx interrupt >> coalescing so I wouldn't expect interrupt handling to be a >> performance >> limitation. > > Interesting. If there's an option to enable it, I very well may have > missed it. > > However it should be noted, that the default behaviour (in 6.0 > release) > seems to be that the hardware generates about around 2000 > interrupts per > second at around 15 - 18 Mbits throughput. > >> There are other issues that can affect performance but you >> haven't mentioned them... > > Obviously these are not mainstream performance issues for people just > trying to connect to an access point. But when the atheros cards are > used in backhaul applications and are running in the low power > embedded > systems you typically see on an antenna mast, polling makes a big > difference. > > Cheers, > Nate > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >