From owner-freebsd-current@FreeBSD.ORG Tue Aug 15 12:02:06 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E25416A4DF for ; Tue, 15 Aug 2006 12:02:06 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 11E1A43D45 for ; Tue, 15 Aug 2006 12:02:05 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 9B66046BC4; Tue, 15 Aug 2006 08:02:04 -0400 (EDT) Date: Tue, 15 Aug 2006 13:02:04 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Bachilo Dmitry In-Reply-To: <200608151627.37828.root@solink.ru> Message-ID: <20060815130002.M45647@fledge.watson.org> References: <200608151627.37828.root@solink.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: current@freebsd.org Subject: Re: throughput and interrupts X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Aug 2006 12:02:06 -0000 On Tue, 15 Aug 2006, Bachilo Dmitry wrote: > I am completely stuck. I have a router, that rules three subnets. At first > there were Allied Telesyn's netcards with realtek chipsets. I am using > ipfw+natd and I had like 3,5 megabytes per second with 20 per cent of > interrupt load. I had "discarded oversized frame" too often so I've replaced > two main cards with 3Com 905 and now my xl0 and xl1 never discard frames. > But I saw no speed boost and interrupts were also very high. I have > discovered some IRQ conflicts and removed them, now I have single device per > IRQ, but still i have near 20-30 % of interrupt load, when i download > something through this router (and natd takes rest % of CPU). Some people > advised me to use polling, so I've built kernel with HZ=1000, polling and > tried it. Interrupts grew to 40-50% and speed decreased to 200-300 kb/s, so > I had to turn the polling off. > > I just don't know what to try, to gain at least 9 mb/s, because this is what > I get at home with the same cards, the same processor, with no polling and > throgh the same provider. I think it would be useful to compare the settled loads of the two systems, if they are otherwise identical, using systat -vmstat 1 and top -S. I'd load the systems up, leave the tools running, and wait a couple of minutes, then compare snapshots of the output from both systems. top -S is particularly informative, as it will tell you which ithreads are burning the most CPU, and systat -vmstat will tell us the context switch rate and interrupt rates of various sources. There are some general optimization techniques that can be applied here -- for example, if it's an SMP box, you might find that net.isr.direct=1 improves performance. However, I think the real problem here is the performance difference, which suggests a problem that needs resolving, rather than optimization being required up front. Thanks, Robert N M Watson Computer Laboratory University of Cambridge