From owner-freebsd-current@FreeBSD.ORG Wed Aug 16 03:15:25 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EF72C16A4DD for ; Wed, 16 Aug 2006 03:15:24 +0000 (UTC) (envelope-from root@solink.ru) Received: from mail.academ.org (mail.academ.org [81.1.226.250]) by mx1.FreeBSD.org (Postfix) with ESMTP id D3D1543D5A for ; Wed, 16 Aug 2006 03:15:23 +0000 (GMT) (envelope-from root@solink.ru) Received: from mail.academ.org (localhost [127.0.0.1]) by mail.academ.org (Postfix) with ESMTP id 872EA818A3 for ; Wed, 16 Aug 2006 09:42:34 +0700 (NOVST) Received: from bocha.solink.office (unknown [85.118.228.14]) by mail.academ.org (Postfix) with ESMTP id 69CB77E7CE for ; Wed, 16 Aug 2006 09:42:34 +0700 (NOVST) From: Bachilo Dmitry Organization: Solink Ltd To: freebsd-current@freebsd.org Date: Wed, 16 Aug 2006 09:42:33 +0700 User-Agent: KMail/1.9.1 References: <200608151627.37828.root@solink.ru> <20060815130002.M45647@fledge.watson.org> In-Reply-To: <20060815130002.M45647@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200608160942.33841.root@solink.ru> X-AV-Checked: ClamAV Subject: Re: throughput and interrupts X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Aug 2006 03:15:25 -0000 =F7 =D3=CF=CF=C2=DD=C5=CE=C9=C9 =CF=D4 =F7=D4=CF=D2=CE=C9=CB 15 =C1=D7=C7= =D5=D3=D4=C1 2006 19:02 Robert Watson =CE=C1=D0=C9=D3=C1=CC(a): > On Tue, 15 Aug 2006, Bachilo Dmitry wrote: > > I am completely stuck. I have a router, that rules three subnets. At > > first there were Allied Telesyn's netcards with realtek chipsets. I am > > using ipfw+natd and I had like 3,5 megabytes per second with 20 per cent > > of interrupt load. I had "discarded oversized frame" too often so I've > > replaced two main cards with 3Com 905 and now my xl0 and xl1 never > > discard frames. But I saw no speed boost and interrupts were also very > > high. I have discovered some IRQ conflicts and removed them, now I have > > single device per IRQ, but still i have near 20-30 % of interrupt load, > > when i download something through this router (and natd takes rest % of > > CPU). Some people advised me to use polling, so I've built kernel with > > HZ=3D1000, polling and tried it. Interrupts grew to 40-50% and speed > > decreased to 200-300 kb/s, so I had to turn the polling off. > > > > I just don't know what to try, to gain at least 9 mb/s, because this is > > what I get at home with the same cards, the same processor, with no > > polling and throgh the same provider. > > I think it would be useful to compare the settled loads of the two system= s, > if they are otherwise identical, using systat -vmstat 1 and top -S. I'd > load the systems up, leave the tools running, and wait a couple of minute= s, > then compare snapshots of the output from both systems. top -S is > particularly informative, as it will tell you which ithreads are burning > the most CPU, and systat -vmstat will tell us the context switch rate and > interrupt rates of various sources. > > There are some general optimization techniques that can be applied here -- > for example, if it's an SMP box, you might find that net.isr.direct=3D1 > improves performance. However, I think the real problem here is the > performance difference, which suggests a problem that needs resolving, > rather than optimization being required up front. > > Thanks, > > Robert N M Watson > Computer Laboratory > University of Cambridge Here is what I see: CPU states: 9.6% user, 0.0% nice, 58.5% system, 31.9% interrupt, 0.0% id= le Mem: 127M Active, 664M Inact, 138M Wired, 48M Cache, 111M Buf, 21M Free Swap: 64M Total, 272K Used, 64M Free PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 11 root 171 52 0K 8K RUN 422.7H 33.06% 33.06% idle 462 root 109 0 1496K 952K RUN 202:44 21.53% 21.53% natd 37 root -44 -163 0K 8K WAIT 99:09 5.08% 5.08% swi1: net 29 root -68 -187 0K 8K RUN 59:11 3.52% 3.52% irq19: xl0 30 root -68 -187 0K 8K WAIT 40:38 3.47% 3.47% irq20: xl1 This machine is Celeron 1,7 Ghz and 1 GB RAM and it gives me 3,7 MB per sec= ond=20 max. While my home router is Celeron 1,7 Ghz with 512 MB RAM, the same xl=20 NICs and I have almost 10 MB per second there with at least 48% of free CPU= =2E=20 I'll keep on testing this stuff, so i'll keep reporting the results. =2D-=20 =2D----------------------- =F3 =D5=D7=C1=D6=C5=CE=C9=C5=CD, =E2=C1=DE=C9=CC=CF =E4=CD=C9=D4=D2=C9=CA =F2=D5=CB=CF=D7=CF=C4=C9=D4=C5=CC=D8 =CF=D4=C4=C5=CC=C1 =D3=C9=D3=D4=C5=CD= =CE=CF=CA =C9=CE=D4=C5=C7=D2=C1=C3=C9=C9 =EF=EF=EF "=EB=CF=CD=D0=C1=CE=C9=D1 =F3=CF=EC=C9=CE=CB"