From owner-freebsd-hardware@FreeBSD.ORG Fri Nov 14 11:30:11 2008 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 571891065670 for ; Fri, 14 Nov 2008 11:30:11 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n29.bullet.mail.mud.yahoo.com (n29.bullet.mail.mud.yahoo.com [68.142.207.48]) by mx1.freebsd.org (Postfix) with SMTP id 2889D8FC14 for ; Fri, 14 Nov 2008 11:30:11 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [68.142.200.221] by n29.bullet.mail.mud.yahoo.com with NNFMP; 14 Nov 2008 11:30:10 -0000 Received: from [68.142.201.242] by t9.bullet.mud.yahoo.com with NNFMP; 14 Nov 2008 11:30:10 -0000 Received: from [127.0.0.1] by omp403.mail.mud.yahoo.com with NNFMP; 14 Nov 2008 11:30:10 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 774753.17095.bm@omp403.mail.mud.yahoo.com Received: (qmail 77148 invoked by uid 60001); 14 Nov 2008 11:30:10 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Message-ID; b=eEq/RjgdiNSv3WojhDEJfT44+SJjHfM0Tw9mP+9huVEdJLskq1UfUbv3u1Sw7ILl7B1F9UhGR7i53vBSvi00XmJixZ6fsuUIewXEGWlJhJMsyPDSx/oZqPWo94IxPlCGjVa4KObI0nmvm0cdU161gkLahWqovJ2nZd/jd4j8HzI=; X-YMail-OSG: pz_Ei3kVM1nqJS_7_q7b9SlPAnBH51L5eHd5kISihmCJ9g3DFMlnS7wlpPMhkDKpZE2hG8Xvgy2aO178VCIGc.2uqdrPzQVRfYHynhMJTgzkRiDULEZjmL5uKnwbH8a.ePvkBr3ykuCmp7EVdNR40eqAPBs- Received: from [58.71.34.137] by web45809.mail.sp1.yahoo.com via HTTP; Fri, 14 Nov 2008 03:30:10 PST X-Mailer: YahooMailRC/1155.20 YahooMailWebService/0.7.260.1 Date: Fri, 14 Nov 2008 03:30:10 -0800 (PST) From: Won De Erick To: Jeremy Chadwick MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <305614.76266.qm@web45809.mail.sp1.yahoo.com> Cc: freebsd-net@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2008 11:30:11 -0000 > ----- Original Message ---- > From: Won De Erick > To: Jeremy Chadwick > Cc: freebsd-hardware@freebsd.org > Sent: Thursday, November 13, 2008 4:07:37 PM > Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage > > Noted on this, I will update you through this thread. With FreeBSD 7.1 Beta2, here is the result: 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 There's a slight difference w/ the previous result though, but I observed that overall CPU utilization didn't change. ### Complete result: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 21 root 1 171 ki31 0K 16K CPU5 5 91:29 100.00% idle: cpu5 14 root 1 171 ki31 0K 16K CPU12 c 91:29 100.00% idle: cpu12 22 root 1 171 ki31 0K 16K CPU4 4 91:29 100.00% idle: cpu4 17 root 1 171 ki31 0K 16K CPU9 9 91:28 100.00% idle: cpu9 12 root 1 171 ki31 0K 16K CPU14 e 91:27 100.00% idle: cpu14 23 root 1 171 ki31 0K 16K CPU3 3 90:58 100.00% idle: cpu3 25 root 1 171 ki31 0K 16K CPU1 1 86:41 100.00% idle: cpu1 19 root 1 171 ki31 0K 16K CPU7 7 84:00 100.00% idle: cpu7 24 root 1 171 ki31 0K 16K CPU2 2 83:53 100.00% idle: cpu2 18 root 1 171 ki31 0K 16K CPU8 8 79:01 100.00% idle: cpu8 26 root 1 171 ki31 0K 16K RUN 0 74:18 100.00% idle: cpu0 11 root 1 171 ki31 0K 16K CPU15 0 0:00 100.00% idle: cpu15 13 root 1 171 ki31 0K 16K CPU13 0 0:00 100.00% idle: cpu13 20 root 1 171 ki31 0K 16K CPU6 6 90:18 99.27% idle: cpu6 52 root 1 -68 - 0K 16K CPU11 b 38:43 95.36% irq32: bce1 51 root 1 -68 - 0K 16K CPU10 a 25:50 85.16% irq31: bce0 16 root 1 171 ki31 0K 16K RUN a 65:39 15.97% idle: cpu10 28 root 1 -32 - 0K 16K WAIT 8 12:28 5.18% swi4: clock sio 15 root 1 171 ki31 0K 16K RUN b 52:46 3.76% idle: cpu11 45 root 1 -64 - 0K 16K WAIT 7 7:29 1.17% irq17: uhci0 47 root 1 -64 - 0K 16K WAIT 6 1:11 0.10% irq16: ciss0 Is there some ways how the intensive [network] load can be distributed among the IDLE processors? Another thing, I observed that in the above test, the net.isr is enabled by default. When I tried disabling this, # sysctl net.isr.direct=0 net.isr.direct: 1 -> 0 the result: 52 root 1 -68 - 0K 16K WAIT b 64:00 42.97% irq32: bce1 51 root 1 -68 - 0K 16K WAIT a 38:22 12.26% irq31: bce0 The CPU utilizations considerably dropped! What was changed in the fbsd7.1? How useful this net.isr is? Is this needed in an environment with heavy network traffic? Can someone explain? > However is there any possibility of the following: > > > I don't know if there's a way to split the interrupt request for each bce's Rx and Tx, > > which means a total of four IRQs, and eventually four cores (or 4 CPUs) > > for the transactions. With this way, the IDLE processors would be utilized. > > What I mean here is, for the two interfaces: > > one IRQ for bce0 Rx > one IRQ for bce0 Tx > one IRQ for bce1 Rx > one IRQ for bce1 Tx > > > Thanks, > > Won > > > > ________________________________ > From: Jeremy Chadwick > To: Won De Erick > Cc: freebsd-hardware@freebsd.org > Sent: Thursday, November 13, 2008 3:46:02 PM > Subject: Re: IRQ31 and IRQ32 on HPDL585 running FreeBSD 7.0 are consuming HIGH CPU usage > > On Wed, Nov 12, 2008 at 11:38:15PM -0800, Won De Erick wrote: > > I am conducting a CPU utilization testing with my box(HP DL 585 running FreeBSD 7.0), and come up with the results below: > > > > 52 root 1 -68 - 0K 16K CPU11 b 123:53 100.00% irq32: bce1 > > 51 root 1 -68 - 0K 16K CPU10 a 119:28 89.06% irq31: bce0 > > > > irq31 and irq32 are consuming high CPU usage, which i think the cause of hard reset. > > There was a ***major*** bce(4) cleanup that just happened. Your 7.0 box > will not have these changes. Please upgrade your box to RELENG_7 > (a.k.a. 7.1-PRERELEASE), csup'd recently (today preferably), and try > your tests again: > > http://lists.freebsd.org/pipermail/freebsd-stable/2008-November/046482.html > > -- > | Jeremy Chadwick jdc at parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | >