From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 18 00:45:43 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50C991065674 for ; Tue, 18 Nov 2008 00:45:43 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from n55.bullet.mail.sp1.yahoo.com (n55.bullet.mail.sp1.yahoo.com [98.136.44.188]) by mx1.freebsd.org (Postfix) with SMTP id 2E9E18FC17 for ; Tue, 18 Nov 2008 00:45:42 +0000 (UTC) (envelope-from won.derick@yahoo.com) Received: from [69.147.65.148] by n55.bullet.mail.sp1.yahoo.com with NNFMP; 18 Nov 2008 00:45:42 -0000 Received: from [69.147.84.123] by t11.bullet.mail.sp1.yahoo.com with NNFMP; 18 Nov 2008 00:45:42 -0000 Received: from [127.0.0.1] by omp209.mail.sp1.yahoo.com with NNFMP; 18 Nov 2008 00:45:42 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 807172.20181.bm@omp209.mail.sp1.yahoo.com Received: (qmail 27144 invoked by uid 60001); 18 Nov 2008 00:45:42 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID; b=xURvVOcyWOufgW9rA0zsTLoilMkqZq4Wer4QjIn0YzHQy5uNPRlfeOUHN0Sd3cIodrZcgEJDVoWvPa/lEOcGatRaH5m3zDMyzZAgCPXBIzhK/C/3sJWULSDlc9H3FWbgz2FhAcZSUZuPyUty2DbM+uIqapr1z9f2BIoGiyhMkkM=; X-YMail-OSG: ZRHODTIVM1mC56W7hU5Q9EIL8TT68rf6lzm8eccBFkwcffkOs3FAHF4e_4u_gQraYAqupd8xWG7yWJ97V15dTL7Pq0jKn0LpeFQ9lBiMNbhETsDi2gO8RLnnK2dGtsli0hzisJD8bs0UHSKjJLrQ5jzmc.Y- Received: from [58.71.34.137] by web45805.mail.sp1.yahoo.com via HTTP; Mon, 17 Nov 2008 16:45:42 PST X-Mailer: YahooMailRC/1155.29 YahooMailWebService/0.7.260.1 References: <245613.95922.qm@web45809.mail.sp1.yahoo.com> Date: Mon, 17 Nov 2008 16:45:42 -0800 (PST) From: Won De Erick To: Ivan Voras , freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <718671.27101.qm@web45805.mail.sp1.yahoo.com> X-Mailman-Approved-At: Tue, 18 Nov 2008 02:49:52 +0000 Cc: Subject: Re: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Nov 2008 00:45:43 -0000 > ----- Original Message ---- > From: Ivan Voras > To: freebsd-hackers@freebsd.org > Sent: Tuesday, November 18, 2008 5:21:04 AM > Subject: Re: NET.ISR and CPU utilization performance w/ HP DL 585 using FreeBSD 7.1 Beta2 > > Won De Erick wrote: > > > I compiled the following em driver for Intel NIC Pro (82571) w/ FreeBSD 7.1 Beta 2 on HPDL 585 machine having 16CPUs. > > > > http://people.yandex-team.ru/~wawa/ > > > > With net.isr.direct=1, I made some changes on kthreads(default=2) for em0 and em1's rx. > > > > dev.em.0.rx_kthreads: 6 > > .... > > dev.em.1.rx_kthreads: 6 > > > > With these settings, the result is: > > > > CPU: 0.0% user, 0.0% nice, 57.2% system, 3.6% interrupt, 39.2% idle > > Mem: 17M Active, 7228K Inact, 156M Wired, 76K Cache, 21M Buf, 31G Free > > Swap: 4096M Total, 4096M Free > > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > 52 root 1 43 - 0K 16K CPU12 c 41:38 100.00% em0_rx_kthread_1 > > 51 root 1 43 - 0K 16K CPU3 3 41:38 100.00% em0_rx_kthread_0 > > 54 root 1 -68 - 0K 16K CPU2 2 39:39 100.00% em1_txcleaner > > 1283 root 1 43 - 0K 16K CPU1 1 38:55 100.00% em0_rx_kthread_3 > > 1282 root 1 43 - 0K 16K CPU10 a 38:55 100.00% em0_rx_kthread_2 > > 1344 root 1 43 - 0K 16K CPU9 9 25:51 100.00% em0_rx_kthread_5 > > 1343 root 1 43 - 0K 16K CPU4 4 25:51 100.00% em0_rx_kthread_4 > > 12 root 1 171 ki31 0K 16K CPU14 e 44:28 91.70% idle: cpu14 > > This is very interesting. Do you see real performance (network > throughput) increase? > There's a little improvement on the throughput, but packet errors occurred on both interfaces. # netstat -I em1 -w 1 -d input (em1) output packets errs bytes packets errs bytes colls drops 32494 483 23083087 15681 0 23719154 0 82 30547 330 23104447 16062 0 23077442 0 44 # netstat -I em0 -w 1 -d input (em0) output packets errs bytes packets errs bytes colls drops 19889 640 24144754 21307 0 8719922 0 0 18071 2436 25966238 21088 0 8766995 0 0 Is there any other thing that I can tweak to solve the problem?