From owner-freebsd-performance@FreeBSD.ORG Sun Mar 19 08:59:46 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D2C8116A400 for ; Sun, 19 Mar 2006 08:59:46 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id 21E5A43D70 for ; Sun, 19 Mar 2006 08:59:41 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id D7AE5119D2E; Sun, 19 Mar 2006 09:59:01 +0100 (CET) Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id 13062-02; Sun, 19 Mar 2006 09:59:01 +0100 (CET) Received: from oxy (dsl217-197-187-71.pool.tvnet.hu [217.197.187.71]) by green.field.hu (Postfix) with ESMTP id 52D91119CC4; Sun, 19 Mar 2006 09:59:01 +0100 (CET) Message-ID: <000401c64b33$7561d940$0201a8c0@oxy> From: "OxY" To: "Jin Guojun [VFFS]" , "Chuck Swiger" References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com><000601c64a87$51d7dee0$0201a8c0@oxy> <441BFF26.90807@mac.com> <000e01c64a8f$1b2bec80$0201a8c0@oxy> <441CAA8D.3020308@lbl.gov> Date: Sun, 19 Mar 2006 09:59:40 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Mar 2006 08:59:46 -0000 CPU utilization is 0% if apache is not running and 10-20%, when running and serving 30-40 concurrent downloads (traffic is 3-4MB/s on fxp0 interface) i measured the network performance with 'iperf' util, started the server on my box and benchmarked with a client on the other gigabit machine. it showed 0% packet drop, when apache was not running and 4-7%, when running.. then i checked how it behave when i shut down apache and init a local file copy from one (not system!) disk to other (not system disk either). packet drop was 5-10%, due to the higher load. so i think interrupts or just the load takes the network performance, but i have no clue how to fix it. is it possible that the 2000+ xp amd is just weak to serve such a traffic? (em0 traffic's maximum is 18-23MB/s) i think it might be around 30MB/s without packet drop. I did FTP measurement, because what i want is to copy files with high speed from the other gigabit machine. However FTP needs resources (CPU, I/O, etc), but iperf not! iperf shows 20% CPU utilization when apache not running and when there's no packet drop. ps: Now apache says: 14 requests currently being processed traffic is 1MB/s on fxp0, and em0 benchmark with iperf says (64k udp window size): [ 3] 0.0-10.0 sec 235 MBytes 197 Mbits/sec [ 3] Sent 167375 datagrams [ 3] Server Report: [ 3] 0.0-10.0 sec 229 MBytes 192 Mbits/sec 0.066 ms 4115/167375 (2.5%) the other gigabit machine is OK, because i have 0% packet drop, when my machine is totally idle. ----- Original Message ----- From: "Jin Guojun [VFFS]" To: "OxY" ; "Chuck Swiger" Cc: Sent: Sunday, March 19, 2006 1:49 AM Subject: Re: packet drop with intel gigabit / marwell gigabit > It is still not clear how you did measurement. > Did FTP show such % drop? or Did you measure it by other tools? > How did you measured incoming traffic? > > http://field.hu/netstat.txt shows 0 tcp packet drop. > > Anyway, the first thing first is to have CPU utilization when you see > packet drop. > This can be get from running "top" or "vmstat 1". As well as run > netstat -i -p tcp | grep -i drop > If CPU utilization is approaching 100%, either the traffic is no 2 MBps, > or some process is taking CPU time. For this reason, "top" is a better > tool to use. At this point, if you run netstat command multiple times, > you would see drop counter increasing. > Once you find out what process takes CPU time, then further tuning can be > determined. > > If CPU utilization is well below 70-80%, then you need to use tcpdump and > tcptrace to visualize what cause packet drop, then perform a solution. > > Jin > > ----- Original Message ----- > From: "OxY" > To: "Chuck Swiger" > Cc: > Sent: Saturday, March 18, 2006 2:23 PM > Subject: Re: packet drop with intel gigabit / marwell gigabit > > >> currently i use HZ=2000 >> here's the output of netstat -i, -s, and vmstat -i : >> (currently i am uploading on the gigabit with ftp, 3 threads) >> >> Field root# vmstat -i >> interrupt total rate >> irq0: clk 27503959 1993 >> irq1: atkbd0 1 0 >> irq3: fxp0 2 0 >> irq7: 146 0 >> stray irq7 146 0 >> irq8: rtc 1765569 127 >> irq10: atapci1 2807786 203 >> irq11: atapci0 475039 34 >> irq13: npx0 1 0 >> irq14: ata0 99 0 >> Total 32552748 2359 >> >> Field root# netstat -i >> Name Mtu Network Address Ipkts Ierrs Opkts Oerrs >> Coll >> fxp0 1500 00:a0:c9:8d:79:68 13163545 0 21899372 1 >> 0 >> fxp0 1500 195.38.96.64/ field 141 - 6 - - >> em0 1500 00:0e:0c:a2:ac:42 68644181 4 66793904 0 >> 0 >> em0 1500 195.38.96.64/ field 211255811 - - - >> lo0 16384 129622061 0 129622061 0 0 >> >> netstat -s is here: >> http://field.hu/netstat.txt >> >> ----- Original Message ----- >> From: "Chuck Swiger" >> To: "OxY" >> Cc: >> Sent: Saturday, March 18, 2006 1:37 PM >> Subject: Re: packet drop with intel gigabit / marwell gigabit >> >> >>> OxY wrote: >>>> yeah, i googled these settings, but i put them back to default then! >>>> i measured iperf performance, and it showed that the packet drop is >>>> depending on the system load.. >>> >>> If you are using the normal interrupt-driven configuration, you should >>> look at >>> netstat -i, -s, and vmstat -i. If you're turning on device polling, you >>> ought >>> to retry your testing at higher HZ (try 2000 or 5000): >>> >>> echo 'kern.hz="2000"' >> /boot/loader.conf >>> >>> -- >>> -Chuck