Date: Sun, 19 Mar 2006 09:59:40 +0100 From: "OxY" <oxy@field.hu> To: "Jin Guojun [VFFS]" <g_jin@lbl.gov>, "Chuck Swiger" <cswiger@mac.com> Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit Message-ID: <000401c64b33$7561d940$0201a8c0@oxy> References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com><000601c64a87$51d7dee0$0201a8c0@oxy> <441BFF26.90807@mac.com> <000e01c64a8f$1b2bec80$0201a8c0@oxy> <441CAA8D.3020308@lbl.gov>
next in thread | previous in thread | raw e-mail | index | archive | help
CPU utilization is 0% if apache is not running and 10-20%, when running and serving 30-40 concurrent downloads (traffic is 3-4MB/s on fxp0 interface) i measured the network performance with 'iperf' util, started the server on my box and benchmarked with a client on the other gigabit machine. it showed 0% packet drop, when apache was not running and 4-7%, when running.. then i checked how it behave when i shut down apache and init a local file copy from one (not system!) disk to other (not system disk either). packet drop was 5-10%, due to the higher load. so i think interrupts or just the load takes the network performance, but i have no clue how to fix it. is it possible that the 2000+ xp amd is just weak to serve such a traffic? (em0 traffic's maximum is 18-23MB/s) i think it might be around 30MB/s without packet drop. I did FTP measurement, because what i want is to copy files with high speed from the other gigabit machine. However FTP needs resources (CPU, I/O, etc), but iperf not! iperf shows 20% CPU utilization when apache not running and when there's no packet drop. ps: Now apache says: 14 requests currently being processed traffic is 1MB/s on fxp0, and em0 benchmark with iperf says (64k udp window size): [ 3] 0.0-10.0 sec 235 MBytes 197 Mbits/sec [ 3] Sent 167375 datagrams [ 3] Server Report: [ 3] 0.0-10.0 sec 229 MBytes 192 Mbits/sec 0.066 ms 4115/167375 (2.5%) the other gigabit machine is OK, because i have 0% packet drop, when my machine is totally idle. ----- Original Message ----- From: "Jin Guojun [VFFS]" <g_jin@lbl.gov> To: "OxY" <oxy@field.hu>; "Chuck Swiger" <cswiger@mac.com> Cc: <freebsd-performance@freebsd.org> Sent: Sunday, March 19, 2006 1:49 AM Subject: Re: packet drop with intel gigabit / marwell gigabit > It is still not clear how you did measurement. > Did FTP show such % drop? or Did you measure it by other tools? > How did you measured incoming traffic? > > http://field.hu/netstat.txt shows 0 tcp packet drop. > > Anyway, the first thing first is to have CPU utilization when you see > packet drop. > This can be get from running "top" or "vmstat 1". As well as run > netstat -i -p tcp | grep -i drop > If CPU utilization is approaching 100%, either the traffic is no 2 MBps, > or some process is taking CPU time. For this reason, "top" is a better > tool to use. At this point, if you run netstat command multiple times, > you would see drop counter increasing. > Once you find out what process takes CPU time, then further tuning can be > determined. > > If CPU utilization is well below 70-80%, then you need to use tcpdump and > tcptrace to visualize what cause packet drop, then perform a solution. > > Jin > > ----- Original Message ----- > From: "OxY" <oxy@field.hu> > To: "Chuck Swiger" <cswiger@mac.com> > Cc: <freebsd-performance@freebsd.org> > Sent: Saturday, March 18, 2006 2:23 PM > Subject: Re: packet drop with intel gigabit / marwell gigabit > > >> currently i use HZ=2000 >> here's the output of netstat -i, -s, and vmstat -i : >> (currently i am uploading on the gigabit with ftp, 3 threads) >> >> Field root# vmstat -i >> interrupt total rate >> irq0: clk 27503959 1993 >> irq1: atkbd0 1 0 >> irq3: fxp0 2 0 >> irq7: 146 0 >> stray irq7 146 0 >> irq8: rtc 1765569 127 >> irq10: atapci1 2807786 203 >> irq11: atapci0 475039 34 >> irq13: npx0 1 0 >> irq14: ata0 99 0 >> Total 32552748 2359 >> >> Field root# netstat -i >> Name Mtu Network Address Ipkts Ierrs Opkts Oerrs >> Coll >> fxp0 1500 <Link#1> 00:a0:c9:8d:79:68 13163545 0 21899372 1 >> 0 >> fxp0 1500 195.38.96.64/ field 141 - 6 - - >> em0 1500 <Link#2> 00:0e:0c:a2:ac:42 68644181 4 66793904 0 >> 0 >> em0 1500 195.38.96.64/ field 211255811 - - - >> lo0 16384 <Link#3> 129622061 0 129622061 0 0 >> >> netstat -s is here: >> http://field.hu/netstat.txt >> >> ----- Original Message ----- >> From: "Chuck Swiger" <cswiger@mac.com> >> To: "OxY" <oxy@field.hu> >> Cc: <freebsd-performance@freebsd.org> >> Sent: Saturday, March 18, 2006 1:37 PM >> Subject: Re: packet drop with intel gigabit / marwell gigabit >> >> >>> OxY wrote: >>>> yeah, i googled these settings, but i put them back to default then! >>>> i measured iperf performance, and it showed that the packet drop is >>>> depending on the system load.. >>> >>> If you are using the normal interrupt-driven configuration, you should >>> look at >>> netstat -i, -s, and vmstat -i. If you're turning on device polling, you >>> ought >>> to retry your testing at higher HZ (try 2000 or 5000): >>> >>> echo 'kern.hz="2000"' >> /boot/loader.conf >>> >>> -- >>> -Chuck
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?000401c64b33$7561d940$0201a8c0>