From owner-freebsd-performance@FreeBSD.ORG Mon Mar 20 19:48:50 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D0D116A423 for ; Mon, 20 Mar 2006 19:48:50 +0000 (UTC) (envelope-from g_jin@lbl.gov) Received: from smtp108.sbc.mail.mud.yahoo.com (smtp108.sbc.mail.mud.yahoo.com [68.142.198.207]) by mx1.FreeBSD.org (Postfix) with SMTP id E5C7143D45 for ; Mon, 20 Mar 2006 19:48:49 +0000 (GMT) (envelope-from g_jin@lbl.gov) Received: (qmail 49947 invoked from network); 20 Mar 2006 19:48:49 -0000 Received: from unknown (HELO ?192.168.2.10?) (jinmtb@sbcglobal.net@68.127.178.44 with plain) by smtp108.sbc.mail.mud.yahoo.com with SMTP; 20 Mar 2006 19:48:48 -0000 Message-ID: <441F0771.6030807@lbl.gov> Date: Mon, 20 Mar 2006 11:50:09 -0800 From: "Jin Guojun [VFFS]" User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.5) Gecko/20050108 X-Accept-Language: zh, zh-CN, en MIME-Version: 1.0 To: OxY References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com><000601c64a87$51d7dee0$0201a8c0@oxy> <441BFF26.90807@mac.com> <000e01c64a8f$1b2bec80$0201a8c0@oxy> <441CAA8D.3020308@lbl.gov> <000401c64b33$7561d940$0201a8c0@oxy> <441D3698.10300@lbl.gov> <000601c64b44$db8dcb00$0201a8c0@oxy> <441E1BF1.6050205@lbl.gov> <000601c64c08$2a7b4990$0201a8c0@oxy> In-Reply-To: <000601c64c08$2a7b4990$0201a8c0@oxy> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 20 Mar 2006 19:55:39 +0000 Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Mar 2006 19:48:50 -0000 OxY wrote: > > ----- Original Message ----- From: "Jin Guojun [VFFS]" > To: "OxY" > Cc: > Sent: Monday, March 20, 2006 4:05 AM > Subject: Re: packet drop with intel gigabit / marwell gigabit > >>>> .... >>>> First let's clear the notation -- Is 30MB/s (MBytes/s) = 240Mb/s >>>> (Mbit/s) or MB/s means Mbits/s >>>> If MB/s is MBytes/s and you also write this amount data to a disk, >>>> plus other traffic on fxp0 to disk too, >>>> then your problem may be bonded by memory bandwidth because CPU >>>> utilization is low: >>>> (240 + 24~32) x 2 is about 535 Mbit/s (some chipset/motherboard >>>> has low memory BW for AMD) >>>> If this is true, then this no thing you can tune. What does the >>>> chipset (Motherboard) this machine have? >>> >>> >>> >>> 30MB/s is Megabytes/sec, currently i have 18-20MB/s peak and 15MB/s >>> avg. >>> it's not 535Mbit/s, because i only download it to my machine, no >>> upload. >>> disks are different from apache disks, these disks have own >>> controller in one pci slot. >>> the packet drop is 5-7% at 200Mbit iperf test, 100Mbit drop is >>> around zero. >>> i have on motherboard which has VIA KT400 northbridge >>> http://uk.asus.com/products4.aspx?modelmenu=2&model=226&l1=3&l2=13&l3=62 >>> >> >> >> Yes, this is one of problem chipset. I bought one about 3 years ago. >> After one day testing, I returned it for changing a A7V600 (VIA KT600 >> chipset), >> which is 30% more memory bandwidth than KT400. A7V600 can only >> receive max >> 604 Mb/s TCP, so You can imagine what the KT400 can do :-) >> I do not have a record (because it is too bad), but taking minimum >> 25% off, >> it probably about 420-430 Mb/s (50MB/s). Now you can do the math when >> the >> machine also writing data to a disk (assume disk a fast enough). I >> would expect >> 2/3 of 430 Mb/s, which is about 280~290 Mb/s (35 MB/s). >> If you experiment these numbers, you are at there. No improvement you >> can make >> further. > > > i have doubts, because when i have 3-4MB/s traffic on fxp0 then em0 peak > is 18MB/s, but when fxp0 is almost idle, have 500kB/s traffic, then > em0 can only > do 20MB/s.. Since you did not get anything better than 35MB/s, then, what is your doubt -- the maximum I/O A7V8X can do? The 35 MB/s is the theoretical ceiling based on 2100+ CPU. 2000+ will be slower. In previous email, you mentioned you had 240 Mb/s (30 MB/s) on em0 with some traffic on fxp0, it is pretty much close to your hardware physical limitation. Forget drop in this figure, because this demonstrated how much hardware can do, rather than lossless transmission. Once you have determined the ceiling, you need to keep a margin for lossless Tx. for other overhead, such as context switch, etc. 20 MB/s is not good enough for this board, you may expect 28-30 MB/s with fine tuning. Unless you will be happy with 28 MB/s, it does not make sense to waste time to try to bump I/O above 30 MB/s for your application if you have another motherboard. Again, this motherboard is designed for entertainment boxes not for network I/O based applications. -Jin