From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 10 06:12:04 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D375337B401; Thu, 10 Apr 2003 06:12:04 -0700 (PDT) Received: from samson.dc.luth.se (samson.dc.luth.se [130.240.112.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D0A343FB1; Thu, 10 Apr 2003 06:12:03 -0700 (PDT) (envelope-from bj@dc.luth.se) Received: from ra.dc.luth.se (bj@ra.dc.luth.se [130.240.112.180]) by samson.dc.luth.se (8.12.5/8.12.5) with ESMTP id h3ADBgjY022790; Thu, 10 Apr 2003 15:11:42 +0200 (MET DST) Message-Id: <200304101311.h3ADBgjY022790@samson.dc.luth.se> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Eric Anderson In-reply-to: Your message of Thu, 10 Apr 2003 07:31:25 CDT. <3E95641D.1080100@centtech.com> Dcc: X-Disposition-notification-to: Borje.Josefsson@dc.luth.se X-uri: http://www.dc.luth.se/~bj/index.html Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Thu, 10 Apr 2003 15:11:42 +0200 From: Borje Josefsson cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org cc: "Jin Guojun \[DSD\]" cc: David Gilbert Subject: Re: tcp_output starving -- is due to mbuf get delay? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: bj@dc.luth.se List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2003 13:12:05 -0000 On Thu, 10 Apr 2003 07:31:25 CDT Eric Anderson wrote: > David Gilbert wrote: > >>>>>>"Jin" =3D=3D Jin Guojun <[DSD]" > writes: > >>>>> > > = > > Jin> Some details was left behind -- The machine is 2 GHz Intel P4 > > Jin> with 1 GB memory, so the delay is not from either CPU or lack of= > > Jin> memory. > > = > > I just want to quickly jump in with the comment that our GigE tests o= f > > routing through FreeBSD have exposed several-order-of-magnitude > > differences by changing ram/motherboard/which-slot-the-card-is-in. > > = > > Do not assume that a fast CPU is the key. We went through 10 > > motherboards to commission the current routers ... and sometimes > > faster cpus would route slower (in various combinations of > > motherboards and RAM). > = > Wow - this is very interesting. I have two machines with GigE in them,= = > one has 6 gige nics (intel pro/1000T server), in a dual Xeon 1.5Ghz = > (P4), with 2gb of ram. I haven't put it into production yet, so if = > there are any tests I can do, let me know. It would be good to know th= e = > pitfalls before I bring it up live.. :) > = I am also *very* interested in participating in this. I did apply [some = version of] Sean Chittendens patch, but that didn't help. With that patch= = applied my system deadlocked(?) when stressing it vith ttcp. To me (but = bear in mind that I'm not at all a kernel hacker) one of the if-statement= s = in the patch seem to cover to little - i.e. when having the fastscan OIS = unset - code that has to do with fastscan is still executed, at least in = the version I tried. I get Mar 27 11:42:41 stinky kernel: sbappend: bad tail 0x0xc0ef8200 instead of= = 0x0xc0ef4b00 when "fastscan" is unset. I have two test systems with xeon CPUs, that can handle a 1000 km (21 ms)= = full speed GE-connection quite well with NetBSD (I get approx 970 Mbit/se= c = with ttcp), but I run out of CPU when using FreeBSD long before I can fil= l = then network. I could stay with NetBSD, but since I am used to FreeBSD I'= d = rader have that instead. We have tested with Linux too on the same = distance, and it seems that they can handle this too. What we did in NetBSD (-current) was to increase IFQ_MAXLEN in (their) = sys/net/if.h, apart from that it's only "traditional" TCP tuning. My hosts are connected directly to core routers in a 10Gbps nationwide = network, so if anybody is interested in some testing I am more than = willing to participate. If anybody produces a patch, I have a third syste= m = that I can use for piloting of that too. --B=F6rje