From owner-freebsd-performance@FreeBSD.ORG Fri Apr 11 09:24:16 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3741937B401; Fri, 11 Apr 2003 09:24:16 -0700 (PDT) Received: from samson.dc.luth.se (samson.dc.luth.se [130.240.112.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 97D6443FB1; Fri, 11 Apr 2003 09:24:14 -0700 (PDT) (envelope-from bj@dc.luth.se) Received: from dc.luth.se (root@bompe.dc.luth.se [130.240.60.42]) by samson.dc.luth.se (8.12.5/8.12.5) with ESMTP id h3BGOAjY021255; Fri, 11 Apr 2003 18:24:10 +0200 (MET DST) Received: from bompe.dc.luth.se (bj@localhost.dc.luth.se [127.0.0.1]) by dc.luth.se (8.12.6/8.11.3) with ESMTP id h3BGO9Kl087165; Fri, 11 Apr 2003 18:24:09 +0200 (CEST) (envelope-from bj@bompe.dc.luth.se) Message-Id: <200304111624.h3BGO9Kl087165@dc.luth.se> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Terry Lambert In-reply-to: Your message of Fri, 11 Apr 2003 09:08:19 PDT. <3E96E873.9CC19544@mindspring.com> Dcc: X-Disposition-notification-to: Borje.Josefsson@dc.luth.se X-uri: http://www.dc.luth.se/~bj/index.html Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Date: Fri, 11 Apr 2003 18:24:09 +0200 From: Borje Josefsson cc: Mattias Pantzare cc: freebsd-hackers@freebsd.org cc: freebsd-performance@freebsd.org cc: Eric Anderson cc: David Gilbert Subject: Re: tcp_output starving -- is due to mbuf get delay? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: bj@dc.luth.se List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2003 16:24:16 -0000 On Fri, 11 Apr 2003 09:08:19 PDT Terry Lambert wrote: > Mattias Pantzare wrote: > > Terry Lambert wrote: > > > Latency =3D pool retention time =3D queue size > > = > > Then explain this, FreeBSD to FreeBSD on that link uses all CPU on th= e > > sender, the reciver is fine, but performance is not. NetBSD to FreeBS= D > > fills the link (1 Gbit/s). On the same computers. MTU 4470. Send and > > receive maximum windows where tuned to the same values on NetBSD and > > FreeBSD. > = > I rather expect that the number of jumbogram buffers on FreeBSD is > tiny and/or your MTU is not being properly negotiated between the > endpoints, and you are fragging the bejesus out of your packets. Both endpoints have MTU set to 4470, as have all the routers inbetween. = "traceroute -n -Q 1 -q 1 -w 1 -f remotehost 4470" and netstat both report= s = 4470. = > A good thing to look at at this point would be: > = > o Clean boot of FreeBSD target > o Run NetBSD against it > o Save statistics What type of statistics do You mean? > o Clean boot of FreeBSD target > o Run FreeBSD against it > o Save statistics > o Compare saved statistics of NetBSD vs. FreeBSD > against the target machine > = > > And packet loss will affect the performance diffrently if you have a > > large bandwith-latency product. > = > You mean "bandwidth delay product". Yes, assuming you have packet > loss. From your description of your setup, packet loss should not > be possible, so we can discount it as a factor. Of cause packet loss is possible on a nationwide network. If I loose a = packet on the (expected) 10 second test (with NetBSD), recovering from = that drops performance from 900+ to ~550 Mbps. Thos shows very clearly if= = I run "netstat 1". > You may want to > disable fast restart on the FreeBSD sender. Which OID is that? As a side note, I tried to set tcp.inflight_enable, but that made things = much worse. --B=F6rje