From owner-freebsd-hackers@FreeBSD.ORG  Fri Apr 11 09:34:27 2003
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id 2FD8637B401; Fri, 11 Apr 2003 09:34:27 -0700 (PDT)
Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net
	[207.217.120.139])	by mx1.FreeBSD.org (Postfix) with ESMTP
	id 728C743FBD; Fri, 11 Apr 2003 09:34:26 -0700 (PDT)
	(envelope-from tlambert2@mindspring.com)
Received: from pool0023.cvx40-bradley.dialup.earthlink.net ([216.244.42.23]
	helo=mindspring.com)
	by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128)
	(Exim 3.33 #1)	id 1941Tq-0001O5-00; Fri, 11 Apr 2003 09:34:23 -0700
Message-ID: <3E96EE33.FAF4FABB@mindspring.com>
Date: Fri, 11 Apr 2003 09:32:51 -0700
From: Terry Lambert <tlambert2@mindspring.com>
X-Mailer: Mozilla 4.79 [en] (Win98; U)
X-Accept-Language: en
MIME-Version: 1.0
To: bj@dc.luth.se
References: <200304111624.h3BGO9Kl087165@dc.luth.se>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4fcc023af216ec73058faf13ee1779ec9350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c
cc: freebsd-hackers@freebsd.org
cc: freebsd-performance@freebsd.org
cc: "Jin Guojun \[DSD\]" <j_guojun@lbl.gov>
cc: Eric Anderson <anderson@centtech.com>
cc: David Gilbert <dgilbert@velocet.ca>
Subject: Re: tcp_output starving -- is due to mbuf get delay?
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 11 Apr 2003 16:34:27 -0000

Borje Josefsson wrote:
> > A good thing to look at at this point would be:
> >
> >       o       Clean boot of FreeBSD target
> >       o       Run NetBSD against it
> >       o       Save statistics
> 
> What type of statistics do You mean?

Dropped packets; frags; delayed acks.  The stuff you get from
"netstat -s" and "netstat -m".

> > You mean "bandwidth delay product".  Yes, assuming you have packet
> > loss.  From your description of your setup, packet loss should not
> > be possible, so we can discount it as a factor.
> 
> Of cause packet loss is possible on a nationwide network. If I loose a
> packet on the (expected) 10 second test (with NetBSD), recovering from
> that drops performance from 900+ to ~550 Mbps. Thos shows very clearly if
> I run "netstat 1".

You are running these tests over .se's nationwide network?

> > You may want to
> > disable fast restart on the FreeBSD sender.
> 
> Which OID is that?

See other posting; I listed a bunch of OIDs to play with.

One other, if you are running -CURRENT, would be:

	net.isr.netisr_enable		-> 1

This basically implements part 1 of 3 of LRP, which should
reduce your per packet latency by about 50ms +/- 50ms.

Note:	The logic here is inverted; you'd expect "0=No NETISR",
but it's just the opposite.


> As a side note, I tried to set tcp.inflight_enable, but that made things
> much worse.

It's less useful on GigE than elsewhere (IMO).  Use netisr_enable
instead.

-- Terry