Date: Fri, 1 Nov 2002 11:04:32 -0500 From: Fran Lawas-Grodek <Fran.Lawas-Grodek@grc.nasa.gov> To: Luigi Rizzo <rizzo@icir.org> Cc: Cindy.Tran@grc.nasa.gov, freebsd-net@FreeBSD.ORG, Mark.Allman@grc.nasa.gov Subject: Re: Problem in High Speed and Long Delay with FreeBSD Message-ID: <20021101110432.D3259@grc.nasa.gov> In-Reply-To: <20021031150217.A35590@carp.icir.org>; from rizzo@icir.org on Thu, Oct 31, 2002 at 03:02:17PM -0800 References: <20021031135601.B23802@grc.nasa.gov> <20021031150217.A35590@carp.icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
These are our sysctl settings:
kern.ipc.maxsockbuf=4194304
net.inet.tcp.sendspace=3125000
net.inet.tcp.recvspace=3125000
net.inet.ip.intr_queue_maxlen=500
nmbclusters=32768
After reading your suggestion, we were able to achieve a
slightly better throughput from 32Mbps on the 250ms delayed
network, to 46Mbps overall throughput, by increasing the -l
buffer length from 1024 to 8192 bytes. Increasing
the above intr_queue_maxlen from the default 50 to 500 also
helped a bit. Our time sequence plot now shows a sender buffer
window of 1.5MB being used, from the 1MB of the earlier tests.
(BTW, a typo in the my posted ttcp example -- "-b" should
be "-b 3125000", not "-b 312500".)
We still do not understand why we cannot not get a better window
usage of our requested socket buffer of 3MB, a better throughput of
60+Mbps.
Any other thoughts?
Fran Lawas-Grodek
Cindy Tran
NASA Glenn Research Center
________________________________________________________________
Frances J. Lawas-Grodek |
NASA Glenn Research Center | phone: (216) 433-5052
21000 Brookpark Rd, MS 142-4 | fax : (216) 433-8000
Cleveland, Ohio 44135 | email: fran@grc.nasa.gov
________________________________________________________________
On Thu, Oct 31, 2002 at 03:02:17PM -0800, Luigi Rizzo wrote:
> you might want to have a look at the sysctl variable
> kern.ipc.sockbuf_waste_factor too.
>
> Remember that memory is charged to socket buffers depending on how
> many clusters are allocated, even if they are not fully used.
> E.g. in your example you are probably doing 1KB writes each of
> which consumes a 2KB cluster plus a 256byte mbuf, so no
> matter what you will never manage to reach more than (roughly)
> a window larger than kern.ipc.maxsockbuf/2.
>
> The max raw amounf of memory allocated in a socket buffer is
> typically
>
> min( tcp_{send|recv}buf * kern.ipc.sockbuf_waste_factor,
> kern.ipc.maxsockbuf)
>
> and probably you are hitting the roof on the second one.
>
> cheers
> luigi
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20021101110432.D3259>