Date: Sun, 2 Dec 2001 13:32:24 -0800 (PST) From: Matthew Dillon <dillon@apollo.backplane.com> To: Lamont Granquist <lamont@scriptkiddie.org> Cc: Richard Sharpe <sharpe@ns.aus.com>, <freebsd-hackers@FreeBSD.ORG> Subject: Re: Patch #3 (TCP / Linux / Performance) Message-ID: <200112022132.fB2LWO093222@apollo.backplane.com> References: <20011202121147.G92925-100000@coredump.scriptkiddie.org>
next in thread | previous in thread | raw e-mail | index | archive | help
:On Sun, 2 Dec 2001, Matthew Dillon wrote:
:> Throughput 47.2446 MB/sec (NB=59.0558 MB/sec 472.446 MBit/sec) 20 procs
:>
:> It seems to max-out at around 75,000 packets per second (input + output).
:>
:> I doubt these results could be duplicated on anything but a DELL2550.
:> It dedicates an entire internal 64 bit 66MHz PCI bus just to the
:> on-board gigabit ethernet.
:
:What is the remaining bottleneck in these tests? CPU? Interrupts? What
:would you need to do to get that closer to the theoretical limit
:(something around 920 Mbs for GigE IIRC)?
The tbench test is mainly a transactional test (don't ask me why
it only reports bandwidth). So the ultimate limiting factor is going
to be cpu in regards to interrupt processing and transactional latency.
It is fairly easy to saturated the GigE on the DELL if all you are
doing is pushing a huge amount of data over a single TCP connection.
This is connecting to inetd running a dd if=/dev/zero bs=32k on a
machine with the rfc sysctl's turned on and 262144 byte send and
receive buffers, without jumbo frames (my gigE switch doesn't support
them :-( ).
input (Total) output
packets errs bytes packets errs bytes colls
82064 0 124238900 41033 0 2215718 0
82077 0 124263124 41039 0 2216142 0
82065 0 124244956 41033 0 2215818 0
82103 0 124302488 41052 0 2216836 0
82067 0 124247984 41035 0 2215872 0
81684 0 123668122 40842 0 2205558 0
82114 0 124319142 41057 0 2217122 0
Wow. 120,000 packets per second (In+Out)
If I reduce the mtu to 250 it can sustain around 210,000 pps (In+Out).
If I write a little program to do one-byte transactions.. i.e. send a
byte, wait for a byte back, repeat, I get around 16,000 transactions/sec
over localhost, 7100 transactions/sec over 100BaseTX, and oddly only
3400 transactions/sec over GigE.
The minimum round trip over 100BaseTX for a 64 byte packet is around
0.125 mS, which comes to around 8000 transactions/sec per connection,
so the 7100 number is reasonable.
The minimum round trip over my GigE seems to average around 0.230mS,
resulting in a theoretical transaction rate (per connection) of 4340 t/s.
The 3400 number seems a little low but looks reasonable.
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200112022132.fB2LWO093222>