Date: Fri, 10 Dec 2004 18:05:27 -0500 (EST) From: Andrew Gallatin <gallatin@cs.duke.edu> To: Andre Oppermann <andre@freebsd.org> Cc: freebsd-net@freebsd.org Subject: Re: Rewritten TCP reassembly Message-ID: <16826.11191.630110.592646@grasshopper.cs.duke.edu> In-Reply-To: <41BA0088.9000107@freebsd.org> References: <41BA0088.9000107@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Andre Oppermann writes:
> I've totally rewritten the TCP reassembly function to be a lot more
> efficient. In tests with normal bw*delay products and packet loss
> plus severe reordering I've measured an improvment of at least 30% in
> performance. For high and very high bw*delay product links the
> performance improvement is most likely much higher.
>
I ran netperf with 20 times for each of 3 socket buffer sizes (128KB,
256KB, and 1MB) before and after patching. All tests were run with
net.isr.enable=1, and machdep.cpu_idle_hlt=0. CPU was pretty much
maxed thoughout. (SMP kernel, 1 HTT p4).
I'm not sure how much weight you should give these result. I still
haven't gotten around to fixing kttcp, so the memory copy overhead is
pretty high. For xmit, b/w goes from 2.90Gb/sec to 3.95Gb/sec when
sendfile is used. (linux receiver on same hw).
I've also not gotten around to making any of Robert's suggested driver
optimizations.
I noticed that at least systat was not printing the number of tcp
out-of-order packets after applying the patch. I forgot to check
netstat itself.
Drew
x before.131072
+ after.131072
+--------------------------------------------------------------------------+
| ++ x + x x|
|+ *+ + + ++ +xx++ x +x* ++ + xx+x x x + x x xxxx|
| |_____________AM|___________|_______AM__________________| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 20 3336.63 3516.95 3449.87 3447.838 52.708815
+ 20 3323 3466.34 3391.97 3389.686 37.154513
Difference at 95.0% confidence
-58.152 +/- 29.1859
-1.68662% +/- 0.846499%
(Student's t, pooled s = 45.5998)
x before.262144
+ after.262144
+--------------------------------------------------------------------------+
| + |
| + x |
|+ ++ + ++ x x x |
|+ + ++++++++ + + x x x x x xx xxx xxxx x x|
| |______A______| |_______AM_______| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 20 3197.77 3421.67 3316.64 3314.8555 59.787371
+ 20 2889.64 3122.17 2965.01 2964.9865 50.076867
Difference at 95.0% confidence
-349.869 +/- 35.2961
-10.5546% +/- 1.06479%
(Student's t, pooled s = 55.1463)
x before.1048576
+ after.1048576
+--------------------------------------------------------------------------+
| x + |
|xx x + x xx + x+x++x * x* x+*x+x+* x*+ + + + ++|
| |_________________|A____M__________A__M____________| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 20 2658.05 2732.45 2706.91 2700.1955 23.318833
+ 20 2676.13 2748.97 2724.07 2719.9665 20.531036
Difference at 95.0% confidence
19.771 +/- 14.0613
0.732206% +/- 0.52075%
(Student's t, pooled s = 21.9692)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?16826.11191.630110.592646>