Date: Sat, 19 Aug 2000 17:30:58 -0600 (CST) From: Ryan Thompson <ryan@sasknow.com> To: freebsd-chat@freebsd.org Subject: Link speed on an SDSL provisioned T1 Message-ID: <Pine.BSF.4.21.0008191659420.59530-100000@ren.sasknow.com>
next in thread | raw e-mail | index | archive | help
Hi, everybody.. I thought some people might find this interesting.
I did some hard-core tests in terms of testing link speeds. I am on a
1.5Mbps SDSL link to my ISP in Saskatoon, SK, Canada. For the purposes of
this test, I only looked at download speeds (as upload speeds are
generally harder to test). I connected simultaneously to 6 FTP servers on
as many unique backbones as I could find, and backgrounded the tests
10-fold in parallel. Suffice to say, I saturated the link.
I pushed through approximately 125MB in total today, and about as much
early in the morning yesterday, doing these tests. I'll tell you how I
did them:
I used ipfw to record byte counts, with rules configured using the
following logic:
Pass anything on the loopback device
Pass anything on the local network
And (with a few other counting rules)
Count all traffic from anybody to LAN # Incoming total
Count all traffic from LAN to anybody # Outgoing total
Then follow the rest of my firewall rules. If BSD ipfw isn't fresh in
your mind, 'count' rules count all traffic that matches the rule, then
proceed to the next rule in sequence. Thus, a single packet can increment
many count rules, but only one 'allow' or 'deny' rule. I can show you my
actual ipfw ruleset if you like. Case in point, I have a reliable way to
monitor traffic to one server.
I then took every other interface on every other connected machine down
for the duration of the tests. Thus, with the exception of an
insignificant amount of RIP, ICMP, and misc traffic to our uplink gateway,
ALL traffic flow occurred between the test machine and the Internet.
mrtg statistics on the router very closely resemble my tests, below.
For the hell of it, despite the fact that FreeBSD comes with good default
network settings, I bumped the TCP send/receive windows past 32K (from
16K), and disabled TCP slowstart, for some of the trials. However, I
experienced no detectable difference in throughput. I have also confirmed
that the same 10/100 NIC can push and pull in excess of 75Mb/sec in local
transfers (probably more; the tests there were just single peer-to-peer
FTP transfers--lots of protocol and filesystem overhead), so I doubt that
it's a problem with my network setup, system load or hardware.
So, when I ran my ftp script and waited for all downloads to ramp up, I
executed commands like ipfw show | grep ^005 && sleep 60 && ipfw show |
grep ^005 (since all my count rules are in the 500's). Another thing to
note, here, is YES there was a small amount of other traffic occuring,
besides the FTP transfers... so we're not looking only for FTP transfer
rates, here. The ipfw rules catch EVERYTHING, and we want everything to
measure link speed.
THEN, I proceeded to subtract the differences in incoming traffic from
anybody to SaskNow. So, this will lead to the link speed over TCP, right?
Right. After running over 30 trials in two separate sessions (one during
peak, one during off-peak hours), I found the following.
KB = 1024 bytes
Kb = 1024 bits
MB = 1048576 bytes (1024 kilobytes)
Mb = 1048576 bits (1024 kilobits)
Average transferred per one-minute trial: 7195 +/- 21 KB **
/ 60 sec 119.9 +/- .35 KB/sec
* 8 Kbits/KB 959.3 +/- 2.8 Kb/sec
** The reliability of these statistics is astounding. The 10 most deviant
results follow:
7174 KB, 7180 KB, 7184 KB, 7186 KB, 7201 KB,
7191 KB, 7199 KB, 7198 KB, 7198 KB, 7193 KB
Thus, these stats are reliable to within (maximum tolerance) 0.2%, but
realistically, the average tolerance might be below 0.1%. I'm not a
statistician by trade, but I don't think one could ever hope for better.
Disregarding the average, let's take the highest result obtained of 7201
KB in 60 seconds. (Really, we're interested in the MAXIMUM transfer rate,
right?) Doing the conversions as above, that translates into 960.1
Kb/sec. Still, the difference is small.
OK, this a little far from the theoretical maximum of 1572 Kb/sec.
(which I never expect to reach ;-) However, as I mentioned above, this is
link speed over _TCP_. Figure in the TCP protocol overhead, as well as
overhead introduced by frame headers, and the recommended number is about
12% overhead. (Or 88% of the physical link speed). Thus, 1572 Kb/sec
looks more like 1367 Kb/sec for a maximum.
NOW, according to the router manufacturer's (Netopia's) specifications for
the 7100 series SDSL routers, 1.568Mbps is attainable to a maximum of
about 9,500 wire feet. Between 9,500 and 12,500ft, the link speed drops
to 1.04Mbps. See:
http://www.netopia.com/equipment/routers/r7100/r7100_faq.html#speed
Again taking the protocol and link overhead into account, but in reverse,
960 Kb/sec + 12% = 1.075 Kb/sec estimated "raw" transfer rate. 1075
Kb/sec * 1 Mb / 1024 Kb = 1.049 Mb/sec. This is within 1% of the maximum
rated link speed past 9,500ft.
Therefore, considering the 12% overhead is only an estimate, and given the
extremely low variance in the transfer rates, I have come to the following
conclusions:
1) I have successfully reached and sustained the downstream limit of the
connection
2) That limit is, in fact, the 1.04Mb/sec limit imposed by the router
at distances between 9,500ft and 12,500ft.
So, I am now confident that this problem is caused by too much copper
between us, or an improper setup, thereby reducing the rate.
I found that the street distance over our flat city between my ISP and our
office is about 5,900ft (1,800m). I had thought that this would be well
within range for a full 1.5Mbps. I suppose it is possible that the wire
distance is much greater than this, but I have no way to confirm or deny
that. I believe our telco likes bridged taps. ;-)
In any case, how cheated should I feel about only attaining 1Mbps as
opposed to 1.5?
So, besides moving next door to the ISP and stringing UTP through the
windows, is there anything that can be done to break out of this 1.04Mbps
limit?
Thanks!
- Ryan
--
Ryan Thompson <ryan@sasknow.com>
Systems Administrator, Accounts
Phone: +1 (306) 664-1161
SaskNow Technologies http://www.sasknow.com
#106-380 3120 8th St E Saskatoon, SK S7H 0W2
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-chat" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.21.0008191659420.59530-100000>