Date: Sun, 19 Jul 2009 22:52:21 +0200 From: Marc "UBM" Bocklet <ubm@u-boot-man.de> To: freebsd-current@freebsd.org Subject: Really slow gigabit link in one direction (em -> sk) Message-ID: <20090719225221.cf9d8417.ubm@u-boot-man.de>
index | next in thread | raw e-mail
Hiho! :-)
Transfer speed from the em adapter in our fileserver to another machine
(sk adapter) is really slow (measurements taken with iperf without
options):
------------------------------------------------------------
Client connecting to 192.168.0.23, TCP port 5001
TCP window size: 32.5 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.0.222 port 52856 connected with 192.168.0.23 port
5001 [ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.2 sec 181 MBytes 149 Mbits/sec
On the other hand, speed from the sk adapter to em is acceptable (for
an untuned connection and a possibly crappy cable, imho):
------------------------------------------------------------
Client connecting to 192.168.0.222, TCP port 5001
TCP window size: 32.5 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.0.23 port 32579 connected with 192.168.0.222 port
5001 [ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 505 MBytes 423 Mbits/sec
Cable is about 15m long, cat-5. On the receiving end sits a Marvell sk
gigabit card. I've tried setting the mtu on both sides to 9000 and to
other values as well, but it makes no difference. Neither does
increasing
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
to 8MB, as some sources suggested (actually, it does make a difference,
it nets ca. 100Mbit more in the sk->em direction, but does nothing for
the em->sk direction). Am I doing something wrong? Should I burn my
cable? :-)
Additional info for the machine containing the sk card:
skc0@pci0:2:8:0: class=0x020000 card=0xc2311297 chip=0x432011ab
rev=0x13 hdr=0x00 vendor = 'Marvell Semiconductor (Was: Galileo
Technology Ltd)'
device = 'Yukon 88E8001/8003/8010 PCI Gigabit
Ethernet Controller (Copper)'
class = network
subclass = ethernet
sk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500 options=b<RXCSUM,TXCSUM,VLAN_MTU>
ether 00:30:1b:b5:fb:d9
inet 192.168.0.23 netmask 0xffffff00 broadcast 192.168.0.255
media: Ethernet autoselect (1000baseT <full-duplex,flag0,flag1>)
status: active
net.inet.tcp.mssdflt: 512
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
net.inet.tcp.keepinit: 75000
net.inet.tcp.delacktime: 100
net.inet.tcp.v6mssdflt: 1024
net.inet.tcp.hostcache.purge: 0
net.inet.tcp.hostcache.prune: 300
net.inet.tcp.hostcache.expire: 3600
net.inet.tcp.hostcache.count: 310
net.inet.tcp.hostcache.bucketlimit: 30
net.inet.tcp.hostcache.hashsize: 512
net.inet.tcp.hostcache.cachelimit: 15360
net.inet.tcp.wlock_looped: 0
net.inet.tcp.wlock_relocked: 0
net.inet.tcp.wlock_upgraded: 11246
net.inet.tcp.tcp_wlock_atfirst: 19669
net.inet.tcp.rlock_atfirst: 4875339
net.inet.tcp.read_locking: 1
net.inet.tcp.recvbuf_max: 16777216
net.inet.tcp.recvbuf_inc: 16384
net.inet.tcp.recvbuf_auto: 1
net.inet.tcp.insecure_rst: 0
net.inet.tcp.ecn.maxretries: 1
net.inet.tcp.ecn.enable: 0
net.inet.tcp.abc_l_var: 2
net.inet.tcp.rfc3465: 1
net.inet.tcp.rfc3390: 1
net.inet.tcp.rfc3042: 1
net.inet.tcp.drop_synfin: 0
net.inet.tcp.delayed_ack: 1
net.inet.tcp.blackhole: 0
net.inet.tcp.log_in_vain: 0
net.inet.tcp.sendbuf_max: 16777216
net.inet.tcp.sendbuf_inc: 8192
net.inet.tcp.sendbuf_auto: 1
net.inet.tcp.tso: 1
net.inet.tcp.newreno: 1
net.inet.tcp.local_slowstart_flightsize: 4
net.inet.tcp.slowstart_flightsize: 1
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.reass.overflows: 2610
net.inet.tcp.reass.maxqlen: 48
net.inet.tcp.reass.cursegments: 2
net.inet.tcp.reass.maxsegments: 1600
net.inet.tcp.sack.globalholes: 0
net.inet.tcp.sack.globalmaxholes: 65536
net.inet.tcp.sack.maxholes: 128
net.inet.tcp.sack.enable: 1
net.inet.tcp.inflight.stab: 20
net.inet.tcp.inflight.max: 1073725440
net.inet.tcp.inflight.min: 6144
net.inet.tcp.inflight.rttthresh: 10
net.inet.tcp.inflight.debug: 0
net.inet.tcp.inflight.enable: 1
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.pcbcount: 96
net.inet.tcp.do_tcpdrain: 1
net.inet.tcp.tcbhashsize: 512
net.inet.tcp.log_debug: 0
net.inet.tcp.minmss: 216
net.inet.tcp.syncache.rst_on_sock_fail: 1
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.syncache.hashsize: 512
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.cachelimit: 15360
net.inet.tcp.syncache.bucketlimit: 30
net.inet.tcp.syncookies_only: 0
net.inet.tcp.syncookies: 1
net.inet.tcp.timer_race: 0
net.inet.tcp.finwait2_timeout: 60000
net.inet.tcp.fast_finwait2_recycle: 0
net.inet.tcp.always_keepalive: 1
net.inet.tcp.rexmit_slop: 200
net.inet.tcp.rexmit_min: 30
net.inet.tcp.msl: 30000
net.inet.tcp.nolocaltimewait: 0
net.inet.tcp.maxtcptw: 5120
Additional info for the computer with the em card:
em0@pci0:1:7:0: class=0x020000 card=0x13768086 chip=0x107c8086 rev=0x05
hdr=0x00 vendor = 'Intel Corporation'
device = 'Gigabit Ethernet Controller (Copper) rev 5 (82541PI)'
class = network
subclass = ethernet
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
9000 options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
ether 00:1b:21:17:2e:5d
inet 192.168.0.222 netmask 0xffffff00 broadcast 192.168.0.255
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
net.inet.tcp.rfc1323: 1
net.inet.tcp.mssdflt: 512
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
net.inet.tcp.keepinit: 75000
net.inet.tcp.delacktime: 100
net.inet.tcp.v6mssdflt: 1024
net.inet.tcp.hostcache.purge: 0
net.inet.tcp.hostcache.prune: 300
net.inet.tcp.hostcache.expire: 3600
net.inet.tcp.hostcache.count: 1
net.inet.tcp.hostcache.bucketlimit: 30
net.inet.tcp.hostcache.hashsize: 512
net.inet.tcp.hostcache.cachelimit: 15360
net.inet.tcp.wlock_looped: 0
net.inet.tcp.wlock_relocked: 0
net.inet.tcp.wlock_upgraded: 281
net.inet.tcp.tcp_wlock_atfirst: 296
net.inet.tcp.rlock_atfirst: 1608326
net.inet.tcp.read_locking: 1
net.inet.tcp.recvbuf_max: 16777216
net.inet.tcp.recvbuf_inc: 16384
net.inet.tcp.recvbuf_auto: 1
net.inet.tcp.insecure_rst: 0
net.inet.tcp.ecn.maxretries: 1
net.inet.tcp.ecn.enable: 0
net.inet.tcp.abc_l_var: 2
net.inet.tcp.rfc3465: 1
net.inet.tcp.rfc3390: 1
net.inet.tcp.rfc3042: 1
net.inet.tcp.drop_synfin: 0
net.inet.tcp.delayed_ack: 1
net.inet.tcp.blackhole: 0
net.inet.tcp.log_in_vain: 0
net.inet.tcp.sendbuf_max: 16777216
net.inet.tcp.sendbuf_inc: 8192
net.inet.tcp.sendbuf_auto: 1
net.inet.tcp.tso: 1
net.inet.tcp.newreno: 1
net.inet.tcp.local_slowstart_flightsize: 4
net.inet.tcp.slowstart_flightsize: 1
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.reass.overflows: 0
net.inet.tcp.reass.maxqlen: 48
net.inet.tcp.reass.cursegments: 0
net.inet.tcp.reass.maxsegments: 1600
net.inet.tcp.sack.globalholes: 0
net.inet.tcp.sack.globalmaxholes: 65536
net.inet.tcp.sack.maxholes: 128
net.inet.tcp.sack.enable: 1
net.inet.tcp.inflight.stab: 20
net.inet.tcp.inflight.max: 1073725440
net.inet.tcp.inflight.min: 6144
net.inet.tcp.inflight.rttthresh: 10
net.inet.tcp.inflight.debug: 0
net.inet.tcp.inflight.enable: 1
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.pcbcount: 11
net.inet.tcp.do_tcpdrain: 1
net.inet.tcp.tcbhashsize: 512
net.inet.tcp.log_debug: 0
net.inet.tcp.minmss: 216
net.inet.tcp.syncache.rst_on_sock_fail: 1
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.syncache.hashsize: 512
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.cachelimit: 15360
net.inet.tcp.syncache.bucketlimit: 30
net.inet.tcp.syncookies_only: 0
net.inet.tcp.syncookies: 1
net.inet.tcp.timer_race: 0
net.inet.tcp.finwait2_timeout: 60000
net.inet.tcp.fast_finwait2_recycle: 0
net.inet.tcp.always_keepalive: 1
net.inet.tcp.rexmit_slop: 200
net.inet.tcp.rexmit_min: 30
net.inet.tcp.msl: 30000
net.inet.tcp.nolocaltimewait: 0
net.inet.tcp.maxtcptw: 5120
Bye
Marc
--
"And what rough beast, its hour come round at last,
Slouches towards Bethlehem to be born?"
W.B. Yeats, The Second Coming
help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090719225221.cf9d8417.ubm>