Date: Fri, 08 Jul 2005 00:23:07 +0100 From: Alex Zbyslaw <xfb52@dial.pipex.com> To: freebsd-questions <freebsd-questions@freebsd.org> Subject: SSH and gigabit NICs Message-ID: <42CDB95B.3030703@dial.pipex.com>
next in thread | raw e-mail | index | archive | help
The setup:
Both machines FreeBSD/i386 5.4
a) AMD64 machine with on-board Marvel Gigabit NIC
b) Athlon XP with cheap SMC Gigabit NIC (also Marvel)
Cabling is brand new Cat5e. Have tried various different cables of=20
different lengths to no effect.
To rule out problems with a cheap switch, I have just wired the NICs=20
together.
To benchmark, I had a huge bz2 file (430Mb) which I copied with scp and=20
ftp from machine a to machine b.
On cheap NetGear 100Mb cards, the transfers both took ~40 seconds which=20
is ~80Mbit.
On the new Gigabit hardware, ftp drops to 17 seconds, but scp takes=20
longer! Out of the box (no tweaking of any relevant parameters) it now=20
takes over 53 seconds.
After tweaking tons of stuff, I can make scp take maybe 43 seconds, but=20
at those settings, ftp takes well over a minute!
What is going on? I know that 17 seconds for the ftp is hardly stellar=20
(200+Mbit or so) but for =A350 I could live with that. But for ssh to ge=
t=20
slower just boggles. These days, almost anything you do over a network=20
ends up using ssh -- specifically I was hoping to make rsyncs faster --=20
but for them to get slower?
I've seen odd ssh network behaviour on other boxen. A couple old Linux=20
servers were 2-3 times slower for ssh than ftp, but I put it down to=20
oldness and Linux and general weirdness. They were on a 100Mbit network.=
When I monitor with "systat -ifstat" I can see ftp keeping up a=20
reasonably regular transfer rate, but when I watch the ssh, it yoyos up=20
and down wildly, but still never gets above about 80Mbit. Both machines =
have plenty of idle CPU and the ssh is not compressed.
Does anyone have a clue what might be going on?
So far I have tried:
HZ=3D1000 on both machines. No effect.
various net.inet.tcp.recvspace and net.inet.tcp.recvspace values on=20
both machines. About 4096 (down from the default of 32768) makes ssh=20
work "best", but stuffs ftp. 65536 improves ftp a bit but ssh goes up=20
to 53 seconds (~64Mbit).
MTU values of 5-9000. ~6000 the scp seems to start a bit faster=20
(maybe 100Mbit) but soon drops back into the 60s.
Before anyone asks, the driver doesn't seem to support polling.
--Alex, baffled and really quite annoyed.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?42CDB95B.3030703>