Date: Fri, 20 Apr 2007 11:32:03 -0700 From: "Kevin Oberman" <oberman@es.net> To: Jeremy Chadwick <koitsu@FreeBSD.org> Cc: Sven Willenberger <sven@dmv.com>, freebsd-stable@freebsd.org Subject: Re: CARP and em0 timeout watchdog Message-ID: <20070420183203.053714506A@ptavv.es.net> In-Reply-To: Your message of "Fri, 20 Apr 2007 09:04:31 PDT." <20070420160431.GA17356@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
--==_Exmh_1177093923_52974P
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
> Date: Fri, 20 Apr 2007 09:04:31 -0700
> From: Jeremy Chadwick <koitsu@FreeBSD.org>
> Sender: owner-freebsd-stable@freebsd.org
>
> On Fri, Apr 20, 2007 at 11:51:56AM -0400, Sven Willenberger wrote:
> > Having done more diagnostics I have found out it is not CARP related at
> > all. It turns out that the same timeouts will happen when ftp'ing to the
> > physical address IPs as well. There is also an odd situation here
> > depending on which protocol I use. The two boxes are connected to a Dell
> > Powerconnect 2616 gig switch with CAT6. If I scp files from the
> > 192.168.0.18 to the 192.168.0.19 box I can transfer gigs worth without a
> > hiccup (I used dd to create various sized testfiles from 32M to 1G in
> > size and just scp testfile* to the other box). On the other hand, if I
> > connect to 192.168.0.19 using ftp (either active or passive) where ftp
> > is being run through inetd, the interface resets (watchdog) within
> > seconds (a few MBs) of traffic. Enabling polling does nothing, nor does
> > changing net.inet.tcp.{recv,send}space. Any ideas why I would be seeing
> > such behavioral differences between scp and ftp?
>
> You'll get a much higher throughput rate with FTP than you will with
> SSH, simply because encryption overhead is quite high (even with the
> Blowfish cipher). With a very fast processor and on a gigE network
> you'll probably see 8-9MByte/sec via SSH while 60-70MByte/sec via FTP.
> That's the only difference I can think of.
OK. Let's put the blame where it belongs. It's probably not the
encryption/decryption that slows down scp. It's the OpenSSH code. It is
only slightly related to CPU speed on reasonably modern CPUs. My Athlon
64 system goes to 23% CPU while transferring a large (150MB) file using
AES128-CBC. My Ethernet runs at over 11 MBytes/sec on a FastEthernet
about 5 nanoseconds long.
If you have a system slower than about 600 MHz, then it may be the
encryption.
At least 3 years ago the folks at the Pittsburgh Supercomputer Center
(PSC) were seeing slow scp performance and investigated. The systems
they were running on were pretty fast (it is a Supercomputer Center) and
should have been able to run at nearly 1 Gbps without problems, but
could not. FTP (which is a VERY inefficient protocol) was much faster.
They examined the OpenSSH source code and found the problem. They
published patches to OpenSSH and have continued to maintain them, but
the OpenBSD people have yet to incorporate them, so ssh is still slow on
long paths. This only applies to transfers over longer distances.
Transfers over the LAN should not be impacted by this.
More information and the patch are available at:
http://www.psc.edu/networking/projects/hpn-ssh/
--
R. Kevin Oberman, Network Engineer
Energy Sciences Network (ESnet)
Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
E-mail: oberman@es.net Phone: +1 510 486-8634
Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751
--==_Exmh_1177093923_52974P
Content-Type: application/pgp-signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (FreeBSD)
Comment: Exmh version 2.5 06/03/2002
iD8DBQFGKQcikn3rs5h7N1ERAp4nAJ0TR/u3zzA/FDfUz7SeWKpb+GPCdwCgiSw5
LUHGnWSIHqAzPd1p59zjzto=
=7j9b
-----END PGP SIGNATURE-----
--==_Exmh_1177093923_52974P--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070420183203.053714506A>