From owner-freebsd-questions@FreeBSD.ORG Thu Jul 7 23:23:10 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AC0AA16A41C for ; Thu, 7 Jul 2005 23:23:10 +0000 (GMT) (envelope-from xfb52@dial.pipex.com) Received: from smtp-out5.blueyonder.co.uk (smtp-out5.blueyonder.co.uk [195.188.213.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 359FE43D46 for ; Thu, 7 Jul 2005 23:23:09 +0000 (GMT) (envelope-from xfb52@dial.pipex.com) Received: from [82.41.37.55] ([82.41.37.55]) by smtp-out5.blueyonder.co.uk with Microsoft SMTPSVC(5.0.2195.6713); Fri, 8 Jul 2005 00:23:51 +0100 Message-ID: <42CDB95B.3030703@dial.pipex.com> Date: Fri, 08 Jul 2005 00:23:07 +0100 From: Alex Zbyslaw User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-GB; rv:1.7.8) Gecko/20050530 X-Accept-Language: en, en-us, pl MIME-Version: 1.0 To: freebsd-questions Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable X-OriginalArrivalTime: 07 Jul 2005 23:23:51.0600 (UTC) FILETIME=[EF53EB00:01C5834A] Subject: SSH and gigabit NICs X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jul 2005 23:23:10 -0000 The setup: Both machines FreeBSD/i386 5.4 a) AMD64 machine with on-board Marvel Gigabit NIC b) Athlon XP with cheap SMC Gigabit NIC (also Marvel) Cabling is brand new Cat5e. Have tried various different cables of=20 different lengths to no effect. To rule out problems with a cheap switch, I have just wired the NICs=20 together. To benchmark, I had a huge bz2 file (430Mb) which I copied with scp and=20 ftp from machine a to machine b. On cheap NetGear 100Mb cards, the transfers both took ~40 seconds which=20 is ~80Mbit. On the new Gigabit hardware, ftp drops to 17 seconds, but scp takes=20 longer! Out of the box (no tweaking of any relevant parameters) it now=20 takes over 53 seconds. After tweaking tons of stuff, I can make scp take maybe 43 seconds, but=20 at those settings, ftp takes well over a minute! What is going on? I know that 17 seconds for the ftp is hardly stellar=20 (200+Mbit or so) but for =A350 I could live with that. But for ssh to ge= t=20 slower just boggles. These days, almost anything you do over a network=20 ends up using ssh -- specifically I was hoping to make rsyncs faster --=20 but for them to get slower? I've seen odd ssh network behaviour on other boxen. A couple old Linux=20 servers were 2-3 times slower for ssh than ftp, but I put it down to=20 oldness and Linux and general weirdness. They were on a 100Mbit network.= When I monitor with "systat -ifstat" I can see ftp keeping up a=20 reasonably regular transfer rate, but when I watch the ssh, it yoyos up=20 and down wildly, but still never gets above about 80Mbit. Both machines = have plenty of idle CPU and the ssh is not compressed. Does anyone have a clue what might be going on? So far I have tried: HZ=3D1000 on both machines. No effect. various net.inet.tcp.recvspace and net.inet.tcp.recvspace values on=20 both machines. About 4096 (down from the default of 32768) makes ssh=20 work "best", but stuffs ftp. 65536 improves ftp a bit but ssh goes up=20 to 53 seconds (~64Mbit). MTU values of 5-9000. ~6000 the scp seems to start a bit faster=20 (maybe 100Mbit) but soon drops back into the 60s. Before anyone asks, the driver doesn't seem to support polling. --Alex, baffled and really quite annoyed.