From owner-freebsd-stable@FreeBSD.ORG Wed Oct 11 08:12:03 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7512416A407 for ; Wed, 11 Oct 2006 08:12:03 +0000 (UTC) (envelope-from rebehn@ant.uni-bremen.de) Received: from antsrv1.ant.uni-bremen.de (antsrv1.ant.uni-bremen.de [134.102.176.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 02D7D43D53 for ; Wed, 11 Oct 2006 08:12:02 +0000 (GMT) (envelope-from rebehn@ant.uni-bremen.de) Received: from bremerhaven.ant.uni-bremen.de ([134.102.176.10]) by antsrv1.ant.uni-bremen.de with esmtp (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GXZC4-0000R6-Tt; Wed, 11 Oct 2006 10:12:01 +0200 Message-ID: <452CA751.5060104@ant.uni-bremen.de> Date: Wed, 11 Oct 2006 10:12:01 +0200 From: Heinrich Rebehn User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060910 SeaMonkey/1.0.5 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4523764E.3070309@ant.uni-bremen.de> <45252483.5040708@ant.uni-bremen.de> <20061005160819.GA13417@icarus.home.lan> In-Reply-To: <20061005160819.GA13417@icarus.home.lan> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: NFS client slow on amd64 6.2-PRERELEASE #2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Oct 2006 08:12:03 -0000 Jeremy Chadwick wrote: > On Thu, Oct 05, 2006 at 05:28:03PM +0200, Heinrich Rebehn wrote: >> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10 >> 10485760 bytes transferred in 4.967248 secs (2110980 bytes/sec) >> >> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M count=100 >> 104857600 bytes transferred in 69.020366 secs (1519227 bytes/sec) >> >> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10 >> 10485760 bytes transferred in 5.289492 secs (1982376 bytes/sec) >> >> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M count=100 >> 104857600 bytes transferred in 58.715595 secs (1785856 bytes/sec) > > This isn't a valid test of performance, in my opinion. bs=1M is a > bad idea. You shouldn't be using dd for this kind-of test at all. > I should also note that bs=1M is not the same thing as obs=1M ibs=1M. > dd's weird like that (someone can explain it, I'm sure -- I've seen > it discussed in the pasT). Ok, let's try cp: root@antsrv1 [~] # mount_nfs -U -L antsrv2:/export/disk2 /mnt root@antsrv1 [/tmp] # time cp 10MB.dat /mnt/x real 0m6.884s user 0m0.000s sys 0m0.027s root@antsrv1 [/tmp] # time cp 100MB.dat /mnt/x real 1m8.744s user 0m0.000s sys 0m0.251s Again, this yields less than 2MB/sec :-( > > scp is an ""acceptable"" real-life test (not the best, but it's > legitimate), and you only did it from antsrv1 --> antsrv2, not the > other direction. It's really too bad the OpenBSD guys refuse to > incorporate the HP (high-performance) patches into OpenSSH, and > being able to say "-c none" would *really* help when it comes to > benchmarking network I/O via scp (in our case, we do dump over ssh > across a segregated private LAN -- the encryption overhead slows > our backups down to a crawl, and is worthless in our environment > since as I said, segregated private LAN...) So "-c none" would then have given even better performance compared to nfs. > > That said: > > I have seen cases where network peformance on BSD works fantastic > uni-directionally -- for example, using FTP to "get" a file from > a FreeBSD box on a 100mbit LAN results in a speed of about > 300-400kbit/sec, while doing a "put" to the same box results in > 90mbit/sec. > > The problem in that case turned out to be duplex-related. Both > boxes were auto-negotiating with the Cisco switch correctly, and > indeed the Cisco labelled them as auto-100/full, but as anyone who > is familiar with Ciscos knows, auto-negotiation on Catalysts is > far from reliable. Both boxes reported auto-neg and being at 100/full > as well. I ended up hard-setting the boxes to use 100/full, and > set the switch ports to 100/full, then rebooted both boxes (yes, > this is sometimes required, as driver auto-neg code is a bit tweaky); > voila, problem fixed. > > If you can connect these two boxes directly via a crossover cable, > and you still see the problems, then yes, there's something definitely > amiss that needs investigating. :-) > I hardcoded both machines to 1000baseTX full duplex and rebooted both: root@antsrv1 [/tmp] # ifconfig em0 em0: flags=8843 mtu 1500 options=b inet 134.102.176.16 netmask 0xffffff00 broadcast 134.102.176.255 ether 00:04:23:d4:94:89 media: Ethernet 1000baseTX status: active root@antsrv2 [/tmp] # ifconfig em0 em0: flags=8843 mtu 1500 options=b inet 134.102.176.17 netmask 0xffffff00 broadcast 134.102.176.255 ether 00:0e:0c:ba:79:a8 media: Ethernet 1000baseTX status: active The switch is a "dumb" LinkPro GB switch with nothing to configure. How can i connect both boxes via a crossover cable and at the same time connect them to our lan? ANTSRV1 is our production server that i cannot take off line! -- Heinrich Rebehn University of Bremen Physics / Electrical and Electronics Engineering - Department of Telecommunications - Phone : +49/421/218-4664 Fax : -3341