Date: Wed, 11 Oct 2006 10:12:01 +0200 From: Heinrich Rebehn <rebehn@ant.uni-bremen.de> To: freebsd-stable@freebsd.org Subject: Re: NFS client slow on amd64 6.2-PRERELEASE #2 Message-ID: <452CA751.5060104@ant.uni-bremen.de> In-Reply-To: <20061005160819.GA13417@icarus.home.lan> References: <4523764E.3070309@ant.uni-bremen.de> <45252483.5040708@ant.uni-bremen.de> <20061005160819.GA13417@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
Jeremy Chadwick wrote:
> On Thu, Oct 05, 2006 at 05:28:03PM +0200, Heinrich Rebehn wrote:
>> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10
>> 10485760 bytes transferred in 4.967248 secs (2110980 bytes/sec)
>>
>> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M count=100
>> 104857600 bytes transferred in 69.020366 secs (1519227 bytes/sec)
>>
>> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10
>> 10485760 bytes transferred in 5.289492 secs (1982376 bytes/sec)
>>
>> root@antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M count=100
>> 104857600 bytes transferred in 58.715595 secs (1785856 bytes/sec)
>
> This isn't a valid test of performance, in my opinion. bs=1M is a
> bad idea. You shouldn't be using dd for this kind-of test at all.
> I should also note that bs=1M is not the same thing as obs=1M ibs=1M.
> dd's weird like that (someone can explain it, I'm sure -- I've seen
> it discussed in the pasT).
Ok, let's try cp:
root@antsrv1 [~] # mount_nfs -U -L antsrv2:/export/disk2 /mnt
root@antsrv1 [/tmp] # time cp 10MB.dat /mnt/x
real 0m6.884s
user 0m0.000s
sys 0m0.027s
root@antsrv1 [/tmp] # time cp 100MB.dat /mnt/x
real 1m8.744s
user 0m0.000s
sys 0m0.251s
Again, this yields less than 2MB/sec :-(
>
> scp is an ""acceptable"" real-life test (not the best, but it's
> legitimate), and you only did it from antsrv1 --> antsrv2, not the
> other direction. It's really too bad the OpenBSD guys refuse to
> incorporate the HP (high-performance) patches into OpenSSH, and
> being able to say "-c none" would *really* help when it comes to
> benchmarking network I/O via scp (in our case, we do dump over ssh
> across a segregated private LAN -- the encryption overhead slows
> our backups down to a crawl, and is worthless in our environment
> since as I said, segregated private LAN...)
So "-c none" would then have given even better performance compared to nfs.
>
> That said:
>
> I have seen cases where network peformance on BSD works fantastic
> uni-directionally -- for example, using FTP to "get" a file from
> a FreeBSD box on a 100mbit LAN results in a speed of about
> 300-400kbit/sec, while doing a "put" to the same box results in
> 90mbit/sec.
>
> The problem in that case turned out to be duplex-related. Both
> boxes were auto-negotiating with the Cisco switch correctly, and
> indeed the Cisco labelled them as auto-100/full, but as anyone who
> is familiar with Ciscos knows, auto-negotiation on Catalysts is
> far from reliable. Both boxes reported auto-neg and being at 100/full
> as well. I ended up hard-setting the boxes to use 100/full, and
> set the switch ports to 100/full, then rebooted both boxes (yes,
> this is sometimes required, as driver auto-neg code is a bit tweaky);
> voila, problem fixed.
>
> If you can connect these two boxes directly via a crossover cable,
> and you still see the problems, then yes, there's something definitely
> amiss that needs investigating. :-)
>
I hardcoded both machines to 1000baseTX full duplex and rebooted both:
root@antsrv1 [/tmp] # ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=b<RXCSUM,TXCSUM,VLAN_MTU>
inet 134.102.176.16 netmask 0xffffff00 broadcast 134.102.176.255
ether 00:04:23:d4:94:89
media: Ethernet 1000baseTX <full-duplex>
status: active
root@antsrv2 [/tmp] # ifconfig em0
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
options=b<RXCSUM,TXCSUM,VLAN_MTU>
inet 134.102.176.17 netmask 0xffffff00 broadcast 134.102.176.255
ether 00:0e:0c:ba:79:a8
media: Ethernet 1000baseTX <full-duplex>
status: active
The switch is a "dumb" LinkPro GB switch with nothing to configure.
How can i connect both boxes via a crossover cable and at the same time
connect them to our lan? ANTSRV1 is our production server that i cannot
take off line!
--
Heinrich Rebehn
University of Bremen
Physics / Electrical and Electronics Engineering
- Department of Telecommunications -
Phone : +49/421/218-4664
Fax : -3341
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?452CA751.5060104>