From owner-freebsd-stable@FreeBSD.ORG Fri Sep 26 13:35:19 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0EF0C106569D; Fri, 26 Sep 2008 13:35:19 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from cs1.cs.huji.ac.il (cs1.cs.huji.ac.il [132.65.16.10]) by mx1.freebsd.org (Postfix) with ESMTP id B6B0F8FC27; Fri, 26 Sep 2008 13:35:18 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by cs1.cs.huji.ac.il with esmtp id 1KjDTZ-000Jqb-Bh; Fri, 26 Sep 2008 16:35:17 +0300 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Jeremy Chadwick In-reply-to: <20080926095230.GA20789@icarus.home.lan> References: <20080926081806.GA19055@icarus.home.lan> <20080926095230.GA20789@icarus.home.lan> Comments: In-reply-to Jeremy Chadwick message dated "Fri, 26 Sep 2008 02:52:30 -0700." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 26 Sep 2008 16:35:17 +0300 From: Danny Braniss Message-ID: Cc: freebsd-hackers@freebsd.org, freebsd-stable@freebsd.org Subject: Re: bad NFS/UDP performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Sep 2008 13:35:19 -0000 > On Fri, Sep 26, 2008 at 12:27:08PM +0300, Danny Braniss wrote: > > > On Fri, Sep 26, 2008 at 10:04:16AM +0300, Danny Braniss wrote: > > > > Hi, > > > > There seems to be some serious degradation in performance. > > > > Under 7.0 I get about 90 MB/s (on write), while, on the same machine > > > > under 7.1 it drops to 20! > > > > Any ideas? > > > > > > 1) Network card driver changes, > > could be, but at least iperf/tcp is ok - can't get udp numbers, do you > > know of any tool to measure udp performance? > > BTW, I also checked on different hardware, and the badness is there. > > According to INDEX, benchmarks/iperf does UDP bandwidth testing. I know, but I get about 1mgb, which seems somewhat low :-( > > benchmarks/nttcp should as well. > > What network card is in use? If Intel, what driver version (should be > in dmesg). bge: and bce: and intels, but haven't tested there yet. > > > > 2) This could be relevant, but rwatson@ will need to help determine > > > that. > > > http://lists.freebsd.org/pipermail/freebsd-stable/2008-September/045109.html > > > > gut feeling is that it's somewhere else: > > > > Writing 16 MB file > > BS Count /---- 7.0 ------/ /---- 7.1 -----/ > > 1*512 32768 0.16s 98.11MB/s 0.43s 37.18MB/s > > 2*512 16384 0.17s 92.04MB/s 0.46s 34.79MB/s > > 4*512 8192 0.16s 101.88MB/s 0.43s 37.26MB/s > > 8*512 4096 0.16s 99.86MB/s 0.44s 36.41MB/s > > 16*512 2048 0.16s 100.11MB/s 0.50s 32.03MB/s > > 32*512 1024 0.26s 61.71MB/s 0.46s 34.79MB/s > > 64*512 512 0.22s 71.45MB/s 0.45s 35.41MB/s > > 128*512 256 0.21s 77.84MB/s 0.51s 31.34MB/s > > 256*512 128 0.19s 82.47MB/s 0.43s 37.22MB/s > > 512*512 64 0.18s 87.77MB/s 0.49s 32.69MB/s > > 1024*512 32 0.18s 89.24MB/s 0.47s 34.02MB/s > > 2048*512 16 0.17s 91.81MB/s 0.30s 53.41MB/s > > 4096*512 8 0.16s 100.56MB/s 0.42s 38.07MB/s > > 8192*512 4 0.82s 19.56MB/s 0.80s 19.95MB/s > > 16384*512 2 0.82s 19.63MB/s 0.95s 16.80MB/s > > 32768*512 1 0.81s 19.69MB/s 0.96s 16.64MB/s > > > > Average: 75.86 33.00 > > > > the nfs filer is a NetWork Appliance, and is in use, so i get fluctuations in > > the > > measurements, but the relation are similar, good on 7.0, bad on 7.1 > > Do you have any NFS-related tunings in /etc/rc.conf or /etc/sysctl.conf? > no, but diffing the sysctl show: -vfs.nfs.realign_test: 22141777 +vfs.nfs.realign_test: 498351 -vfs.nfsrv.realign_test: 5005908 +vfs.nfsrv.realign_test: 0 +vfs.nfsrv.commit_miss: 0 +vfs.nfsrv.commit_blks: 0 changing them did nothing - or at least with respect to nfs throughput :-) danny