From owner-freebsd-net@FreeBSD.ORG Wed Jan 29 23:01:45 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89D9AE0D; Wed, 29 Jan 2014 23:01:45 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 27134108D; Wed, 29 Jan 2014 23:01:44 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEAIiH6VKDaFve/2dsb2JhbABZg0RWgn65QU+BGXSCJQEBAQMBAQEBIAQnHQECCwUWGAICDRkCKQEJJgYIBwQBGQMEh1wIDaploEYXgSmNBQEBGzQHgm+BSQSJSYp3gRWEBZBtg0seMYEEOQ X-IronPort-AV: E=Sophos;i="4.95,744,1384318800"; d="scan'208";a="91674231" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 29 Jan 2014 18:01:43 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id A3442B40A3; Wed, 29 Jan 2014 18:01:43 -0500 (EST) Date: Wed, 29 Jan 2014 18:01:43 -0500 (EST) From: Rick Macklem To: J David Message-ID: <1352428787.18632865.1391036503658.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: Terrible NFS performance under 9.2-RELEASE? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-net@freebsd.org, Garrett Wollman , Bryan Venteicher X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jan 2014 23:01:45 -0000 J David wrote: > On Tue, Jan 28, 2014 at 7:32 PM, Rick Macklem > wrote: > > Hopefully Garrett and/or you will be able to do some testing of it > > and report back w.r.t. performance gains, etc. > > OK, it has seen light testing. > > As predicted the vtnet drops are eliminated and CPU load is reduced. > Ok, that's good news. Bryan, is increasing VTNET_MAX_TX_SEGS in the driver feasible? However, I do suspect we'll be putting a refined version of the patch in head someday (maybe April, sooner would have to be committed by someone else). I suspect that Garrett's code for server read will work well and I'll cobble something to-gether for server readdir and client write. > The performance is also improved: > > Test Before After > SeqWr 1506 7461 > SeqRd 566 192015 > RndRd 602 218730 > RndWr 44 13972 > > All numbers in kiB/sec. > If you get the chance, you can try a few tunables on the server. vfs.nfsd.fha.enable=0 - ken@ found that FHA was necessary for ZFS exports, to avoid out of order reads from confusing ZFS's sequential reading heuristic. However, FHA also means that all readaheads for a file are serialized with the reads for the file (same fh->same nfsd thread). Somehow, it seems to me that doing reads concurrently in the server (given shared vnode locks) could be a good thing. --> I wonder what the story is for UFS? So, it would be interesting to see what disabling FHA does for the sequential read test. I think I already mentioned the DRC cache ones: vfs.nfsd.tcphighwater=100000 vfs.nfsd.tcpcachetimeo=600 (actually I think Garrett uses 300) Good to see some progress, rick ps: Daniel reports that he will be able to test the patch this weekend, to see if it fixes his problem that required TSO to be disabled, so we'll wait and see. > There were initially still some problems with lousy hostcache values > on the client after the test, which is what causes the iperf > performance to tank after the NFS test, but after a reboot of both > sides and fresh retest, I haven't reproduced that again. If it comes > back, I'll try to figure out what's going on. > Hopefully a networking type might know what is going on, because this is way out of my area of expertise. > But this definitely looks like a move in the right direction. > > Thanks! > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >