From owner-freebsd-current@FreeBSD.ORG Thu Nov 12 16:55:22 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB3BF106566C; Thu, 12 Nov 2009 16:55:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 128098FC17; Thu, 12 Nov 2009 16:55:21 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApoEAO7M+0qDaFvI/2dsb2JhbADdaYQ8BIFs X-IronPort-AV: E=Sophos;i="4.44,729,1249272000"; d="scan'208";a="55184341" Received: from darling.cs.uoguelph.ca ([131.104.91.200]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 12 Nov 2009 11:55:10 -0500 Received: from localhost (localhost.localdomain [127.0.0.1]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id 2FE6D940179; Thu, 12 Nov 2009 11:55:10 -0500 (EST) X-Virus-Scanned: amavisd-new at darling.cs.uoguelph.ca Received: from darling.cs.uoguelph.ca ([127.0.0.1]) by localhost (darling.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ejJiy5HqNdyL; Thu, 12 Nov 2009 11:55:09 -0500 (EST) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id F3E79940020; Thu, 12 Nov 2009 11:55:08 -0500 (EST) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id nACH2ng15010; Thu, 12 Nov 2009 12:02:49 -0500 (EST) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Thu, 12 Nov 2009 12:02:49 -0500 (EST) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Ivan Voras In-Reply-To: <9bbcef730911111352t12188bdajbca71bcf35a5beb5@mail.gmail.com> Message-ID: References: <11167f520911111050j36dd94far667c81e6f5c18e69@mail.gmail.com> <20091111204903.GI89052@dan.emsphone.com> <11167f520911111326v13bb442bt36e853afbecdf834@mail.gmail.com> <9bbcef730911111352t12188bdajbca71bcf35a5beb5@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: "Sam Fourman Jr." , freebsd-current@freebsd.org, Dan Nelson Subject: Re: Help ZFS FreeBSD 8.0 RC2 Write performance issue X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 16:55:22 -0000 On Wed, 11 Nov 2009, Ivan Voras wrote: > > I think NFS uses sync disk IO access by default, this may be your > problem if you are write-heavy. Try setting vfs.nfsrv.async to 1 to > see if this is the cause of your problems. Just fyi, I took a quick look and I don't think this will be a good idea for NFSv3. (It allows the server to cheat for NFSv2 and avoid synchronous writes, which was contrary to the standard, but became fashionable for performance reasons, before NFSv3 came out.) For NFSv3, the writes are normally done asynchronously, followed by a Commit RPC to force them to disk, done by the client when it is flushing its buffer cache. When you set this sysctl, the NFSv3 server (sys/nfsserver, not the experimental one), all it does is reply to the write RPC that it has already been committed. This may have two effects, depending upon the client: 1 - The client may then choose to not bother with a Commit RPC. --> This shouldn't help performance much, because the data will normally have made to disk by now. *** This might be an interesting experiment to try on a ZFS server though, since if it does make a significant difference, it suggests that ZFS does a lot of work figuring out that the blocks are already on stable storage or something like that. (ie. It might hint at where to look for a ZFS related perf. problem.) OR 2 - Nothing, because the client doesn't notice it doesn't need to commit it and does the Commit RPC anyhow. Also, it is potentially dangerous, since if the server crashes after the client has done the write, but before the server has written it to disk, the data may be lost. (ie. The client might have flushed the dirty blocks out of its buffer cache, because it didn't think it needed to do a Commit RPC.) rick