From owner-freebsd-current@FreeBSD.ORG Thu Nov 12 18:44:11 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D96A1065670; Thu, 12 Nov 2009 18:44:11 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-pw0-f44.google.com (mail-pw0-f44.google.com [209.85.160.44]) by mx1.freebsd.org (Postfix) with ESMTP id C42E58FC0C; Thu, 12 Nov 2009 18:44:10 +0000 (UTC) Received: by pwj15 with SMTP id 15so1661291pwj.3 for ; Thu, 12 Nov 2009 10:44:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=YcTooj3NHB0I5H1+l9XJm5vx7rd6JAxegWMPpQqPca8=; b=ge3FZ5TOSHITZzHwRFGtWGoG+hpJhz+Q+f721+2TcELHTv40k2+RzZUFDu5vNpfUf7 c/rW1L/dFeM2IN53/wKLbFWSQj8RnBHbLzTzeEN2uY5SnvcAw01cUIduHPEnflppgcm9 uTDAJNYBGLj9lkl9xoRBTVNFAhdztNiT3d61M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=H51Ypes6ONSXQn/GldEnmaeEAaccP+FN2DOPrW+rfCJlPglhLjLuPgdrIJFi7mtR1X EqUmPW0g+SwaNRrBBVqSU1POxE8Wm1xf0uV1bPs0YEINMvoU8glqDwRqx5+dP2LOv7kT 6ktZsLnd0ZsaqZXxVD4wVyCMb7NYiBzvkTg4s= MIME-Version: 1.0 Received: by 10.142.74.1 with SMTP id w1mr365809wfa.156.1258051450062; Thu, 12 Nov 2009 10:44:10 -0800 (PST) In-Reply-To: References: <11167f520911111050j36dd94far667c81e6f5c18e69@mail.gmail.com> <20091111204903.GI89052@dan.emsphone.com> <11167f520911111326v13bb442bt36e853afbecdf834@mail.gmail.com> <9bbcef730911111352t12188bdajbca71bcf35a5beb5@mail.gmail.com> Date: Thu, 12 Nov 2009 12:44:10 -0600 Message-ID: <11167f520911121044l74744c30u5a4d9ca008ab863c@mail.gmail.com> From: "Sam Fourman Jr." To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org, Dan Nelson , Ivan Voras Subject: Re: Help ZFS FreeBSD 8.0 RC2 Write performance issue X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Nov 2009 18:44:11 -0000 On Thu, Nov 12, 2009 at 11:02 AM, Rick Macklem wrote= : > > > On Wed, 11 Nov 2009, Ivan Voras wrote: > >> >> I think NFS uses sync disk IO access by default, this may be your >> problem if you are write-heavy. Try setting vfs.nfsrv.async to 1 to >> see if this is the cause of your problems. > > Just fyi, I took a quick look and I don't think this will be a good > idea for NFSv3. (It allows the server to cheat for NFSv2 and avoid > synchronous writes, which was contrary to the standard, but became > fashionable for performance reasons, before NFSv3 came out.) > > For NFSv3, the writes are normally done asynchronously, followed > by a Commit RPC to force them to disk, done by the client when it > is flushing its buffer cache. > > When you set this sysctl, the NFSv3 server (sys/nfsserver, not the > experimental one), all it does is reply to the write RPC that it > has already been committed. This may have two effects, depending > upon the client: > 1 - The client may then choose to not bother with a Commit RPC. > =A0 =A0--> This shouldn't help performance much, because the data > =A0 =A0 =A0 =A0will normally have made to disk by now. > *** This might be an interesting experiment to try on a ZFS server > =A0 =A0though, since if it does make a significant difference, it > =A0 =A0suggests that ZFS does a lot of work figuring out that the > =A0 =A0blocks are already on stable storage or something like that. > =A0 =A0(ie. It might hint at where to look for a ZFS related perf. > =A0 =A0 problem.) > OR > 2 - Nothing, because the client doesn't notice it doesn't need to > =A0 =A0commit it and does the Commit RPC anyhow. > > Also, it is potentially dangerous, since if the server crashes after > the client has done the write, but before the server has written it > to disk, the data may be lost. (ie. The client might have flushed the > dirty blocks out of its buffer cache, because it didn't think it needed > to do a Commit RPC.) > > rick I guess I am confused a bit. Why is it that sftp has better write speed than NFS? and from what I am hearing there isn't a good way to fox it? what about if I try and tweak the nfs connection paramaters. all of these tests were done with whatever default is. anyone have any Ideas on better client connection parameters? Sam Fourman Jr.