From owner-freebsd-current@FreeBSD.ORG Sat Jul 7 12:27:02 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93818106564A for ; Sat, 7 Jul 2012 12:27:02 +0000 (UTC) (envelope-from vince@unsane.co.uk) Received: from unsane.co.uk (unsane-pt.tunnel.tserv5.lon1.ipv6.he.net [IPv6:2001:470:1f08:110::2]) by mx1.freebsd.org (Postfix) with ESMTP id E4F508FC12 for ; Sat, 7 Jul 2012 12:27:01 +0000 (UTC) Received: from vincemacbook.unsane.co.uk (vincemacbook.unsane.co.uk [10.10.10.20]) (authenticated bits=0) by unsane.co.uk (8.14.5/8.14.5) with ESMTP id q67CQxuh001266 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 7 Jul 2012 13:27:00 +0100 (BST) (envelope-from vince@unsane.co.uk) Message-ID: <4FF82B13.8000906@unsane.co.uk> Date: Sat, 07 Jul 2012 13:26:59 +0100 From: Vincent Hoffman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Rick Macklem References: <497527423.2441665.1341141539082.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <497527423.2441665.1341141539082.JavaMail.root@erie.cs.uoguelph.ca> X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: Occassional "permission denied" in the middle of a large transfer over NFS X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2012 12:27:02 -0000 On 01/07/2012 12:18, Rick Macklem wrote: > Vincent Hoffman wrote: >> On 01/07/2012 01:53, Rick Macklem wrote: >>> To modify mountd to use the kernel changes is more work than I have >>> time for, in part because mountd.c is a very ugly old piece of C >>> code, imho. >>> >>> I do have a patch that suspends/resumes execution of the nfsd >>> threads >>> (new, experimental for FreeBSD8.n only) when mountd reloads >>> /etc/exports. >>> This approach has certain disadvantages vs Andrey's transactional >>> load/commit >>> model, but it is simple and might be sufficient for your problem. >>> If you want to try this patch, it is at: >>> http://people.freebsd.org/~rmacklem/atomic-export.patch >>> (The patch affects both the kernel and mountd.c.) >> Happy to try it, to be certain I have understood, this is for the >> newnfs >> server (experimental in 8.x default for 8 >> 9+) > Yes, that is correct. For the old (default on 8.x) it will have no effect. > >> I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when >> i >> get a minute. > Also, to enable it, you must add a "-S" flag to mountd_flags, after you've > replaced the mountd executable. > > The patch is pretty straightforward, but has not seen much testing. > Hopefully, it might be useful for you, rick Hi Rick, I'm afraid this didnt make any real difference for me. Since I couldnt test it on the live system I tried it on a test vm. on the vm (nfs server) I set a looping mount/umount while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ; done and on the client I set a loop of tars of large directorys to the nfs mount running under time to see how well it survived. Then replicated the test with the patch and without. [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt x nopatch.txt + atomicpatch.txt +--------------------------------------------------------------------------------------------------+ | * | | * | | x* | | xx* x | | +x** xx | | **** xxx x | | **** xxx +x+ + | | ****+*xx +x+ x + | | ****+*x****++++x + + x | | *************+*xx+ +++x * x x | | ****************x**++*x+***x+ x*+ x ++*+ + x+ +x + + +| |||_______M_M_A__A_______|______| | +--------------------------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 101 1.25 106.8 14.08 21.892178 22.196005 + 101 1.21 186.93 18.46 27.995842 30.523218 No difference proven at 95.0% confidence (excuse wrapped ascii art) I think I'll have a look at the nfse patch set and see how that performs. Thanks for all your work on NFS on FreeBSD. Vince > >>> Also, you could easily hack mount.c so that it doesn't send a SIGHUP >>> to mountd (which causes the exports to be reloaded) every time a >>> local >>> fs is mounted. >> True and I may have to do that for the production NAS for the time >> being. >> Thanks for looking at this. >> >> Vince >>> rick >>> >>>>> thanks, Vince