Date: Sun, 08 Jul 2012 18:35:50 +0100 From: Vincent Hoffman <vince@unsane.co.uk> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: freebsd-current@freebsd.org Subject: Re: Occassional "permission denied" in the middle of a large transfer over NFS Message-ID: <4FF9C4F6.9020402@unsane.co.uk> In-Reply-To: <4FF82B13.8000906@unsane.co.uk> References: <497527423.2441665.1341141539082.JavaMail.root@erie.cs.uoguelph.ca> <4FF82B13.8000906@unsane.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07/07/2012 13:26, Vincent Hoffman wrote: > On 01/07/2012 12:18, Rick Macklem wrote: >> Vincent Hoffman wrote: >>> On 01/07/2012 01:53, Rick Macklem wrote: >>>> To modify mountd to use the kernel changes is more work than I have >>>> time for, in part because mountd.c is a very ugly old piece of C >>>> code, imho. >>>> >>>> I do have a patch that suspends/resumes execution of the nfsd >>>> threads >>>> (new, experimental for FreeBSD8.n only) when mountd reloads >>>> /etc/exports. >>>> This approach has certain disadvantages vs Andrey's transactional >>>> load/commit >>>> model, but it is simple and might be sufficient for your problem. >>>> If you want to try this patch, it is at: >>>> http://people.freebsd.org/~rmacklem/atomic-export.patch >>>> (The patch affects both the kernel and mountd.c.) >>> Happy to try it, to be certain I have understood, this is for the >>> newnfs >>> server (experimental in 8.x default for 8 >>> 9+) >> Yes, that is correct. For the old (default on 8.x) it will have no effect. >> >>> I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when >>> i >>> get a minute. >> Also, to enable it, you must add a "-S" flag to mountd_flags, after you've >> replaced the mountd executable. >> >> The patch is pretty straightforward, but has not seen much testing. >> Hopefully, it might be useful for you, rick > Hi Rick, > > I'm afraid this didnt make any real difference for me. > Since I couldnt test it on the live system I tried it on a test vm. > on the vm (nfs server) I set a looping mount/umount > while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ; done > > and on the client I set a loop of tars of large directorys to the nfs > mount running under time to see how well it survived. Then replicated > the test with the patch and without. > > > [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt > x nopatch.txt > + atomicpatch.txt > +--------------------------------------------------------------------------------------------------+ > | > * > | > | > * > | > | > x* > | > | xx* > x > | > | +x** > xx > | > | **** xxx > x > | > | **** xxx +x+ > + > | > | ****+*xx +x+ x > + > | > | ****+*x****++++x + + > x | > | *************+*xx+ +++x * x > x | > | ****************x**++*x+***x+ x*+ x ++*+ + x+ +x + > + +| > |||_______M_M_A__A_______|______| > | > +--------------------------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 101 1.25 106.8 14.08 21.892178 22.196005 > + 101 1.21 186.93 18.46 27.995842 30.523218 > No difference proven at 95.0% confidence > > > (excuse wrapped ascii art) > > I think I'll have a look at the nfse patch set and see how that performs. Replying to myself just as a record, I have tried nfse and I didnt get the permission denied at all. The only issue I had with it is that it strictly adheres to the syntax in exports(5) while mountd is a little more flexible. I had /usr/local/export -alldirs -maproot=root 85.xx.xx.xx /usr is the root of that filesystem mountd - allowed this but actually silently exports /usr (and all dirs below) nfse - didnt allow this, it needed to be the correct /usr -alldirs -maproot=root 85.xx.xx.xx which is I guess a POLA violation but follows the documentation. this was using nfse in mountd compatibility mode. I havent played with its more advanced features. Vince > Thanks for all your work on NFS on FreeBSD. > > Vince >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4FF9C4F6.9020402>