From owner-freebsd-net@FreeBSD.ORG Fri Dec 19 00:05:26 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EECF3A77 for ; Fri, 19 Dec 2014 00:05:26 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 8B003215B for ; Fri, 19 Dec 2014 00:05:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtoEAEFnk1SDaFve/2dsb2JhbABbg1hYBIMCwxIOhSRKAoE3AQEBAQF9hAwBAQEDAQEBASArIAYFGxgCAg0ZAikBCSYGCAcEARwEiAMIDbkjli0BAQEBAQEBAQEBAQEBAQEBAQEBAQEXgSGMRIErEAIBGy8FB4JogUEFiUOIBYMcgyMwhGKHa4M4IoQKIDEBBoE+fgEBAQ X-IronPort-AV: E=Sophos;i="5.07,604,1413259200"; d="scan'208";a="177829649" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 18 Dec 2014 19:05:24 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D8644AEA47; Thu, 18 Dec 2014 19:05:24 -0500 (EST) Date: Thu, 18 Dec 2014 19:05:24 -0500 (EST) From: Rick Macklem To: Gerrit =?utf-8?B?S8O8aG4=?= Message-ID: <1694130912.577722.1418947524871.JavaMail.root@uoguelph.ca> In-Reply-To: <20141216092948.605dc8e2e0fec3fa4a5f8ec1@aei.mpg.de> Subject: Re: compiling on nfs directories MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Dec 2014 00:05:27 -0000 Gerrit Kuhn wrote: > On Mon, 15 Dec 2014 15:59:29 -0500 (EST) Rick Macklem > wrote about Re: compiling on nfs directories: > > > RM> Also, note that he didn't see the problem with FreeBSD8.3, which > would > RM> have been following the same rules on the server as 10.1. > RM> > RM> What I suspect might cause this is one of two things: > RM> 1 - The modify time of the file is now changing at a time the > Linux > RM> client doesn't expect, due to changes in ZFS or maybe TOD > clock > RM> resolution. (At one time, the TOD clock was only at a > resolution > RM> of 1sec, so the client wouldn't see the modify time change > often. > RM> I think it is now at a much higher resolution, but would have > to > RM> look at the code/test to be sure.) > RM> 2 - I think you mention this one later in your message, in that > the > RM> build might be depending on file locking. If this is the > case, > RM> trying NFSv4, which does better file locking, might fix the > RM> problem. > > Meanwhile I have googled around a bit more, and one of the few > reasons > other people see the error messages I see appears to be a broken > clock that > makes "make" recompile stuff on the installation stage. As I was > already > wondering why compilation took longer than I had actually expected, I > may > be seeing something similar (still need to look into that), although > my > clock is fine (but time stamps on the NFS might be messed up somehow > like > you mention above under "1"). avg@ just MFC'd a one line ZFS patch (the one I vaguely remembered before). It is r275401 in head: https://svnweb.freebsd.org/base/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c?r1=274337&r2=275401 and appears to fix a problem with mtime being updated. This patch might be worth a try? (You could email avg@freebsd.org for more information on it.) rick > > RM> Gerrit, I would suggest that you do "nfsstat -m" on the Linux > client, > RM> to see what the mount options are. The Linux client might be > using > RM> NFSv4 already. > > This is what it says about my nfs-root: > > --- > pt-nds ~ # nfsstat -m > / from 192.168.32.253:/tank/diskless/nds > Flags: > rw,relatime,vers=3,rsize=4096,wsize=4096,namlen=255,hard,nolock,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.32.253,mountvers=3,mountproto=tcp,local_lock=all,addr=192.168.32.253 > --- > > This is what I set up for pxe-booting: > > --- > label gentoo-cs2 > menu label linux-3.8.13-gentoo-2 > kernel bzImage-3.8.13-gentoo-2 > append ip=dhcp root=/dev/nfs rw > nfsroot=192.168.32.253:/tank/diskless/nds,nolock,tcp,v3 > rootdelay=15 > --- > > > So I definitely run "nfsv3" and "nolock". I remember trying to use > nfsv4 on the diskless machines some years ago, but back then it was > not ready for prime time. > > RM> Also, avoid "soft, intr" especially if you are using NFSv4, since > these > RM> can cause slow server response to result in a failure of a > read/write > RM> when it shouldn't fail, due to timeout or interruption by a > signal. > > There is "hard" in there as a default option. However, I might try > turning on locking (I regarded it as superfluous up to now as I have > only one client using the filesystem). > > RM> If you could find out more about what causes the specific build > failure > RM> on the Linux side, that might help. > > As I said above, I have some hints that indicate something might be > wrong with timestamps, but I still need to dig deeper into that. > > RM> If you can reproduce a build failure quickly/easily, you can > capture > RM> packets via "tcpdump -s 0 -w host " on > the > RM> server and then look at it in wireshark to see what the server is > RM> replying when the build failure occurs. (I don't mind looking at > a > RM> packet trace if it is relatively small, if you email it to me as > an > RM> attachment.) > > I can reproduce it 100%, but it only happens on the installation > stage, after having compiled the whole stuff. So I don't know if I > will be able to produce a dump of reasonable size that contains the > issue, but I'll try. > > RM> ps: I am not familiar with the Linux mount options, but if it has > RM> stuff like "nocto", you could try those. > > The manpage has the following: > > --- > cto / nocto Selects whether to use close-to-open cache > coherence > semantics. If neither option is specified (or > if cto is > specified), the client uses close-to-open > cache coher- > ence semantics. If the nocto option is > specified, the > client uses a non-standard heuristic to > determine when > files on the server have changed. > > Using the nocto option may improve performance > for read- > only mounts, but should be used only if the > data on the > server changes only occasionally. The DATA AND > METADATA > COHERENCE section discusses the behavior of > this option > in more detail. > --- > > > So "cto" appears to be the default and is probably what is used right > now. I'll put "nocto" on my list of things to try (although the > description is not really that incouraging... :-). > > > cu > Gerrit > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to > "freebsd-net-unsubscribe@freebsd.org" >