From owner-freebsd-net@FreeBSD.ORG Tue Dec 16 18:43:29 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 79DF1FCC for ; Tue, 16 Dec 2014 18:43:29 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5067CAC3 for ; Tue, 16 Dec 2014 18:43:29 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E1608B94C; Tue, 16 Dec 2014 13:43:27 -0500 (EST) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: compiling on nfs directories Date: Tue, 16 Dec 2014 13:37:58 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <2048229686.13136235.1418677169130.JavaMail.root@uoguelph.ca> In-Reply-To: <2048229686.13136235.1418677169130.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201412161337.58789.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 16 Dec 2014 13:43:28 -0500 (EST) Cc: Rick Macklem , Gerrit =?utf-8?q?K=C3=BChn?= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Dec 2014 18:43:29 -0000 On Monday, December 15, 2014 3:59:29 pm Rick Macklem wrote: > Mehmet Erol Sanliturk wrote: > > On Mon, Dec 15, 2014 at 1:24 AM, Gerrit K=C3=BChn > > > > wrote: > > > > > > Hi all, > > > > > > I ran into some weird issue here last week: > > > I have an NFS-Server for storage and diskless booting (pxe / nfs > > > root) > > > running under FreeBSD. The clients are running Gentoo Linux. Some > > > time > > > ago, I replaced the server, going from a HDD-based storage array > > > (ZFS) > > > under FreeBSD 8.3 to an SSD-based array under FreeBSD 10-stable (as > > > of > > > February this year - I know this needs updates). > > > > > > Only now I recognized that this somehow appears to have broken some > > > of my > > > Gentoo ebuilds that do not install cleanly anymore. They complain > > > about > > > "soiled libtool library files found" and "insecure RUNPATHs" in the > > > installation stage of shared libs. > > > > > > I was not able to find any useful solution for this in the Net so > > > far. > > > However, I was able to verify that this is somehow an issue with > > > the nfs > > > server by plugging in a USB-drive into the diskless clients and > > > mounting > > > this as /var/tmp/portage (the directory structure where Gentoo's > > > ebuilds > > > are compiled). This makes the error messages go away, and > > > everything works > > > again (like it did before the server update). > > > > > > Are there any suggestions what might be causing this and how to fix > > > it? > > > > > > > > > cu > > > Gerrit > > > > > > > >=20 > >=20 > > With respect to information given in your message , may pure guess is > > the > > following : > >=20 > >=20 > > When a client generates a file in NFS server , it assumes that > > everything > > is written into the file . > > The next step ( reading the generated file ) starts , BUT the file is > > NOT > > completely written into disk yet , > > therefore it reads an incomplete file which causes errors in the > > client . > >=20 > Well, not exactly. The NFS client chooses whether or not the written > data must be committed to stable storage (disk) right away via a flag > argument on the write. It is up to the client to keep track of what has > been written and if the FILE_STABLE flag wasn't set, must do a separate > Commit RPC to force the data to stable storage on the server. > It is also up to the NFS client to keep track of the file's size while > it is being grown, since the NFS server's size may be smaller until > the data gets written to the server. > Also, note that he didn't see the problem with FreeBSD8.3, which would > have been following the same rules on the server as 10.1. >=20 > What I suspect might cause this is one of two things: > 1 - The modify time of the file is now changing at a time the Linux > client doesn't expect, due to changes in ZFS or maybe TOD clock > resolution. (At one time, the TOD clock was only at a resolution > of 1sec, so the client wouldn't see the modify time change often. > I think it is now at a much higher resolution, but would have to > look at the code/test to be sure.) No, it's still only a second resolution on FreeBSD by default. You can make this precise on the NFS server by setting the vfs.timestamp_precision sysctl to 3. We should probably be using that by default for at least server-class systems. =2D-=20 John Baldwin