From owner-freebsd-net@FreeBSD.ORG Mon Dec 15 15:04:35 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2A7E4BE for ; Mon, 15 Dec 2014 15:04:35 +0000 (UTC) Received: from mail-yk0-x230.google.com (mail-yk0-x230.google.com [IPv6:2607:f8b0:4002:c07::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CB5DA1D for ; Mon, 15 Dec 2014 15:04:35 +0000 (UTC) Received: by mail-yk0-f176.google.com with SMTP id q200so4946076ykb.7 for ; Mon, 15 Dec 2014 07:04:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=SKHPvyiXTZbVjHW8yZEUrW8N0L87a0nBNMp+3Eui3T0=; b=IyFkRg/e6z8gcKQ3WRSh0IJWy+MWbcD4k+LGgeKO9bpot3qsLNg/U3YyARPneJ06fv hUDat5KOFkrSit1CTswtI9yHd11bvti/8+EacweSwXcynFtc0cQS2//C6n8OulGB+qHD TVarPhbJ7Vn7LNA/421MlvxLy9eIigEkcEoEVznYNjiHvGnoU1ugVxqlwqc6g0kBkmzg RCJY+0WoL52dYa0hpRY8cGbUX0QfgHhvGuioQ13BGY3NWJaBZlT992eF+fZVRCbOA/ZH ZKlU/EeLUkKJY9kpOHuipyDHVP2GXe6eoSqWPB7NjpQxlL0hvOFuvBX3pnmaUXz0Ig8+ LlmA== MIME-Version: 1.0 X-Received: by 10.236.26.43 with SMTP id b31mr5365480yha.53.1418655874434; Mon, 15 Dec 2014 07:04:34 -0800 (PST) Received: by 10.170.90.131 with HTTP; Mon, 15 Dec 2014 07:04:34 -0800 (PST) In-Reply-To: <20141215102429.bcbae18ae63464a7254fc580@aei.mpg.de> References: <20141215102429.bcbae18ae63464a7254fc580@aei.mpg.de> Date: Mon, 15 Dec 2014 07:04:34 -0800 Message-ID: Subject: Re: compiling on nfs directories From: Mehmet Erol Sanliturk To: =?UTF-8?B?R2Vycml0IEvDvGhu?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2014 15:04:35 -0000 On Mon, Dec 15, 2014 at 1:24 AM, Gerrit K=C3=BChn wrote: > > Hi all, > > I ran into some weird issue here last week: > I have an NFS-Server for storage and diskless booting (pxe / nfs root) > running under FreeBSD. The clients are running Gentoo Linux. Some time > ago, I replaced the server, going from a HDD-based storage array (ZFS) > under FreeBSD 8.3 to an SSD-based array under FreeBSD 10-stable (as of > February this year - I know this needs updates). > > Only now I recognized that this somehow appears to have broken some of my > Gentoo ebuilds that do not install cleanly anymore. They complain about > "soiled libtool library files found" and "insecure RUNPATHs" in the > installation stage of shared libs. > > I was not able to find any useful solution for this in the Net so far. > However, I was able to verify that this is somehow an issue with the nfs > server by plugging in a USB-drive into the diskless clients and mounting > this as /var/tmp/portage (the directory structure where Gentoo's ebuilds > are compiled). This makes the error messages go away, and everything work= s > again (like it did before the server update). > > Are there any suggestions what might be causing this and how to fix it? > > > cu > Gerrit > > With respect to information given in your message , may pure guess is the following : When a client generates a file in NFS server , it assumes that everything is written into the file . The next step ( reading the generated file ) starts , BUT the file is NOT completely written into disk yet , therefore it reads an incomplete file which causes errors in the client . In FreeBSD NFS server , there is NOT ( or I could NOT be able to find ) a facility to store written data immediately into disk . NFS server is collecting data up to a point ( number of bytes ) and then writing it to disk , during this phase ( whether the NFS server is busy or not ) is not important ) . With this structure , the tasks which a program writes a small number of bytes to be read by another program can not be processed by a NFS server only . I did not try "locking in NFS server" : If this route is taken , then it is necessary to adjust the clients for such periods to wait that NFS server has removed the lock which themselves can continue ( Each such read requires a waiting loop without generating an error message about unavailable data and termination . ) . In Linux NFS server , there is an option to immediately write the received data into disk . This is improving the above situation considerable but not completely solving the problem ( because during reads of data , data in cache is NOT concatenated to the data in disk ) . Another MAJOR problem is that , the NFS server is NOT concatenating data in cache to data in disk during reads : This defect is making NFS server useless for , let's say "real time" , applications used concurrently or as a single one by the clients without using another "Server" within NFS server . In your case , during software builds , a step is using the previously generated files : In local disk , writing and reading are sequential , in the sense that written data is found during reading . In NFS server this is not the case . With respect to my knowledge obtained from messages in FreeBSD mailing lists about making a possibility to read data immediately after it is written into NFS server is NOT available . Thank you very much . Mehmet Erol Sanliturk