From owner-freebsd-arch@FreeBSD.ORG Fri Apr 16 08:36:11 2010 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD6241065673; Fri, 16 Apr 2010 08:36:11 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail08.syd.optusnet.com.au (mail08.syd.optusnet.com.au [211.29.132.189]) by mx1.freebsd.org (Postfix) with ESMTP id 629D08FC17; Fri, 16 Apr 2010 08:36:10 +0000 (UTC) Received: from c122-106-149-225.carlnfd1.nsw.optusnet.com.au (c122-106-149-225.carlnfd1.nsw.optusnet.com.au [122.106.149.225]) by mail08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o3G8a7AV026419 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 16 Apr 2010 18:36:08 +1000 Date: Fri, 16 Apr 2010 18:36:07 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Rick Macklem In-Reply-To: Message-ID: <20100416181926.F1082@delplex.bde.org> References: <4BBEE2DD.3090409@freebsd.org> <4BBF3C5A.7040009@freebsd.org> <20100411114405.L10562@delplex.bde.org> <20100414135230.U12587@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@FreeBSD.org, Andriy Gapon Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Apr 2010 08:36:11 -0000 On Wed, 14 Apr 2010, Rick Macklem wrote: > On Wed, 14 Apr 2010, Bruce Evans wrote: > [stuff snipped] >> >> Indeed, I was only caring about a LAN environment. Especially with >> LANs optimized for latency (50-100 uS), nfs performance is poor for >> small files, at least for the old nfs client, mainly due to close to >> open consistency defeating caching, but not a problem for bulk transfers. > > And I'll admit I was thinking that for a low latency LAN, a large read/write > RPC wouldn't have a negative impact, but it sounds like > you've found 16Kb to be optimal for this case. I'll try to find old benchmark results or repeat the benchmarks. > For NFSv4, if the client has a delegation for the file, it doesn't > have worry about close/open consistency, so there is some hope w.r.t. > small files for this case. Do you have benchmarks? A kernel build (without -j) is a good test. Due to include bloat and include nesting bloat, a kernel build opens and closes the same small include files hundreds or thousands of times each, with O(10^5) includes altogether, so an RPC to read attributes on each open costs a lot of latency. nfs on a LAN does well to take only 10% longer than a local file system on a LAN and after disabling close/open constency takes only about half as much longer, by reducing the nomber of RPCs by about a factor of 2. The difference should be even more noticable on a WAN. Building with -j reduces the extra length by not stalling the whild build waiting for each RPC. I probably needed it to take only 10% longer. Bruce