From owner-freebsd-stable@FreeBSD.ORG Tue Jul 6 03:57:52 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6EC21065673 for ; Tue, 6 Jul 2010 03:57:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 5A01A8FC0C for ; Tue, 6 Jul 2010 03:57:52 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEAC5GMkyDaFvG/2dsb2JhbACfenG/FIUlBA X-IronPort-AV: E=Sophos;i="4.53,544,1272859200"; d="scan'208";a="83062330" Received: from amazon.cs.uoguelph.ca ([131.104.91.198]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 05 Jul 2010 23:57:48 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by amazon.cs.uoguelph.ca (Postfix) with ESMTP id 95A69350170 for ; Mon, 5 Jul 2010 23:57:50 -0400 (EDT) X-Virus-Scanned: amavisd-new at amazon.cs.uoguelph.ca Received: from amazon.cs.uoguelph.ca ([127.0.0.1]) by localhost (amazon.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WB7m1NpdyUEJ for ; Mon, 5 Jul 2010 23:57:49 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by amazon.cs.uoguelph.ca (Postfix) with ESMTP id 9B45235016A for ; Mon, 5 Jul 2010 23:57:49 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o663cTX09083; Mon, 5 Jul 2010 23:38:31 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Mon, 5 Jul 2010 23:38:29 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: "Rick C. Petty" In-Reply-To: <20100627221607.GA31646@kay.kiwi-computer.com> Message-ID: References: <20100627221607.GA31646@kay.kiwi-computer.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-stable@freebsd.org Subject: Re: Why is NFSv4 so slow? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2010 03:57:52 -0000 On Sun, 27 Jun 2010, Rick C. Petty wrote: > First off, many thanks to Rick Macklem for making NFSv4 possible in > FreeBSD! > > I recently updated my NFS server and clients to v4, but have since noticed > significant performance penalties. For instance, when I try "ls a b c" (if > a, b, and c are empty directories) on the client, it takes up to 1.87 > seconds (wall time) whereas before it always finished in under 0.1 seconds. > If I repeat the test, it takes the same amount of time in v4 (in v3, wall > time was always under 0.01 seconds for subsequent requests, as if the > directory listing was cached). > > If I try to play an h264 video file on the filesystem using mplayer, it > often jitters and skipping around in time introduces up to a second or so > pause. With NFSv3 it behaved more like the file was on local disk (no > noticable pauses or jitters). > I just came across a case where things get really slow during testing of some experimental caching stuff. (It was caused by the experimental stuff not in head, but...) It turns out that if numvnodes > desiredvnodes, it sleeps for 1sec before allocating a new vnode. This might explain your approx. 1sec delays. When this happens, "ps axlH" will probably show a process sleeping on "vlruwk" and desiredvnodes can be increased by setting a larger value for kern.maxvnodes. (numvnodes can be seen as vfs.numvnodes) I don't think there is actually a vnode leak, but you might find that the experimental nfs subsystem is a vnode hog. rick