From owner-freebsd-stable@FreeBSD.ORG Sun Jun 27 22:42:50 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A2FF106564A for ; Sun, 27 Jun 2010 22:42:50 +0000 (UTC) (envelope-from rick@svn.kiwi-computer.com) Received: from svn.kiwi-computer.com (174-20-59-6.mpls.qwest.net [174.20.59.6]) by mx1.freebsd.org (Postfix) with SMTP id 9A6FB8FC13 for ; Sun, 27 Jun 2010 22:42:49 +0000 (UTC) Received: (qmail 37613 invoked by uid 2000); 27 Jun 2010 22:16:07 -0000 Date: Sun, 27 Jun 2010 17:16:07 -0500 From: "Rick C. Petty" To: freebsd-stable@freebsd.org Message-ID: <20100627221607.GA31646@kay.kiwi-computer.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Subject: Why is NFSv4 so slow? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd2009@kiwi-computer.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jun 2010 22:42:50 -0000 First off, many thanks to Rick Macklem for making NFSv4 possible in FreeBSD! I recently updated my NFS server and clients to v4, but have since noticed significant performance penalties. For instance, when I try "ls a b c" (if a, b, and c are empty directories) on the client, it takes up to 1.87 seconds (wall time) whereas before it always finished in under 0.1 seconds. If I repeat the test, it takes the same amount of time in v4 (in v3, wall time was always under 0.01 seconds for subsequent requests, as if the directory listing was cached). If I try to play an h264 video file on the filesystem using mplayer, it often jitters and skipping around in time introduces up to a second or so pause. With NFSv3 it behaved more like the file was on local disk (no noticable pauses or jitters). Has anyone seen this behavior upon switching to v4 or does anyone have any suggestions for tuning? Both client and server are running the same GENERIC kernel, 8.1-PRERELEASE as of 2010-May-29. They are connected via gigabit. Both v3 and v4 tests were performed on the exact same hardware and I/O, CPU, network loads. All I did was toggle nfsv4_server_enable (and nfsuserd/nfscbd of course). It seems like a server-side issue, because if I try an nfs3 client mount to the nfs4 server and run the same tests, I see only a slight improvement in performance. In both cases, my mount options were "rdirplus,bg,intr,soft" (and "nfsv4" added in the one case, obviously). On the server, I have these tunables explicitly set: kern.ipc.maxsockbuf=524288 vfs.newnfs.issue_delegations=1 On the client, I just have the maxsockbuf setting (this is twice the default value). I'm open to trying other tunables or patches. TIA, -- Rick C. Petty