From owner-freebsd-fs@FreeBSD.ORG Sun Mar 1 19:10:02 2009 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C06621065672 for ; Sun, 1 Mar 2009 19:10:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AD7428FC1C for ; Sun, 1 Mar 2009 19:10:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n21JA2I3077083 for ; Sun, 1 Mar 2009 19:10:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n21JA2K5077082; Sun, 1 Mar 2009 19:10:02 GMT (envelope-from gnats) Date: Sun, 1 Mar 2009 19:10:02 GMT Message-Id: <200903011910.n21JA2K5077082@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Robert Watson Cc: Subject: Re: kern/131360: [nfs] poor scaling behavior of the NFS server under load X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Robert Watson List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Mar 2009 19:10:03 -0000 The following reply was made to PR kern/131360; it has been noted by GNATS. From: Robert Watson To: FreeBSD-gnats-submit@FreeBSD.org Cc: Subject: Re: kern/131360: [nfs] poor scaling behavior of the NFS server under load Date: Sun, 1 Mar 2009 19:06:25 +0000 (GMT) On Wed, 4 Feb 2009, Martin Birgmeier wrote: > [Please note that the email address given is not valid due > to security reasons. Just reply to the gnats entry, and > I'll follow it via the web interface.] > > Between FreeBSD 6.3 and 7.1, the behavior of the NFS server > changed for the worse. Under 6.3, the load generated by the > nfsd's would never exceed their number (I am using > nfs_server_flags="-u -t -n 8"). With 7.1, when the client > generates a lot of requests, it seems that the load on the > NFS server can grow nearly without bounds, rendering the > server all but unusable. Hi Martin: Could I ask you to clarify a few things about the NFS server configuration: - What device/device driver is on the server? - Is the mount over TCP or UDP? There are a number of changes between 7.0, one of the potential changes to look at is the increased UDP parallelism support due to read-locking in the UDP stack overwhelming the capabilities of some device/device drivers. We've had a couple of reports of this specifically with the bge driver. If you're using UDP, converting to TCP would be interesting; if it's bge, then we could give you a patch to locally revert the read-locking changes to see if that helps manage contention better (i.e., more contention higher on a sec of locks moves the contention off the single device driver lock). Robert N M Watson Computer Laboratory University of Cambridge