From owner-freebsd-hackers@FreeBSD.ORG Fri Sep 19 15:30:28 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D502716A4B3 for ; Fri, 19 Sep 2003 15:30:28 -0700 (PDT) Received: from pimout1-ext.prodigy.net (pimout1-ext.prodigy.net [207.115.63.77]) by mx1.FreeBSD.org (Postfix) with ESMTP id B93E443FAF for ; Fri, 19 Sep 2003 15:30:27 -0700 (PDT) (envelope-from rsharpe@richardsharpe.com) Received: from ns.aus.com (adsl-63-206-120-73.dsl.snfc21.pacbell.net [63.206.120.73])h8JMUOou145516; Fri, 19 Sep 2003 18:30:25 -0400 Received: from ns.aus.com (durable [127.0.0.1]) by ns.aus.com (8.12.8/8.12.8) with ESMTP id h8JMZgMI006561; Fri, 19 Sep 2003 15:35:42 -0700 Received: from localhost (rsharpe@localhost) by ns.aus.com (8.12.8/8.12.8/Submit) with ESMTP id h8JMZeMt006557; Fri, 19 Sep 2003 15:35:42 -0700 X-Authentication-Warning: ns.aus.com: rsharpe owned process doing -bs Date: Fri, 19 Sep 2003 15:35:40 -0700 (PDT) From: Richard Sharpe X-X-Sender: rsharpe@durable To: John-Mark Gurney In-Reply-To: <20030919182640.GJ75714@funkthat.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-hackers@freebsd.org Subject: Re: Throughput problems with NFS between Linux and FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Sep 2003 22:30:28 -0000 On Fri, 19 Sep 2003, John-Mark Gurney wrote: > Richard Sharpe wrote this message on Fri, Sep 19, 2003 at 10:38 -0700: > > We recently encountered a problem with NFS throughput between a FreeBSD > > server (we are using 4.6.2, but the same code seems to be in 5.1 as well). > > > > When using Linux 2.4.19 or 2.4.21 as a client, although this might extend > > to other clients, and copying a large file, you will see the behavior > > shown in > > [...] > > > The problem seems to be the following code > > > > if (so->so_type == SOCK_STREAM) > > siz = NFS_MAXPACKET + sizeof (u_long); > > else > > siz = NFS_MAXPACKET; > > error = soreserve(so, siz, siz); > > > > in src/sys/nfs/nfs_syscalls.c. > > > > We added a sysctl to allow finer control over what is passed to soreserve. > > > > With the fix in, it goes up to around wire speed when lots of data is in > > the cache. > > What is the fix? You don't say what adjustments to soreserve's parameters > are necessary to improve performance? Have you done testing against other > clients to see how your changes will affect performance on those machines? The beest fix is: if (so->so_type == SOCK_STREAM) - siz = NFS_MAXPACKET + sizeof (u_long); + siz = NFS_MAXPACKET + sizeof (u_long) + MSS; else siz = NFS_MAXPACKET; error = soreserve(so, siz, siz); in src/sys/nfs/nfs_syscalls.c. Since the client should only hang onto the ack for one segment, and that will work even if you have end-to-end jumbo frames. A simpler fix might be to replace MSS with 2048. Regards ----- Richard Sharpe, rsharpe[at]ns.aus.com, rsharpe[at]samba.org, sharpe[at]ethereal.com, http://www.richardsharpe.com