Date: Wed, 28 Jan 2004 11:18:31 +0100 (CET) From: Bjoern Groenvall <bg@sics.se> To: FreeBSD-gnats-submit@FreeBSD.org Cc: kris@obsecurity.org Subject: kern/62024: Poor large file performance files when using NFSv3 over TCP. Message-ID: <200401281018.i0SAIVLL023843@manian.sics.se> Resent-Message-ID: <200401281020.i0SAKKkn054275@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 62024
>Category: kern
>Synopsis: Poor large file performance files when using NFSv3 over TCP.
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Wed Jan 28 02:20:19 PST 2004
>Closed-Date:
>Last-Modified:
>Originator: Bjoern Groenvall
>Release: FreeBSD 5.2-CURRENT i386
>Organization:
SICS
>Environment:
System: FreeBSD manian.sics.se 5.2-CURRENT FreeBSD 5.2-CURRENT #0: Fri Jan 23 14:40:38 CET 2004 bg@manian.sics.se:/usr/src/sys/i386/compile/MANIAN i386
>Description:
Poor large file performance files when using NFSv3 over TCP.
One reason that FreeBSD users experience poor NFSv3/TCP performance is
that the defaults for rsize and wsize are unusually small, only
8k. Solaris and HP-UX defaults to 32k for a good reason. I guess TCP
simply needs a little bit more data to chew on to be efficient.
I tested this on 5.2-CURRENT and found that large file read
performance went up from 56Mbit/s to 80Mbit/s, an improvement by 43%.
I have written a patch that makes FreeBSD use the same defaults as
Solaris and HP-UX. Note that with NFSv3 there is no risk associated
with specifying to large values for [rw]size. The server automatically
limits these values in the fsinfo rpc. Patch is attached.
>How-To-Repeat:
Experiment with different values for RWSIZE ranging from 8192 to 32768.
mount_nfs -T -r RWSIZE -w RWSIZE somhost:/somefs /mnt
ls -l /mnt/LARGEFILE
dd if=/mnt/LARGEFILE of=/dev/null bs=512k
>Fix:
--- /usr/src/sys/nfsclient/nfs_vfsops.c.orig Sat Nov 22 03:21:49 2003
+++ /usr/src/sys/nfsclient/nfs_vfsops.c Tue Jan 27 19:26:35 2004
@@ -359,6 +359,7 @@
maxfsize = fxdr_hyper(&fsp->fs_maxfilesize);
if (maxfsize > 0 && maxfsize < nmp->nm_maxfilesize)
nmp->nm_maxfilesize = maxfsize;
+ nmp->nm_mountp->mnt_stat.f_iosize = nfs_iosize(nmp);
nmp->nm_state |= NFSSTA_GOTFSINFO;
}
m_freem(mrep);
@@ -785,8 +786,12 @@
nmp->nm_timeo = NFS_TIMEO;
nmp->nm_retry = NFS_RETRANS;
- nmp->nm_wsize = NFS_WSIZE;
- nmp->nm_rsize = NFS_RSIZE;
+ if ((argp->flags & NFSMNT_NFSV3) && argp->sotype == SOCK_STREAM) {
+ nmp->nm_wsize = nmp->nm_rsize = NFS_MAXDATA;
+ } else {
+ nmp->nm_wsize = NFS_WSIZE;
+ nmp->nm_rsize = NFS_RSIZE;
+ }
nmp->nm_readdirsize = NFS_READDIRSIZE;
nmp->nm_numgrps = NFS_MAXGRPS;
nmp->nm_readahead = NFS_DEFRAHEAD;
@@ -832,10 +837,14 @@
*vpp = NFSTOV(np);
/*
- * Get file attributes for the mountpoint. This has the side
- * effect of filling in (*vpp)->v_type with the correct value.
+ * Get file attributes and transfer parameters for the
+ * mountpoint. This has the side effect of filling in
+ * (*vpp)->v_type with the correct value.
*/
- VOP_GETATTR(*vpp, &attrs, curthread->td_ucred, curthread);
+ if (argp->flags & NFSMNT_NFSV3)
+ nfs_fsinfo(nmp, *vpp, curthread->td_ucred, curthread);
+ else
+ VOP_GETATTR(*vpp, &attrs, curthread->td_ucred, curthread);
/*
* Lose the lock but keep the ref.
@@ -905,6 +914,13 @@
if (error)
return (error);
vp = NFSTOV(np);
+ /*
+ * Get transfer parameters and attributes for root vnode once.
+ */
+ if ((nmp->nm_state & NFSSTA_GOTFSINFO) == 0 &&
+ (nmp->nm_flag & NFSMNT_NFSV3)) {
+ nfs_fsinfo(nmp, vp, curthread->td_ucred, curthread);
+ }
if (vp->v_type == VNON)
vp->v_type = VDIR;
vp->v_vflag |= VV_ROOT;
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200401281018.i0SAIVLL023843>