From owner-svn-src-stable-10@FreeBSD.ORG Mon Dec 30 21:17:21 2013 Return-Path: Delivered-To: svn-src-stable-10@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D70AA83; Mon, 30 Dec 2013 21:17:21 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6CFB81163; Mon, 30 Dec 2013 21:17:21 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id rBULHLaw026528; Mon, 30 Dec 2013 21:17:21 GMT (envelope-from rmacklem@svn.freebsd.org) Received: (from rmacklem@localhost) by svn.freebsd.org (8.14.7/8.14.7/Submit) id rBULHKFO026523; Mon, 30 Dec 2013 21:17:20 GMT (envelope-from rmacklem@svn.freebsd.org) Message-Id: <201312302117.rBULHKFO026523@svn.freebsd.org> From: Rick Macklem Date: Mon, 30 Dec 2013 21:17:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r260107 - in stable/10/sys: fs/nfsclient nfsclient X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-10@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SVN commit messages for only the 10-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Dec 2013 21:17:21 -0000 Author: rmacklem Date: Mon Dec 30 21:17:20 2013 New Revision: 260107 URL: http://svnweb.freebsd.org/changeset/base/260107 Log: MFC: r259084 For software builds, the NFS client does many small synchronous (with FILE_SYNC) writes because non-contiguous byte ranges in the same buffer cache block are being written. This patch adds a new mount option "noncontigwr" which allows the non-contiguous byte ranges to be combined, with the dirty byte range becoming the superset of the bytes that are dirty, if the file has not been file locked. This reduces the number of writes significantly for software builds. The only case where this change might break existing applications is where an application is writing non-overlapping byte ranges within the same buffer cache block of a file from multiple clients concurrently. Since such an application would normally do file locking on the file, avoiding the byte range merge for files that have been file locked should be sufficient for most (maybe all?) cases. Modified: stable/10/sys/fs/nfsclient/nfs_clbio.c stable/10/sys/fs/nfsclient/nfs_clvfsops.c stable/10/sys/fs/nfsclient/nfs_clvnops.c stable/10/sys/fs/nfsclient/nfsnode.h stable/10/sys/nfsclient/nfsargs.h Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/fs/nfsclient/nfs_clbio.c ============================================================================== --- stable/10/sys/fs/nfsclient/nfs_clbio.c Mon Dec 30 21:06:08 2013 (r260106) +++ stable/10/sys/fs/nfsclient/nfs_clbio.c Mon Dec 30 21:17:20 2013 (r260107) @@ -874,7 +874,7 @@ ncl_write(struct vop_write_args *ap) struct vattr vattr; struct nfsmount *nmp = VFSTONFS(vp->v_mount); daddr_t lbn; - int bcount; + int bcount, noncontig_write, obcount; int bp_cached, n, on, error = 0, error1; size_t orig_resid, local_resid; off_t orig_size, tmp_off; @@ -1037,7 +1037,15 @@ again: * unaligned buffer size. */ mtx_lock(&np->n_mtx); - if (uio->uio_offset == np->n_size && n) { + if ((np->n_flag & NHASBEENLOCKED) == 0 && + (nmp->nm_flag & NFSMNT_NONCONTIGWR) != 0) + noncontig_write = 1; + else + noncontig_write = 0; + if ((uio->uio_offset == np->n_size || + (noncontig_write != 0 && + lbn == (np->n_size / biosize) && + uio->uio_offset + n > np->n_size)) && n) { mtx_unlock(&np->n_mtx); /* * Get the buffer (in its pre-append state to maintain @@ -1045,8 +1053,8 @@ again: * nfsnode after we have locked the buffer to prevent * readers from reading garbage. */ - bcount = on; - bp = nfs_getcacheblk(vp, lbn, bcount, td); + obcount = np->n_size - (lbn * biosize); + bp = nfs_getcacheblk(vp, lbn, obcount, td); if (bp != NULL) { long save; @@ -1058,9 +1066,12 @@ again: mtx_unlock(&np->n_mtx); save = bp->b_flags & B_CACHE; - bcount += n; + bcount = on + n; allocbuf(bp, bcount); bp->b_flags |= save; + if (noncontig_write != 0 && on > obcount) + vfs_bio_bzero_buf(bp, obcount, on - + obcount); } } else { /* @@ -1159,19 +1170,23 @@ again: * area, just update the b_dirtyoff and b_dirtyend, * otherwise force a write rpc of the old dirty area. * + * If there has been a file lock applied to this file + * or vfs.nfs.old_noncontig_writing is set, do the following: * While it is possible to merge discontiguous writes due to * our having a B_CACHE buffer ( and thus valid read data * for the hole), we don't because it could lead to * significant cache coherency problems with multiple clients, * especially if locking is implemented later on. * - * As an optimization we could theoretically maintain - * a linked list of discontinuous areas, but we would still - * have to commit them separately so there isn't much - * advantage to it except perhaps a bit of asynchronization. + * If vfs.nfs.old_noncontig_writing is not set and there has + * not been file locking done on this file: + * Relax coherency a bit for the sake of performance and + * expand the current dirty region to contain the new + * write even if it means we mark some non-dirty data as + * dirty. */ - if (bp->b_dirtyend > 0 && + if (noncontig_write == 0 && bp->b_dirtyend > 0 && (on > bp->b_dirtyend || (on + n) < bp->b_dirtyoff)) { if (bwrite(bp) == EINTR) { error = EINTR; Modified: stable/10/sys/fs/nfsclient/nfs_clvfsops.c ============================================================================== --- stable/10/sys/fs/nfsclient/nfs_clvfsops.c Mon Dec 30 21:06:08 2013 (r260106) +++ stable/10/sys/fs/nfsclient/nfs_clvfsops.c Mon Dec 30 21:17:20 2013 (r260107) @@ -719,7 +719,8 @@ static const char *nfs_opts[] = { "from" "retrans", "acregmin", "acregmax", "acdirmin", "acdirmax", "resvport", "readahead", "hostname", "timeout", "addr", "fh", "nfsv3", "sec", "principal", "nfsv4", "gssname", "allgssname", "dirpath", "minorversion", - "nametimeo", "negnametimeo", "nocto", "pnfs", "wcommitsize", + "nametimeo", "negnametimeo", "nocto", "noncontigwr", "pnfs", + "wcommitsize", NULL }; /* @@ -840,6 +841,8 @@ nfs_mount(struct mount *mp) args.flags |= NFSMNT_ALLGSSNAME; if (vfs_getopt(mp->mnt_optnew, "nocto", NULL, NULL) == 0) args.flags |= NFSMNT_NOCTO; + if (vfs_getopt(mp->mnt_optnew, "noncontigwr", NULL, NULL) == 0) + args.flags |= NFSMNT_NONCONTIGWR; if (vfs_getopt(mp->mnt_optnew, "pnfs", NULL, NULL) == 0) args.flags |= NFSMNT_PNFS; if (vfs_getopt(mp->mnt_optnew, "readdirsize", (void **)&opt, NULL) == 0) { @@ -1792,6 +1795,8 @@ void nfscl_retopts(struct nfsmount *nmp, &blen); nfscl_printopt(nmp, (nmp->nm_flag & NFSMNT_NOCTO) != 0, ",nocto", &buf, &blen); + nfscl_printopt(nmp, (nmp->nm_flag & NFSMNT_NONCONTIGWR) != 0, + ",noncontigwr", &buf, &blen); nfscl_printopt(nmp, (nmp->nm_flag & (NFSMNT_NOLOCKD | NFSMNT_NFSV4)) == 0, ",lockd", &buf, &blen); nfscl_printopt(nmp, (nmp->nm_flag & (NFSMNT_NOLOCKD | NFSMNT_NFSV4)) == Modified: stable/10/sys/fs/nfsclient/nfs_clvnops.c ============================================================================== --- stable/10/sys/fs/nfsclient/nfs_clvnops.c Mon Dec 30 21:06:08 2013 (r260106) +++ stable/10/sys/fs/nfsclient/nfs_clvnops.c Mon Dec 30 21:17:20 2013 (r260107) @@ -3079,6 +3079,10 @@ nfs_advlock(struct vop_advlock_args *ap) np->n_change = va.va_filerev; } } + /* Mark that a file lock has been acquired. */ + mtx_lock(&np->n_mtx); + np->n_flag |= NHASBEENLOCKED; + mtx_unlock(&np->n_mtx); } NFSVOPUNLOCK(vp, 0); return (0); @@ -3098,6 +3102,12 @@ nfs_advlock(struct vop_advlock_args *ap) error = ENOLCK; } } + if (error == 0 && ap->a_op == F_SETLK) { + /* Mark that a file lock has been acquired. */ + mtx_lock(&np->n_mtx); + np->n_flag |= NHASBEENLOCKED; + mtx_unlock(&np->n_mtx); + } } return (error); } Modified: stable/10/sys/fs/nfsclient/nfsnode.h ============================================================================== --- stable/10/sys/fs/nfsclient/nfsnode.h Mon Dec 30 21:06:08 2013 (r260106) +++ stable/10/sys/fs/nfsclient/nfsnode.h Mon Dec 30 21:17:20 2013 (r260107) @@ -157,6 +157,7 @@ struct nfsnode { #define NLOCKWANT 0x00010000 /* Want the sleep lock */ #define NNOLAYOUT 0x00020000 /* Can't get a layout for this file */ #define NWRITEOPENED 0x00040000 /* Has been opened for writing */ +#define NHASBEENLOCKED 0x00080000 /* Has been file locked. */ /* * Convert between nfsnode pointers and vnode pointers Modified: stable/10/sys/nfsclient/nfsargs.h ============================================================================== --- stable/10/sys/nfsclient/nfsargs.h Mon Dec 30 21:06:08 2013 (r260106) +++ stable/10/sys/nfsclient/nfsargs.h Mon Dec 30 21:17:20 2013 (r260107) @@ -99,5 +99,6 @@ struct nfs_args { #define NFSMNT_STRICT3530 0x10000000 /* Adhere strictly to RFC3530 */ #define NFSMNT_NOCTO 0x20000000 /* Don't flush attrcache on open */ #define NFSMNT_PNFS 0x40000000 /* Enable pNFS support */ +#define NFSMNT_NONCONTIGWR 0x80000000 /* Enable non-contiguous writes */ #endif