From owner-freebsd-fs@freebsd.org Mon May 16 21:14:04 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AED72B3EAED for ; Mon, 16 May 2016 21:14:04 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9E3DD1F1B for ; Mon, 16 May 2016 21:14:04 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: by mailman.ysv.freebsd.org (Postfix) id 99E50B3EAEC; Mon, 16 May 2016 21:14:04 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9988CB3EAEB for ; Mon, 16 May 2016 21:14:04 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id 2F6BF1F19; Mon, 16 May 2016 21:14:03 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c122-106-149-109.carlnfd1.nsw.optusnet.com.au (c122-106-149-109.carlnfd1.nsw.optusnet.com.au [122.106.149.109]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id 823C57837EC; Tue, 17 May 2016 07:13:56 +1000 (AEST) Date: Tue, 17 May 2016 07:13:55 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: fs@freebsd.org cc: rmacklem@freebsd.org Subject: fixes for i/o counting in nfs Message-ID: <20160517063058.E2021@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=TuMb/2jh c=1 sm=1 tr=0 a=R/f3m204ZbWUO/0rwPSMPw==:117 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=kj9zAlcOel0A:10 a=ao2s4AhymvbEqQ6vx_0A:9 a=8qM5l07LXi6s6vM6:21 a=vbcHcmFXaA1j43Cz:21 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 May 2016 21:14:04 -0000 nfs doesn't count block inputs in resource usage. It seems to count block outputs well enough (not very well, since buffering and threading causes some i/o's to be done by other threads where the counts are hard to see and harder to associate with the actual user). nfs doesn't support per-mount i/o counts for either input and output. These patches are for an old version of oldnfs. They apply cleanly to oldnfs in FreeBSD-10. I'm not sure if I found all the i/o's and don't trust my thread and mount pointer handling, but they seem to work reasonably and never find a null pointer. The per-mount i/o counts were easier to do than for most file systems since nfs isn't handicapped by using geom. The corresponding code in g_vfs_strategy() has a harder time finding the mount point and often fails, so must check for null pointers and not work when it can't find the mount point. This code doesn't even exist in the version that this patch is for (except I patch it in). X Index: nfs_bio.c X =================================================================== X --- nfs_bio.c (revision 181737) X +++ nfs_bio.c (working copy) X @@ -1568,6 +1581,14 @@ X case VREG: X uiop->uio_offset = ((off_t)bp->b_blkno) * DEV_BSIZE; X nfsstats.read_bios++; X + if (td == NULL) X + curthread->td_ru.ru_inblock++; /* XXX */ X + else X + td->td_ru.ru_inblock++; /* XXX? */ These are XXX'ed since I don't know if td is ever null or always right when it is non-null. But this seems to work right -- some counts go to normal threads and some to nfsiod's. X + if (LK_HOLDER(bp->b_lock.lk_lock) == LK_KERNPROC) X + vp->v_mount->mnt_stat.f_asyncreads++; /* XXX */ X + else X + vp->v_mount->mnt_stat.f_syncreads++; X error = (nmp->nm_rpcops->nr_readrpc)(vp, uiop, cr); This is XXX'ed since I don't trust the LK_KERNPROC check at all. This was blindly copied from g_vfs_strategy(). A separate count for async _reads_ is not very useful anyway. It is mostly for read-ahead. Most reads should be ahead, but complicated buffering in hardware and software makes them hard to count and the counts not very useful. X X if (!error) { X @@ -1674,10 +1695,16 @@ X io.iov_base = (char *)bp->b_data + bp->b_dirtyoff; X uiop->uio_rw = UIO_WRITE; X nfsstats.write_bios++; X + if (td == NULL) X + curthread->td_ru.ru_oublock++; /* XXX */ X + else X + td->td_ru.ru_oublock++; /* XXX? */ As above. X X if ((bp->b_flags & (B_ASYNC | B_NEEDCOMMIT | B_NOCACHE | B_CLUSTER)) == B_ASYNC) X + vp->v_mount->mnt_stat.f_asyncwrites++, X iomode = NFSV3WRITE_UNSTABLE; X else X + vp->v_mount->mnt_stat.f_syncwrites++, X iomode = NFSV3WRITE_FILESYNC; Here the sync/async decision is easy to make correctly. The patch uses a comma splice hack to keep the patch small. X X error = (nmp->nm_rpcops->nr_writerpc)(vp, uiop, cr, &iomode, &must_commit); X Index: nfs_vnops.c X =================================================================== X --- nfs_vnops.c (revision 181737) X +++ nfs_vnops.c (working copy) X @@ -3138,7 +3290,6 @@ X bp->b_iocmd = BIO_WRITE; X X bufobj_wref(bp->b_bufobj); X - curthread->td_ru.ru_oublock++; X splx(s); X X /* This is now counted in nfs_bio.c, and the results are much the same. Apparently it makes little difference to always use curthread. In file systems generally, we pass around td's and they are usually useless, but if they are good for anything at all then it is to record the (first) originator of the i/o so as to charge the originator and not a daemon. I don't know if they are used for that. Bruce