From owner-freebsd-current@FreeBSD.ORG Sat Jul 7 17:14:18 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AA0A216A421 for ; Sat, 7 Jul 2007 17:14:18 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from itchy.rabson.org (mailgate.nlsystems.com [80.177.232.242]) by mx1.freebsd.org (Postfix) with ESMTP id 5E7CE13C447 for ; Sat, 7 Jul 2007 17:14:18 +0000 (UTC) (envelope-from dfr@rabson.org) Received: from herring.rabson.org (herring.rabson.org [80.177.232.250]) by itchy.rabson.org (8.13.3/8.13.3) with ESMTP id l67GZ3nO040535 for ; Sat, 7 Jul 2007 17:35:03 +0100 (BST) (envelope-from dfr@rabson.org) From: Doug Rabson To: freebsd-current@freebsd.org User-Agent: KMail/1.9.6 MIME-Version: 1.0 Content-Disposition: inline X-Length: 2655 X-UID: 298 Date: Sat, 7 Jul 2007 17:35:02 +0100 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <200707071735.02973.dfr@rabson.org> X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=failed version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on itchy.rabson.org X-Virus-Scanned: ClamAV 0.87.1/3607/Fri Jul 6 00:51:19 2007 on itchy.rabson.org X-Virus-Status: Clean Subject: ZFS leaking vnodes (sort of) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2007 17:14:18 -0000 I've been testing ZFS recently and I noticed some performance issues while doing large-scale port builds on a ZFS mounted /usr/ports tree. Eventually I realised that virtually nothing ever ended up on the vnode free list. This meant that when the system reached its maximum vnode limit, it had to resort to reclaiming vnodes from the various filesystem's active vnode lists (via vlrureclaim). Since those lists are not sorted in LRU order, this led to pessimal cache performance after the system got into that state. I looked a bit closer at the ZFS code and poked around with DDB and I think the problem was caused by a couple of extraneous calls to vhold when creating a new ZFS vnode. On FreeBSD, getnewvnode returns a vnode which is already held (not on the free list) so there is no need to call vhold again. This patch appears to fix the problem (only very lightly tested): Index: zfs_vnops.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c,v retrieving revision 1.22 diff -u -r1.22 zfs_vnops.c --- zfs_vnops.c 28 May 2007 02:37:43 -0000 1.22 +++ zfs_vnops.c 7 Jul 2007 13:01:41 -0000 @@ -3493,7 +3493,7 @@ rele = 0; vp->v_data = NULL; ASSERT(vp->v_holdcnt > 1); - vdropl(vp); + VI_UNLOCK(vp); if (!zp->z_unlinked && rele) VFS_RELE(zfsvfs->z_vfs); return (0); Index: zfs_znode.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c,v retrieving revision 1.8 diff -u -r1.8 zfs_znode.c --- zfs_znode.c 6 May 2007 19:05:37 -0000 1.8 +++ zfs_znode.c 7 Jul 2007 13:17:32 -0000 @@ -115,7 +115,6 @@ ASSERT(error == 0); zp->z_vnode = vp; vp->v_data = (caddr_t)zp; - vhold(vp); vp->v_vnlock->lk_flags |= LK_CANRECURSE; vp->v_vnlock->lk_flags &= ~LK_NOSHARE; } else { @@ -601,7 +600,6 @@ ASSERT(err == 0); vp = ZTOV(zp); vp->v_data = (caddr_t)zp; - vhold(vp); vp->v_vnlock->lk_flags |= LK_CANRECURSE; vp->v_vnlock->lk_flags &= ~LK_NOSHARE; vp->v_type = IFTOVT((mode_t)zp->z_phys->zp_mode);