From owner-freebsd-performance@FreeBSD.ORG Thu Feb 21 11:20:06 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94D6916A400; Thu, 21 Feb 2008 11:20:06 +0000 (UTC) (envelope-from bbump@rsts.org) Received: from mail.rsts.org (host-82-161-107-208.midco.net [208.107.161.82]) by mx1.freebsd.org (Postfix) with ESMTP id 24A1A13C4E1; Thu, 21 Feb 2008 11:20:05 +0000 (UTC) (envelope-from bbump@rsts.org) Received: from mail.rsts.org (localhost [127.0.0.1]) by mail.rsts.org (8.13.6/8.12.11) with ESMTP id m1LBK4sk002821; Thu, 21 Feb 2008 04:20:04 -0700 (MST) (envelope-from bbump@rsts.org) Received: from localhost (bbump@localhost) by mail.rsts.org (8.13.6/8.12.11/Submit) with ESMTP id m1LBK4LZ002818; Thu, 21 Feb 2008 04:20:04 -0700 (MST) (envelope-from bbump@rsts.org) X-Authentication-Warning: mail.rsts.org: bbump owned process doing -bs Date: Thu, 21 Feb 2008 04:20:04 -0700 (MST) From: Brett Bump To: Kostik Belousov In-Reply-To: <20080220172216.GM57756@deviant.kiev.zoral.com.ua> Message-ID: <20080221030314.Q2297@mail.rsts.org> References: <20080214114759.R75215@mail.rsts.org> <47B49A16.1080103@FreeBSD.org> <20080214131026.Y75492@mail.rsts.org> <200802151440.m1FEeGVr084431@lava.sentex.ca> <20080215085714.K79197@mail.rsts.org> <47B5F7BF.9050608@FreeBSD.org> <20080219145105.C96725@mail.rsts.org> <20080220172216.GM57756@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Kris Kennaway , freebsd-performance@freebsd.org, Mike Tancsa Subject: Re: System perforamance 4.x vs. 5.x and 6.x X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Feb 2008 11:20:06 -0000 On Wed, 20 Feb 2008, Kostik Belousov wrote: > I cannot reproduce it locally. With patch applied, it compiles both > GENERIC and GENERIC with options QUOTA added just fine. > > Check for partially applied patch. > Thanks Kostik. You can double check me on sizes, but it would appear that all files in the patch were touched, and I double checked my sources with: ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/6.2-RELEASE/src which are identical (this is still a fresh install, I did not run cvsup). -bash-2.05b$ grep RCS quotas-RELENG_6-20070623-1455.patch RCS file: /usr/local/arch/ncvs/src/sys/kern/vfs_syscalls.c,v RCS file: /usr/local/arch/ncvs/src/sys/ufs/ffs/ffs_softdep.c,v RCS file: /usr/local/arch/ncvs/src/sys/ufs/ffs/ffs_vfsops.c,v RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/quota.h,v RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_inode.c,v RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_lookup.c,v RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_quota.c,v RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_vnops.c,v -bash-2.05b$ cd /usr/src/sys -bash-2.05b$ ls -l kern/vfs_syscalls.c ufs/ffs/ffs_softdep.c ufs/ffs/ffs_vfsops.c ufs/ufs/quota.h ufs/ufs/ufs_inode.c ufs/ufs/ufs_lookup.c ufs/ufs/ufs_quota.c ufs/ufs/ufs_vnops.c -rw-r--r-- 1 root wheel 109156 Feb 19 14:32 kern/vfs_syscalls.c -rw-r--r-- 1 root wheel 186964 Feb 19 14:33 ufs/ffs/ffs_softdep.c -rw-r--r-- 1 root wheel 47272 Feb 19 14:33 ufs/ffs/ffs_vfsops.c -rw-r--r-- 1 root wheel 7923 Feb 19 14:34 ufs/ufs/quota.h -rw-r--r-- 1 root wheel 6364 Feb 19 14:34 ufs/ufs/ufs_inode.c -rw-r--r-- 1 root wheel 36499 Feb 19 14:34 ufs/ufs/ufs_lookup.c -rw-r--r-- 1 root wheel 31107 Feb 19 14:34 ufs/ufs/ufs_quota.c -rw-r--r-- 1 root wheel 62960 Feb 19 14:34 ufs/ufs/ufs_vnops.c The only patch barking a problem would appear to be ufs_quota.c: -------------------------- |Index: ufs/ufs/ufs_quota.c |=================================================================== |RCS file: /usr/local/arch/ncvs/src/sys/ufs/ufs/ufs_quota.c,v |retrieving revision 1.74.2.6 |diff -u -r1.74.2.6 ufs_quota.c |--- ufs/ufs/ufs_quota.c 1 Feb 2007 04:45:43 -0000 1.74.2.6 |+++ ufs/ufs/ufs_quota.c 25 Jun 2007 14:52:48 -0000 -------------------------- Patching file ufs_quota.c using Plan A... Hunk #1 failed at 72. Hunk #2 succeeded at 114 (offset -5 lines). Hunk #3 succeeded at 122 (offset -5 lines). Hunk #4 failed at 142. Hunk #5 succeeded at 158 (offset -13 lines). Hunk #6 succeeded at 175 (offset -5 lines). Hunk #7 succeeded at 214 (offset -13 lines). Hunk #8 succeeded at 236 (offset -5 lines). Hunk #9 succeeded at 244 (offset -13 lines). Hunk #10 succeeded at 271 (offset -5 lines). Hunk #11 succeeded at 296 (offset -13 lines). Hunk #12 succeeded at 317 with fuzz 1 (offset -4 lines). Hunk #13 succeeded at 318 (offset -13 lines). Hunk #14 succeeded at 376 (offset -4 lines). Hunk #15 failed at 390. Hunk #16 succeeded at 397 (offset -13 lines). Hunk #17 succeeded at 425 (offset -4 lines). Hunk #18 failed at 471. Hunk #19 succeeded at 490 (offset -19 lines). Hunk #20 succeeded at 567 (offset -4 lines). Hunk #21 succeeded at 589 (offset -19 lines). Hunk #22 succeeded at 631 (offset -4 lines). Hunk #23 succeeded at 660 (offset -19 lines). Hunk #24 succeeded at 791 (offset -4 lines). Hunk #25 succeeded at 809 (offset -19 lines). Hunk #26 succeeded at 864 (offset -4 lines). Hunk #27 succeeded at 878 (offset -19 lines). Hunk #28 succeeded at 920 (offset -4 lines). Hunk #29 succeeded at 927 (offset -19 lines). Hunk #30 succeeded at 974 (offset -4 lines). Hunk #31 succeeded at 984 (offset -19 lines). Hunk #32 succeeded at 1018 (offset -4 lines). Hunk #33 succeeded at 1019 (offset -19 lines). Hunk #34 failed at 1063. Hunk #35 succeeded at 1197 (offset -9 lines). Hunk #36 succeeded at 1209 (offset -19 lines). Hunk #37 succeeded at 1261 (offset -9 lines). Hunk #38 succeeded at 1281 (offset -19 lines). Hunk #39 succeeded at 1323 (offset -9 lines). Hunk #40 succeeded at 1365 (offset -19 lines). Hunk #41 succeeded at 1410 (offset -9 lines). Hunk #42 succeeded at 1412 (offset -19 lines). 5 out of 42 hunks failed--saving rejects to ufs_quota.c.rej done The contents of ufs_quota.c.rej contain: -bash-2.05b$ cat ufs_quota.c.rej *************** *** 72,83 **** */ static char *quotatypes[] = INITQFNAMES; - static int chkdqchg(struct inode *, ufs2_daddr_t, struct ucred *, int); - static int chkiqchg(struct inode *, int, struct ucred *, int); static int dqget(struct vnode *, - u_long, struct ufsmount *, int, struct dquot **); static int dqsync(struct vnode *, struct dquot *); static void dqflush(struct vnode *); #ifdef DIAGNOSTIC static void dqref(struct dquot *); --- 72,85 ---- */ static char *quotatypes[] = INITQFNAMES; + static int chkdqchg(struct inode *, ufs2_daddr_t, struct ucred *, int, int *); + static int chkiqchg(struct inode *, int, struct ucred *, int, int *); static int dqget(struct vnode *, + u_long, struct ufsmount *, int, struct dquot **); static int dqsync(struct vnode *, struct dquot *); static void dqflush(struct vnode *); + static int quotaoff1(struct thread *td, struct mount *mp, int type); + static int quotaoff_inchange(struct thread *td, struct mount *mp, int type); #ifdef DIAGNOSTIC static void dqref(struct dquot *); *************** *** 142,148 **** struct dquot *dq; ufs2_daddr_t ncurblocks; struct vnode *vp = ITOV(ip); - int i, error; /* * Disk quotas must be turned off for system files. Currently --- 142,148 ---- struct dquot *dq; ufs2_daddr_t ncurblocks; struct vnode *vp = ITOV(ip); + int i, error, warn, do_check; /* * Disk quotas must be turned off for system files. Currently *************** *** 335,345 **** * Issue an error message if appropriate. */ static int - chkiqchg(ip, change, cred, type) struct inode *ip; int change; struct ucred *cred; int type; { struct dquot *dq = ip->i_dquot[type]; ino_t ncurinodes = dq->dq_curinodes + change; --- 390,401 ---- * Issue an error message if appropriate. */ static int + chkiqchg(ip, change, cred, type, warn) struct inode *ip; int change; struct ucred *cred; int type; + int *warn; { struct dquot *dq = ip->i_dquot[type]; ino_t ncurinodes = dq->dq_curinodes + change; *************** *** 411,425 **** */ if ((int)ip->i_uid < 0 || (int)ip->i_gid < 0) return; for (i = 0; i < MAXQUOTAS; i++) { if (ump->um_quotas[i] == NULLVP || (ump->um_qflags[i] & (QTF_OPENING|QTF_CLOSING))) continue; if (ip->i_dquot[i] == NODQUOT) { vprint("chkdquot: missing dquot", ITOV(ip)); panic("chkdquot: missing dquot"); } } } #endif --- 471,489 ---- */ if ((int)ip->i_uid < 0 || (int)ip->i_gid < 0) return; + + UFS_LOCK(ump); for (i = 0; i < MAXQUOTAS; i++) { if (ump->um_quotas[i] == NULLVP || (ump->um_qflags[i] & (QTF_OPENING|QTF_CLOSING))) continue; if (ip->i_dquot[i] == NODQUOT) { + UFS_UNLOCK(ump); vprint("chkdquot: missing dquot", ITOV(ip)); panic("chkdquot: missing dquot"); } } + UFS_UNLOCK(ump); } #endif *************** *** 851,905 **** struct dquot **dqp; { struct thread *td = curthread; /* XXX */ - struct dquot *dq; struct dqhash *dqh; struct vnode *dqvp; struct iovec aiov; struct uio auio; - int error; /* XXX: Disallow negative id values to prevent the * creation of 100GB+ quota data files. */ if ((int)id < 0) return (EINVAL); dqvp = ump->um_quotas[type]; if (dqvp == NULLVP || (ump->um_qflags[type] & QTF_CLOSING)) { *dqp = NODQUOT; return (EINVAL); } /* * Check the cache first. */ dqh = DQHASH(dqvp, id); - LIST_FOREACH(dq, dqh, dq_hash) { - if (dq->dq_id != id || - dq->dq_ump->um_quotas[dq->dq_type] != dqvp) - continue; /* - * Cache hit with no references. Take - * the structure off the free list. */ - if (dq->dq_cnt == 0) - TAILQ_REMOVE(&dqfreelist, dq, dq_freelist); - DQREF(dq); - *dqp = dq; - return (0); } /* - * Not in cache, allocate a new one. */ if (TAILQ_FIRST(&dqfreelist) == NODQUOT && numdquot < MAXQUOTAS * desiredvnodes) desireddquot += DQUOTINC; if (numdquot < desireddquot) { - dq = (struct dquot *)malloc(sizeof *dq, M_DQUOT, - M_WAITOK | M_ZERO); numdquot++; } else { if ((dq = TAILQ_FIRST(&dqfreelist)) == NULL) { tablefull("dquot"); *dqp = NODQUOT; return (EUSERS); } if (dq->dq_cnt || (dq->dq_flags & DQ_MOD)) --- 1063,1184 ---- struct dquot **dqp; { struct thread *td = curthread; /* XXX */ + struct dquot *dq, *dq1; struct dqhash *dqh; struct vnode *dqvp; struct iovec aiov; struct uio auio; + int vfslocked, dqvplocked, error; + + #ifdef DEBUG_VFS_LOCKS + if (vp != NULLVP) + ASSERT_VOP_ELOCKED(vp, "dqget"); + #endif + + if (vp != NULLVP && *dqp != NODQUOT) { + return (0); + } /* XXX: Disallow negative id values to prevent the * creation of 100GB+ quota data files. */ if ((int)id < 0) return (EINVAL); + + UFS_LOCK(ump); dqvp = ump->um_quotas[type]; if (dqvp == NULLVP || (ump->um_qflags[type] & QTF_CLOSING)) { *dqp = NODQUOT; + UFS_UNLOCK(ump); return (EINVAL); } + vref(dqvp); + UFS_UNLOCK(ump); + error = 0; + dqvplocked = 0; + /* * Check the cache first. */ dqh = DQHASH(dqvp, id); + DQH_LOCK(); + dq = dqhashfind(dqh, id, dqvp); + if (dq != NULL) { + DQH_UNLOCK(); + hfound: DQI_LOCK(dq); + DQI_WAIT(dq, PINOD+1, "dqget"); + DQI_UNLOCK(dq); + if (dq->dq_ump == NULL) { + dqrele(vp, dq); + dq = NODQUOT; + error = EIO; + } + *dqp = dq; + vfslocked = VFS_LOCK_GIANT(dqvp->v_mount); + if (dqvplocked) + vput(dqvp); + else + vrele(dqvp); + VFS_UNLOCK_GIANT(vfslocked); + return (error); + } + + /* + * Quota vnode lock is before DQ_LOCK. Acquire dqvp lock there + * since new dq will appear on the hash chain DQ_LOCKed. + */ + if (vp != dqvp) { + DQH_UNLOCK(); + vn_lock(dqvp, LK_SHARED | LK_RETRY, td); + dqvplocked = 1; + DQH_LOCK(); /* + * Recheck the cache after sleep for quota vnode lock. */ + dq = dqhashfind(dqh, id, dqvp); + if (dq != NULL) { + DQH_UNLOCK(); + goto hfound; + } } + /* + * Not in cache, allocate a new one or take it from the + * free list. */ if (TAILQ_FIRST(&dqfreelist) == NODQUOT && numdquot < MAXQUOTAS * desiredvnodes) desireddquot += DQUOTINC; if (numdquot < desireddquot) { numdquot++; + DQH_UNLOCK(); + dq1 = (struct dquot *)malloc(sizeof *dq, M_DQUOT, + M_WAITOK | M_ZERO); + mtx_init(&dq1->dq_lock, "dqlock", NULL, MTX_DEF); + DQH_LOCK(); + /* + * Recheck the cache after sleep for memory. + */ + dq = dqhashfind(dqh, id, dqvp); + if (dq != NULL) { + numdquot--; + DQH_UNLOCK(); + mtx_destroy(&dq1->dq_lock); + free(dq1, M_DQUOT); + goto hfound; + } + dq = dq1; } else { if ((dq = TAILQ_FIRST(&dqfreelist)) == NULL) { + DQH_UNLOCK(); tablefull("dquot"); *dqp = NODQUOT; + vfslocked = VFS_LOCK_GIANT(dqvp->v_mount); + if (dqvplocked) + vput(dqvp); + else + vrele(dqvp); + VFS_UNLOCK_GIANT(vfslocked); return (EUSERS); } if (dq->dq_cnt || (dq->dq_flags & DQ_MOD)) Brett