From owner-freebsd-current@FreeBSD.ORG Mon Jun 12 05:41:16 2006 Return-Path: X-Original-To: current@FreeBSD.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F2CD16A41F for ; Mon, 12 Jun 2006 05:41:16 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4325A43D4C for ; Mon, 12 Jun 2006 05:41:16 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 26CDB1A4DA0 for ; Sun, 11 Jun 2006 22:41:16 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id A55DC515BB; Mon, 12 Jun 2006 01:41:15 -0400 (EDT) Date: Mon, 12 Jun 2006 01:41:15 -0400 From: Kris Kennaway To: current@FreeBSD.org Message-ID: <20060612054115.GA42379@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9amGYk9869ThD9tj" Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Cc: Subject: FILEDESC_LOCK() implementation X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jun 2006 05:41:16 -0000 --9amGYk9869ThD9tj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I fixed mutex profiling to a) not be as wrong and b) not suck so very much, and here is a revised profiling trace from mysql supersmack on a 12 cpu E4500, sorted by ratio of cnt_lock/count; filedesc lock contention (via FILEDESC_[UN]LOCK()) is the major mutex contention problem. 0 0 55381 0 672 3195 .057 kern/kern_sig.c:998 (process lock) 0 0 31451 0 3657 2446 .077 kern/subr_sleepqueue.c:374 (process lock) 0 0 31706 0 3570 2623 .082 kern/kern_sx.c:245 (lockbuilder mtxpool) 0 0 458877 0 83592 48024 .104 kern/kern_descrip.c:1988 (filedesc structure) 0 0 45069 0 5342 6085 .135 kern/kern_umtx.c:195 (umtxq_lock) 0 0 458877 0 22609 72747 .158 kern/kern_descrip.c:1967 (filedesc structure) 0 0 40046 0 6845 6925 .172 kern/uipc_usrreq.c:621 (unp_global_mtx) 0 0 31706 0 3159 6075 .191 kern/kern_sx.c:157 (lockbuilder mtxpool) 0 0 66539 0 22823 12844 .193 kern/kern_descrip.c:420 (filedesc structure) 0 0 66569 0 5352 20913 .314 kern/kern_descrip.c:368 (filedesc structure) 0 0 110665 0 38632 76441 .690 kern/kern_synch.c:222 (filedesc structure) # kern/kern_descrip.c:1967,1988 are _fget() kern/kern_descrip.c:368,420 is kern_fcntl() kern/kern_synch.c:222 is msleep() (called in a loop from FILEDESC_LOCK) I wonder if something better can be done with the funky home-grown locking in FILEDESC_LOCK() (see ) to make it more light-weight? /* Lock a file descriptor table. */ #define FILEDESC_LOCK(fd) \ do { \ mtx_lock(&(fd)->fd_mtx); \ (fd)->fd_wanted++; \ while ((fd)->fd_locked) \ msleep(&(fd)->fd_locked, &(fd)->fd_mtx, PLOCK, "fdesc", 0); \ (fd)->fd_locked = 2; \ (fd)->fd_wanted--; \ mtx_unlock(&(fd)->fd_mtx); \ } while (0) Kris --9amGYk9869ThD9tj Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEjP56Wry0BWjoQKURAtfFAJ95pnSOK7si3PseqRR4l1pC5lNXXgCg0YGk tcGvkXhFvjY8sEiSogi1XHo= =kX+Z -----END PGP SIGNATURE----- --9amGYk9869ThD9tj--