From owner-freebsd-current@FreeBSD.ORG Wed Jun 21 20:21:55 2006 Return-Path: X-Original-To: current@FreeBSD.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0861516A47A; Wed, 21 Jun 2006 20:21:55 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D47F43D48; Wed, 21 Jun 2006 20:21:54 +0000 (GMT) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (0kzjppn8jlvmysro@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id k5LKLrfv088026; Wed, 21 Jun 2006 13:21:54 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id k5LKLpYB088024; Wed, 21 Jun 2006 13:21:51 -0700 (PDT) (envelope-from jmg) Date: Wed, 21 Jun 2006 13:21:51 -0700 From: John-Mark Gurney To: Robert Watson Message-ID: <20060621202151.GD82074@funkthat.com> Mail-Followup-To: Robert Watson , current@FreeBSD.org, Kris Kennaway References: <20060612054115.GA42379@xor.obsecurity.org> <20060621183543.GC82074@funkthat.com> <20060621194412.N8526@fledge.watson.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060621194412.N8526@fledge.watson.org> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html Cc: current@FreeBSD.org, Kris Kennaway Subject: Re: FILEDESC_LOCK() implementation X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 20:21:55 -0000 Robert Watson wrote this message on Wed, Jun 21, 2006 at 19:46 +0100: > On Wed, 21 Jun 2006, John-Mark Gurney wrote: > > >Kris Kennaway wrote this message on Mon, Jun 12, 2006 at 01:41 -0400: > >>I fixed mutex profiling to a) not be as wrong and b) not suck so very > >>much, and here is a revised profiling trace from mysql supersmack on a 12 > >>cpu E4500, sorted by ratio of cnt_lock/count; filedesc lock contention > >>(via FILEDESC_[UN]LOCK()) is the major mutex contention problem. > > > >Should we also look at breaking down filedesc lock to have multiple locks > >over the range? I am thinking of writing a program that will have 32 > >threads (sun4v) and all threads will be doing heavy i/o, and will be even > >more heavily contested on FILEDESC than the supersmack benchmark would > >be... > > > >Though this doesn't solve the problem of all 32 threads trying to do i/o > >on a fd in the same block though... > > src/tools/tools/netrate/{http,httpd}, running in threaded mode (-t). http > is a client, and accesses lots of independent fds from different threads, > contending the filedesc lock but not a single fd lock, whereas httpd will > do both, due to accepting connections. Well, in my lfhttpd, I'd only ever have one thread accepting connections, but then I'd have tons of threads contending on the kqueue fd... > I would optimize very carefully here, the trade-offs are tricky, and we may > find that by making locking more complex, we cause cache problems, increase > lock hold periods, etc, even if we decrease contention. I've wondered a > bit about a model where we loan fd's to threads to optimize repeated access > to the same fd by the same thread, but this mostly makes sense in the > context of a 1:1 model rather than an m:n model. And wouldn't work for everyone pounding for kq... Hmmm.. we could remove kq from the fd model too... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."