From owner-freebsd-current@FreeBSD.ORG Thu Jun 22 21:45:57 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3689516A882; Thu, 22 Jun 2006 21:45:57 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3A724581E; Thu, 22 Jun 2006 20:52:30 +0000 (GMT) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.4/8.13.4) with ESMTP id k5MKqMCG051151; Thu, 22 Jun 2006 16:52:29 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Robert Watson Date: Thu, 22 Jun 2006 16:43:45 -0400 User-Agent: KMail/1.9.1 References: <20060612054115.GA42379@xor.obsecurity.org> <200606211745.42525.jhb@freebsd.org> <20060621225953.U8526@fledge.watson.org> In-Reply-To: <20060621225953.U8526@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200606221643.45634.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Thu, 22 Jun 2006 16:52:30 -0400 (EDT) X-Virus-Scanned: ClamAV 0.87.1/1561/Thu Jun 22 11:40:00 2006 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on server.baldwin.cx Cc: Paul Allen , freebsd-current@freebsd.org, current@freebsd.org Subject: Re: FILEDESC_LOCK() implementation X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 21:45:57 -0000 On Wednesday 21 June 2006 18:00, Robert Watson wrote: > > On Wed, 21 Jun 2006, John Baldwin wrote: > > >> The problem is this: when you have threads in the same process, file > >> descriptor lookup is performed against a common file descriptor array. > >> That array is protected by a lock, the filedesc lock. When lots of > >> threads simultaneously perform file descriptor operations, they contend > >> on the file descriptor array lock. So if you have 30 threads all doing > >> I/O, they are constantly looking up file descriptors and bumping into > >> each other. This is particularly noticeable for network workloads, where > >> many operations are very fast, and so they occur in significant quantity. > >> The M:N threading library actually handles this quite well by bounding > >> the number of threads trying to acquire the lock to the number of > >> processors, but with libthr you get pretty bad performance. This > >> contention problem also affects MySQL, etc. > >> > >> You can imagine a number of ways to work on this, but it's a tricky > >> problem that has to be looked at carefully. > > > > Are the lookup operations using a shared lock so that only things like > > open and close would actually contend? > > I'm not sure anyone has tried that. The semantics of the filedesc lock seem > a bit complicated, I don't remember why that is right now. It used to be an sx lock and it was converted to the current "thing" because sx locks were too slow. I'd rather expend the effort on changing sx locks to use the same atomic ops as rwlocks (and use sleepq(9) directly rather than cv(9)) at which point we can probably just make it an sx lock again and use slock's for things like fget(). -- John Baldwin