From owner-freebsd-current@FreeBSD.ORG Wed Jun 21 22:00:54 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 37A4F16A47B; Wed, 21 Jun 2006 22:00:54 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8263E43D62; Wed, 21 Jun 2006 22:00:53 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 798FC46C25; Wed, 21 Jun 2006 18:00:52 -0400 (EDT) Date: Wed, 21 Jun 2006 23:00:52 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: John Baldwin In-Reply-To: <200606211745.42525.jhb@freebsd.org> Message-ID: <20060621225953.U8526@fledge.watson.org> References: <20060612054115.GA42379@xor.obsecurity.org> <20060621201927.GJ28128@groat.ugcs.caltech.edu> <20060621214346.G8526@fledge.watson.org> <200606211745.42525.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Paul Allen , freebsd-current@freebsd.org, current@freebsd.org Subject: Re: FILEDESC_LOCK() implementation X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 22:00:54 -0000 On Wed, 21 Jun 2006, John Baldwin wrote: >> The problem is this: when you have threads in the same process, file >> descriptor lookup is performed against a common file descriptor array. >> That array is protected by a lock, the filedesc lock. When lots of threads >> simultaneously perform file descriptor operations, they contend on the file >> descriptor array lock. So if you have 30 threads all doing I/O, they are >> constantly looking up file descriptors and bumping into each other. This >> is particularly noticeable for network workloads, where many operations are >> very fast, and so they occur in significant quantity. The M:N threading >> library actually handles this quite well by bounding the number of threads >> trying to acquire the lock to the number of processors, but with libthr you >> get pretty bad performance. This contention problem also affects MySQL, >> etc. >> >> You can imagine a number of ways to work on this, but it's a tricky problem >> that has to be looked at carefully. > > Are the lookup operations using a shared lock so that only things like open > and close would actually contend? I'm not sure anyone has tried that. The semantics of the filedesc lock seem a bit complicated, I don't remember why that is right now. Robert N M Watson Computer Laboratory University of Cambridge