From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 21 13:24:29 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 375FC16A494; Thu, 21 Dec 2006 13:24:29 +0000 (UTC) (envelope-from ssouhlal@FreeBSD.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id D8DA713C483; Thu, 21 Dec 2006 13:24:28 +0000 (UTC) (envelope-from ssouhlal@FreeBSD.org) Received: from [192.168.0.100] (c-67-188-127-3.hsd1.ca.comcast.net [67.188.127.3]) by elvis.mu.org (Postfix) with ESMTP id 644231A4D8B; Wed, 20 Dec 2006 22:08:07 -0800 (PST) Message-ID: <458A249D.3030502@FreeBSD.org> Date: Wed, 20 Dec 2006 22:07:25 -0800 From: Suleiman Souhlal User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051204) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Attilio Rao References: <20061220041843.GA10511@dwpc.dwlabs.ca> <3bbf2fe10612200414j4c1c01ecr7b37e956b70b01fa@mail.gmail.com> In-Reply-To: <3bbf2fe10612200414j4c1c01ecr7b37e956b70b01fa@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Duane Whitty Subject: Re: Locking fundamentals X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Dec 2006 13:24:29 -0000 Attilio Rao wrote: > 2006/12/20, Duane Whitty : > >> Hello again, >> >> It seems to me that understanding locking holds the key to >> understanding fbsd internals. >> >> Could someone review my understanding of fbsd locking fundamentals. >> (No assertions here, just questions) >> >> lock_mgr >> -------------------- >> mutexes|sx_lock >> ------------------- ^ >> atomic | mem barriers | > > > Our current locking hierarchy is basically different: > > III level: lockmgr - sema - sx > II level: mutex (sleep/spin/pool) - rwlock - refcount - cv - msleep > I level: atomic instructions - memory barriers - sleepqueues/turnstiles > > (a lower lever means that the upper layer primitives use it as a base. > ie: sx locks are build using 1 pool > mutex and 2 condition variables). > > This scheme is far from being perfect due to the presence of 'level 3 > primitives' which should never exist. > Currently, there is an ongoing efforts to take all the top layer > primitives to the level II. > > On the other side, level I primitives should never be used directly by > kernel code, but should only be used as a bottom layer for > syncronizing primitives. All you need to care is in the layer 2 and 3 > (and possibly should switch to layer 2). I disagree. There are many uses of atomic operations in the kernel that are not for locks or refcounts. It's a bad idea to use locks if you can achieve the same thing locklessly, with atomic operations. I would personally also add "critical sections" (critical_enter()/critical_exit()) at level I. They can be used instead of locks when you know your data will only be accessed on one CPU, and you only need to protect it from (non-FAST) interrupt handlers. -- Suleiman