From owner-freebsd-current@FreeBSD.ORG Fri Dec 15 17:37:54 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BA2D616A494 for ; Fri, 15 Dec 2006 17:37:54 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.184]) by mx1.FreeBSD.org (Postfix) with ESMTP id DED8043CA3 for ; Fri, 15 Dec 2006 17:36:09 +0000 (GMT) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so1134823nfc for ; Fri, 15 Dec 2006 09:37:51 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=RRaX9Px1l4UqXAJQz5sxY7A9HR2KkgOqADj/g4yoxUHMFTr8GzZaVPVbobZZdE+ZcI6OnddRs/QAVAYZd7Yw4Oa5dfZ/wmbEx2LWJDadIv8olpfQskwPa25HDaIl3b5aDPnCfEH0AFzg63wYyXdFS3jmSCEJjnYDOgE/Wa1QFDc= Received: by 10.48.48.1 with SMTP id v1mr146695nfv.1166204265909; Fri, 15 Dec 2006 09:37:45 -0800 (PST) Received: by 10.82.178.4 with HTTP; Fri, 15 Dec 2006 09:37:45 -0800 (PST) Message-ID: <3bbf2fe10612150937q675213e8tfb646401e95db8ec@mail.gmail.com> Date: Fri, 15 Dec 2006 18:37:45 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "John Baldwin" In-Reply-To: <200612131226.48087.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <456950AF.3090308@sh.cvut.cz> <200612121412.13551.jhb@freebsd.org> <20061213133411.S1136@delplex.bde.org> <200612131226.48087.jhb@freebsd.org> X-Google-Sender-Auth: 3426cf3bf6140c4e Cc: Bruce Evans , Suleiman Souhlal , freebsd-current@freebsd.org, bde@freebsd.org, Kostik Belousov , tegge@freebsd.org Subject: Re: kqueue LOR X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Dec 2006 17:37:54 -0000 2006/12/13, John Baldwin : > On Tuesday 12 December 2006 21:48, Bruce Evans wrote: > > > Memory barriers just specify ordering, they don't ensure a cache flush so > > > another CPU reads up to date values. You can use memory barriers in > > > conjunction with atomic operations on a variable to ensure that you can > > > safely read other variables (which is what locks do). For example, in > this > > > > I thought that the acquire/release variants of atomic ops guarantee > > this. They seem to be documented to do this, while mutexes don't seem > > to be documented to do this. The MI (?) implementation of mutexes > > depends on atomic_cmpset_{acq,rel}_ptr() doing this. > > The acq/rel just specify ordering. As Attilio mentioned, we assume that the > atomic_cmpset() that sets the contested flag will fail while racing with > another CPU (even if the CPU can't see the new value, as long as it fails and > keeps spinning mutexes will still work). The 'rel' barrier on CPU A when > releasing a lock forces all the other writes to be posted (and eventually > become "visible") to other CPUs before the write that releases the lock. > The 'acq' barrier on CPU B when acquiring the lock forces the CPU to not > reorder any reads before it acquires the lock, so this makes you not read any > data until you have the lock. Thus, once CPU B has waited long enough > to "see" the write from A to release the lock, we know that 1) it can > also "see" all the other writes from that CPU that the lock protected, and 2) > B hasn't tried to read any of them yet so it shouldn't have any stale values > in registers. None of this requires the OS to do a cache flush. (If you > have an SMP system where the cache can still hold stale values after another > CPU updates values in memory where it is "visible" to the CPU acquiring the > lock, then _acq might need to flush the cache, but that would be a property > of that architecture. However, even that would not require cache flushes so > long as the stale values were evicted from the cache such that they honor the > memory barrier and you don't see the new value of the lock until you see the > new values of the earlier data.) just a note: you can have a pratical overview of this with rwlocks implementation (and with the difference between the exclusive case and the shared case). When we have a 'write' case we just apply the mutex semantic (with very few implementation changes details), so we have to use and acquiring memory barrier for locking and a rel memory barrier for unlocking. The thing is very different when acquiring a rwlock into the 'read' case. Even if it is simple to understand that an acq semantic is needed for locking (due to the fact that we might force the CPU to not read any stale value before to acquire the lock), it is also simple to understand that, since no (protected) value is updated due to the semantic of read lock, there is no real necessity to use a release (or whatelse) memory barrier (since that seriality would be assured from the next thread acq barrier again). In a more pratical manner, since we can use it in this way: struct foo_softc { uint32_t sc_flags; uint32_t sc_gooo; struct rwlock sc_lock; }; .. uint32_t fs; rw_rlock(&sc->sc_lock); fs = sc->sc_flags; rw_runlock(&sc->sc_lock); since no writing of the members of struct foo_softc is previewed it is safe to only use an atomic instruction more than a memory barrier. Attilio -- Peace can only be achieved by understanding - A. Einstein