From owner-freebsd-threads@FreeBSD.ORG Wed Apr 25 14:15:30 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCDDE1065675 for ; Wed, 25 Apr 2012 14:15:29 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id B00D68FC1B for ; Wed, 25 Apr 2012 14:15:29 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 0F7B8B915; Wed, 25 Apr 2012 10:15:29 -0400 (EDT) From: John Baldwin To: freebsd-threads@freebsd.org Date: Wed, 25 Apr 2012 10:12:36 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p13; KDE/4.5.5; amd64; ; ) References: <20120423084120.GD76983@zxy.spb.ru> <20120425062627.GI2358@deviant.kiev.zoral.com.ua> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201204251012.36754.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 25 Apr 2012 10:15:29 -0400 (EDT) Cc: jack.ren@intel.com Subject: Re: About the memory barrier in BSD libc X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2012 14:15:30 -0000 On Wednesday, April 25, 2012 3:05:54 am Fengwei yin wrote: > On Wed, Apr 25, 2012 at 2:26 PM, Konstantin Belousov > wrote: > > On Wed, Apr 25, 2012 at 11:25:40AM +0800, Fengwei yin wrote: > >> On Wed, Apr 25, 2012 at 2:13 AM, Konstantin Belousov > >> wrote: > >> > On Tue, Apr 24, 2012 at 07:00:33PM +0100, Martin Simmons wrote: > >> >> >>>>> On Tue, 24 Apr 2012 19:58:42 +0300, Konstantin Belousov said: > >> >> > > >> >> > + > >> >> > + /* > >> >> > + * Lock the spinlock used to protect __sglue list walk in > >> >> > + * __sfp(). The __sfp() uses fp->_flags == 0 test as an > >> >> > + * indication of the unused FILE. > >> >> > + * > >> >> > + * Taking the lock prevents possible compiler or processor > >> >> > + * reordering of the writes performed before the final _flags > >> >> > + * cleanup, making sure that we are done with the FILE before > >> >> > + * it is considered available. > >> >> > + */ > >> >> > + STDIO_THREAD_LOCK(); > >> >> > fp->_flags = 0; /* Release this FILE for reuse. */ > >> >> > + STDIO_THREAD_UNLOCK(); > >> >> > FUNLOCKFILE(fp); > >> >> > return (r); > >> >> > >> >> Is that assumption about reordering correct? I.e. is FreeBSD's spinlock a > >> >> two-way barrier? > >> >> > >> >> It isn't unusual for the locking primitive to be a one-way barrier, i.e. (from > >> >> Linux kernel memory-barriers.txt) "Memory operations that occur before a LOCK > >> >> operation may appear to happen after it completes." See also acq and rel in > >> >> atomic(9). > >> > Taking the lock prevents the __sfp from iterating the list until the > >> > spinlock is released. Since release makes sure that all previous memory > >> > operations become visible, the acquire in the spinlock lock provides > >> > the neccesary guarentee. > >> > >> IMHO, the lock to me is too heavy here. What about this patch? > >> > >> NOTE: patch just show thoughts. I didn't even check build checking. > > Yes, it might be correct. But FreeBSD does prefer the acq/rel barriers > > over the rmb/wmb. > > > > There is no stand alone acq/rel APIs (Sorry, as new guy to FreeBSD, > don't know too much APIs). They are bound to atomic operations. > And yes, atomic_acq/rel should work also. Yes, you would want to use atomic_store_rel() or some such. Often doing so is much clearer than stand-alone membar's as it indicates which write has special ordering. > > Also, the lock is not that heavy right there, and the committed patch > > provides mostly zero overhead for non-threaded case. > > But lock could introduced contention in SMP case which could be avoid > with rmb/wmb. Seriously, if you are using stdio, you've already given up on performance enough to not care about contention for fclose() vs fopen(). -- John Baldwin