From owner-freebsd-threads@FreeBSD.ORG Mon Apr 23 12:33:06 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06A5D106566C for ; Mon, 23 Apr 2012 12:33:06 +0000 (UTC) (envelope-from yfw.bsd@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id B88148FC0A for ; Mon, 23 Apr 2012 12:33:05 +0000 (UTC) Received: by obqv19 with SMTP id v19so17949998obq.13 for ; Mon, 23 Apr 2012 05:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=WyIUeO1Iuod7ydcEW+/SwN772QmpofuuK9vC1/ydkvI=; b=VkGBsxCUsZC5x49QNzsvfFho95RbhQZueZ6DMWqlMkJwZidbxC7d+Ygs5U9lJdT9lT 0+qVWnmc+NCLF15QvznZOkagnlWsXYZKhCNOj1Gi+yBgSFDUgImFbQhFL1CW29nWslSH timlayWnzOiE9keJoPL+i5Va1WHFMIJ5AclFMOp7vGC3+f7aCCuP+YrvfYcz2u4QP+bP zX2BcIO6+y6cK6JBHYK6QYaC/s8KIRT0z5TVD4t78+Yx68/rQ+AqJrlanqnrDSU7HehE 4/26a5kgj9T9aWoVl3+I89prvptEUt1HDCNuMYSkYfzU4LeYKLwnjvn9KVHp6yuC2od/ ocoQ== MIME-Version: 1.0 Received: by 10.182.232.38 with SMTP id tl6mr7322531obc.16.1335184385378; Mon, 23 Apr 2012 05:33:05 -0700 (PDT) Received: by 10.60.125.135 with HTTP; Mon, 23 Apr 2012 05:33:05 -0700 (PDT) In-Reply-To: <20120423120720.GS2358@deviant.kiev.zoral.com.ua> References: <20120423084120.GD76983@zxy.spb.ru> <20120423094043.GS32749@zxy.spb.ru> <20120423113838.GT32749@zxy.spb.ru> <20120423120720.GS2358@deviant.kiev.zoral.com.ua> Date: Mon, 23 Apr 2012 20:33:05 +0800 Message-ID: From: Fengwei yin To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-threads@freebsd.org, jack.ren@intel.com Subject: Re: About the memory barrier in BSD libc X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2012 12:33:06 -0000 On Mon, Apr 23, 2012 at 8:07 PM, Konstantin Belousov wrote: > On Mon, Apr 23, 2012 at 07:44:34PM +0800, Fengwei yin wrote: >> On Mon, Apr 23, 2012 at 7:38 PM, Slawa Olhovchenkov wro= te: >> > On Mon, Apr 23, 2012 at 07:26:54PM +0800, Fengwei yin wrote: >> > >> >> On Mon, Apr 23, 2012 at 5:40 PM, Slawa Olhovchenkov = wrote: >> >> > On Mon, Apr 23, 2012 at 05:32:24PM +0800, Fengwei yin wrote: >> >> > >> >> >> On Mon, Apr 23, 2012 at 4:41 PM, Slawa Olhovchenkov wrote: >> >> >> > On Mon, Apr 23, 2012 at 02:56:03PM +0800, Fengwei yin wrote: >> >> >> > >> >> >> >> Hi list, >> >> >> >> If this is not correct question on the list, please let me know= and >> >> >> >> sorry for noise. >> >> >> >> >> >> >> >> I have a question regarding the BSD libc for SMP arch. I didn't= see >> >> >> >> memory barrier used in libc. >> >> >> >> How can we make sure it's safe on SMP arch? >> >> >> > >> >> >> > /usr/include/machine/atomic.h: >> >> >> > >> >> >> > #define mb() =A0 =A0__asm __volatile("lock; addl $0,(%%esp)" : := : "memory") >> >> >> > #define wmb() =A0 __asm __volatile("lock; addl $0,(%%esp)" : : := "memory") >> >> >> > #define rmb() =A0 __asm __volatile("lock; addl $0,(%%esp)" : : := "memory") >> >> >> > >> >> >> >> >> >> Thanks for the information. But it looks no body use it in libc. >> >> > >> >> > I think no body in libc need memory barrier: libc don't work with >> >> > peripheral, for atomic opertions used different macros. >> >> >> >> If we check the usage of __sinit(), it is a typical singleton pattern= which >> >> needs memory barrier to make sure no potential SMP issue. >> >> >> >> Or did I miss something here? >> > >> > What architecture with cache incoherency and FreeBSD support? >> >> I suppose it's not related with cache inchoherency (I could be wrong). >> It's related >> with reorder of instruction by CPU. >> >> Here is the link talking about why need memory barrier for singleton: >> http://www.oaklib.org/docs/oak/singleton.html >> >> x86 has strict memory model and may not suffer this kind of issue. But >> ARM need to >> take care of it IMHO. > > Please note that __sinit is idempotent, so double-initialization is not > an issue there. The only possible problematic case would be other thread > executing exit and not noticing non-NULL value for __cleanup while curren= t > thread just set it. > > I am not sure how much real this race is. Each call to _sinit() is immedi= ately > followed by a lock acquire, typically FLOCKFILE(), which enforces full ba= rrier > semantic due to pthread_mutex_lock call. The exit() performs __cxa_finali= ze() > call before checking __cleanup value, and __cxa_finalize() itself locks > atexit_mutex. So the race is tiny and probably possible only for somewhat > buggy applications which perform exit() while there are stdio operations > in progress. > > Also note that some functions assign to __cleanup unconditionally. > > Do you see any real issue due to non-synchronized access to __cleanup ? No. I didn't see real issue. I am just reviewing the code. If you don't think __sinit has issue, let's check another code: line 68 in libc/stdio/fclose.c line 133 in libc/stdio/findfp.c (function __sfp()) Which is trying to free a fp slot by assign 0 to fp->_flags. But if the instrucation could be re-ordered, another CPU could see fp->_flags is assigned to 0 before the cleanup from line 57 to 67. Let's say, if another CPU is in line 133 of __sfp(), it could see fp->_flags become 0 before it's aware of the cleanup (Line 57 to line 67 in libc/stdio/fclose.c) happen. Note: the mutex of FUNLOCKFILE(fp) in line 69 of libc/stdio/fclose.c just could make sure line 70 happen after line 68. It can't impact the re-order of line 57 ~ line 68 by CPU.