From owner-freebsd-arch@FreeBSD.ORG Mon Aug 23 22:57:53 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6DF01065693; Mon, 23 Aug 2010 22:57:53 +0000 (UTC) (envelope-from max@laiers.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.17.8]) by mx1.freebsd.org (Postfix) with ESMTP id 4E6668FC1D; Mon, 23 Aug 2010 22:57:53 +0000 (UTC) Received: from f8x64.laiers.local (dslb-088-066-049-083.pools.arcor-ip.net [88.66.49.83]) by mrelayeu.kundenserver.de (node=mrbap1) with ESMTP (Nemesis) id 0Mgpk0-1ORaVI3EsS-00MQXi; Tue, 24 Aug 2010 00:45:17 +0200 From: Max Laier Organization: FreeBSD To: freebsd-arch@freebsd.org Date: Tue, 24 Aug 2010 00:45:15 +0200 User-Agent: KMail/1.13.5 (FreeBSD/8.1-RELEASE; KDE/4.4.5; amd64; ; ) References: <201008160515.21412.max@love2party.net> <4C7042BA.8000402@freebsd.org> <201008222105.47276.max@love2party.net> In-Reply-To: <201008222105.47276.max@love2party.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201008240045.15998.max@laiers.net> X-Provags-ID: V02:K0:Zg08zW314ugbuSAVqnWD3R1zBlQ41F1tCbKuxtn9A5k yU2CHRFFnha/vLZVaJJe5rrwS8t1X6O+Fkj6/RxECEB0CR65kY EDuasCN2KSlOP8mxJR4mMUuooaZFrzZI7T0+tBGj0Ae8qxAAXx 6CbBDPmQUmOAU/wluVIgC0D3ZUgvulLpD/O2tKtokOF8vDU/32 ub4MKdTcOirYEnQgsZ9DQ== Cc: Stephan Uphoff Subject: Re: rmlock(9) two additions X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Aug 2010 22:57:53 -0000 On Sunday 22 August 2010 21:05:47 Max Laier wrote: > On Saturday 21 August 2010 23:18:50 Stephan Uphoff wrote: > > Max Laier wrote: > > ... > > The rm_try_rlock() will never succeed when the lock was last locked as > > writer. > > Maybe add: > > > > void > > _rm_wunlock(struct rmlock *rm) > > { > > + rm->rm_noreadtoken = 0; > > > > mtx_unlock(&rm->rm_lock); > > > > } > > > > But then > > > > _rm_wlock(struct rmlock *rm) > > > > always needs to use IPIs - even when the lock was used last as a write > > lock. > > I don't think this is a big problem - I can't see many use cases for > rmlocks where you'd routinely see repeated wlocks without rlocks between > them. However, I think there should be a release memory barrier > before/while clearing rm_noreadtoken, otherwise readers may not see the > data writes that are supposed to be protected by the lock?!? > > ... > I believe either solution will work. #1 is a bit more in the spirit of the > rmlock - i.e. make the read case cheap and the write case expensive. I'm > just not sure about the lock semantics. > > I guess a > > atomic_store_rel_int(&rm->rm_noreadtoken, 0); > > should work. thinking about this for a while makes me wonder: Are readers really guaranteed to see all the updates of a writer - even in the current version? Example: writer thread: rm_wlock(); // lock mtx, IPI, wait for reader drain modify_data(); rm_wunlock(); // unlock mtx (this does a atomic_*_rel) reader thread #1: // failed to get the lock, spinning/waiting on mtx mtx_lock(); // this does a atomic_*_acq -> this CPU sees the new data rm->rm_noreadtoken = 0; // now new readers can enter quickly ... reader thread 2# (on a different CPU than reader #1): // enters rm_rlock() "after" rm_noreadtoken was reset -> no memory barrier // does this thread see the modifications? I realize this is a somewhat pathological case, but would it be possible in theory? Or is the compiler_memory_barrier() actually enough? Otherwise, I think we need an IPI on rm_wunlock() that does a atomic_*_acq on every CPU. Thoughts? Max