From owner-freebsd-arch@freebsd.org Fri May 6 13:00:53 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F28F8B2F78C for ; Fri, 6 May 2016 13:00:53 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E0C331F1B for ; Fri, 6 May 2016 13:00:53 +0000 (UTC) (envelope-from martin@lispworks.com) Received: by mailman.ysv.freebsd.org (Postfix) id E0087B2F78A; Fri, 6 May 2016 13:00:53 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF885B2F789; Fri, 6 May 2016 13:00:53 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [46.17.166.21]) by mx1.freebsd.org (Postfix) with ESMTP id 8AE7A1F18; Fri, 6 May 2016 13:00:52 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.14.9/8.14.9) with ESMTP id u46D0mQI006640; Fri, 6 May 2016 14:00:48 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id u46D0mux008057; Fri, 6 May 2016 14:00:48 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id u46D0mhN008053; Fri, 6 May 2016 14:00:48 +0100 Date: Fri, 6 May 2016 14:00:48 +0100 Message-Id: <201605061300.u46D0mhN008053@higson.cam.lispworks.com> From: Martin Simmons To: Konstantin Belousov CC: threads@freebsd.org, arch@freebsd.org In-reply-to: <20160505185810.GF2422@kib.kiev.ua> (message from Konstantin Belousov on Thu, 5 May 2016 21:58:10 +0300) Subject: Re: Robust mutexes implementation References: <20160505131029.GE2422@kib.kiev.ua> <201605051720.u45HKZ76021094@higson.cam.lispworks.com> <20160505185810.GF2422@kib.kiev.ua> X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 May 2016 13:00:54 -0000 >>>>> On Thu, 5 May 2016 21:58:10 +0300, Konstantin Belousov said: > > Yes. In fact I wrote the __containerof stuff only later, initial > version of the patch did relied on the fact that m_lock is at the > beginning of pthread_mutex. > > I rewrote the code in enqueue, and there is one more similar check in > dequeue_mutex(). > > Updated patch is at https://kib.kiev.ua/kib/pshared/robust.2.patch . OK. > On Thu, May 05, 2016 at 06:20:35PM +0100, Martin Simmons wrote: > > Also, is it safe to assume memory ordering between the assignments of > > m->m_lock.m_rb_lnk and curthread->robust_list? Maybe it is OK because the > > kernel will never read curthread->robust_list until after the CPU executing > > enqueue_mutex has passed a memory barrier? > > Inter-CPU ordering (I suppose you mean this, because you mention > barriers) only matter when we consider multi-threaded interaction. > In case of dequeue_mutex, paired to corresponding enqueue_mutex(), > the calls occur in the same thread, and the thread is always > self-consistent. Agreed. > WRT possible reordering, we in fact only care that enqueue writes do > not become visible before lock is obtained, and dequeue must finish > for external observers before lock is released. This is ensured by > the umutex lock semantic, which has neccessary acquire barrier on > lock and release barrier on unlock. I meant the case where CPU 1 claims the lock, executing this from enqueue_mutex: m->m_lock.m_rb_lnk = 0; /* A */ *rl = (uintptr_t)&m->m_lock; /* B */ and then the thread dies and CPU 2 cleans up the dead thread executing this from umtx_handle_rb: error = copyin((void *)rbp, &m, sizeof(m)); /* C */ *rb_list = m.m_rb_lnk; /* D */ where rbp is the value stored into *rl at B by CPU 1. What ensures that CPU 1's store at A is seen by CPU 2 when it reads the value at D? I'm hoping this is covered by something that I don't understand, but I was worried by the comment "consistent even in the case of thread termination at arbitrary moment" above enqueue_mutex. __Martin