From owner-freebsd-arch@freebsd.org Sat Dec 26 10:54:19 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC0D2A52959; Sat, 26 Dec 2015 10:54:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3CFEB17CF; Sat, 26 Dec 2015 10:54:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id tBQAs9dI042457 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 26 Dec 2015 12:54:09 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua tBQAs9dI042457 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id tBQAs93R042456; Sat, 26 Dec 2015 12:54:09 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 26 Dec 2015 12:54:09 +0200 From: Konstantin Belousov To: Daniel Eischen Cc: John Baldwin , freebsd-arch@freebsd.org, freebsd-threads@freebsd.org Subject: Re: libthr shared locks Message-ID: <20151226105409.GH3625@kib.kiev.ua> References: <20151223172528.GT3625@kib.kiev.ua> <4199356.DlQeWDh27F@ralph.baldwin.cx> <5496837.TbTQtANDNj@ralph.baldwin.cx> <20151224191408.GA3625@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Dec 2015 10:54:19 -0000 On Fri, Dec 25, 2015 at 01:18:14PM -0500, Daniel Eischen wrote: > On Thu, 24 Dec 2015, Konstantin Belousov wrote: > > > On Thu, Dec 24, 2015 at 01:46:25PM -0500, Daniel Eischen wrote: > >> Can we still implement things like robust and priority mutexes in a > >> pshared mutex with Konstantin's implementation? The rest of the world > >> (Linux, Solaris anyway) use structs and get by just fine ABI-wise. > > > > Yes, we do. I intend to do this as the next stage, planned the > > implementation and did some preparations for it in the current patch. > > This raises the question, why do you suppose that my approach with > > the off-page would not allow it ? I am asking not to argument, but to > > understand a possible shortcoming in the approach, which I missed. > > No, I was merely inquiring, if this is on your agenda, that's > good. > > > From the implementation PoV, off-page is completely equivalent to the > > in-line structs approach, and the difference is only in the initial > > translation of say pthread_mutex_t to the structure (off-page lookup vs. > > identity). The issues with maintaining the lists of the owned robust of > > pi mutexes are same, and in fact you could see some non-trivial traces > > of this in the _mutex_fork() reimplementation in my patch. > > > > Linux and Solaris get (large enough) structs early enough to avoid the ABI > > issues. I remember Linux changing FILE layout in early days of glibc 2.0 > > and resulting pain, while they already had the GNU symbol versioning > > working and used. > > I guess the issue now is the apparent speedups without having the > extra de-reference. 8-10% for things like mysql might be worth > considering the userland structs. I am sorry for the delay in answering, I read the David' patch you pointed out, and it took time to get through 9KLOCs patch which is not applicable to the current sources. I should note that the patch is not immediately useable, not due to the source code drift, but because the patch combines many unrelated things, and I do not agree with everything in it. E.g., I found quite interesting the notion that our libthr fork() does not work when called from the signal handler. I do not quite understand how the detection of the situation is done in the patch, and I do not agree with the cleanup of the 'in sighandler' state on longjmp. I think that this part of the patch is obsoleted by libthr intercepting signals and postponing signal delivery until a critical section is exited. Patch adds yet another private malloc(), now to libthr. I think that cooperating with rtld and share the same allocator would be more reasonable. Notes above are to explain why I think that productizing the David' patch is huge work, IMO significantly more than just 'test and commit'. Returning to the point of potential gain in performance due to the ABI change. As I already stated in my previous replies, I am quite worried about the impact of the ABI change, and I think that potential destabilizations, which would manifest itself in the subtle and hard to diagnose ways (i.e. it is not a sigsegv outright on the start) is huge setback for the change. OTOH, we should be able to plan such change, which requires much more drastic measures to be implementable. I started thinking about it, and I noted that what is needed is solved by renaming libthr to something else, e.g. libthr2. One of the issue which is not solved by the dso version bump (not to even mention versioned symbols bump) is the static linker bringing both libthr.so.3 and libthr.so.4 into the same process, by the mere use of -lpthread by different objects at different times. Then, libthr and hypothetical libthr2 should check and prevent activation of image which loads both. But these are longtime to evaluate and implement. The feature at hand does not require ABI change, as my patch demostrated. Yes, shared mutexes would be more naturally implemented with the inline locks, and avoidance of an indirection in the libthr is also good, but lets not mix significant ABI change and some less ambitious feature. My approach does not lock out future strategies.