From owner-freebsd-arch@freebsd.org Wed Dec 23 18:08:32 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D396FA50C7A for ; Wed, 23 Dec 2015 18:08:32 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B7F6B189B for ; Wed, 23 Dec 2015 18:08:32 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id B271DA50C78; Wed, 23 Dec 2015 18:08:32 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97F87A50C76; Wed, 23 Dec 2015 18:08:32 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.netplex.net", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5728A1899; Wed, 23 Dec 2015 18:08:32 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.15.1/8.15.1/NETPLEX) with ESMTP id tBNI3x8L002901; Wed, 23 Dec 2015 13:03:59 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Wed, 23 Dec 2015 13:03:59 -0500 (EST) Date: Wed, 23 Dec 2015 13:03:59 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net Reply-To: Daniel Eischen To: Konstantin Belousov cc: threads@freebsd.org, arch@freebsd.org Subject: Re: libthr shared locks In-Reply-To: <20151223172528.GT3625@kib.kiev.ua> Message-ID: References: <20151223172528.GT3625@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Dec 2015 18:08:33 -0000 On Wed, 23 Dec 2015, Konstantin Belousov wrote: > A well-known limitation of the FreeBSD libthr implementation of the > pthread locking objects is the missed support for the process-shared > locks. The hardest part of implementing the support is the neccessity > of providing ABI compatibility for the current implementation. > > Right now, the ABI-visible handle to the locks is a single pointer > word. As an example which allows to make the description less vague, > let consider pthread_mutex_t. It is defined in > sys/sys/_pthreadtypes.h as > typedef struct pthread_mutex *pthread_mutex_t; > The pointer points to the following structure, after the > pthread_mutex_init(3) is called > struct pthread_mutex { > struct umutex m_lock; > int m_flags; > struct pthread *m_owner; > int m_count; > int m_spinloops; > int m_yieldloops; > TAILQ_ENTRY(pthread_mutex) m_qe; > }; > struct umutex { > volatile __lwpid_t m_owner; /* Owner of the mutex */ > __uint32_t m_flags; /* Flags of the mutex */ > __uint32_t m_ceilings[2]; /* Priority protect ceiling */ > __uint32_t m_spare[4]; > }; > > Would the ABI modified to make the pthread_mutex_t large enough to > hold struct pthread_mutex, the rest of the implementation of the > shared mutex is relatively trivial, if not already done. > > Changing this ABI is very hard. libthr provides the symbol > versioning, which allows to provide compatible shims for the previous > ABI variant. But since userspace tends to use the pthread objects in > the layouts of the library objects, this causes serious ABI issues > when mixing libraries built against different default versions of > libthr. > > My idea to provide the shared locks, while not changing the ABI for > libthr, is to use marker pointers to indicate the shared objects. The > real struct pthread_mutex, which carries the locking information, is > allocated by at the off-page from some anonymous posix shared memory > object. I'd rather just change the pthread_foo lock types to include the space. I really don't like the way libc, rtld, etc have to jump through hoops to initialize the locks. If the lock types included the storage, then they wouldn't have to. My idea was to keep the first storage unit of the lock as a pointer/self-reference, so that it is easy to keep the ABI. You can set the 0 bit in the pointer to indicate whether the lock was the old ABI, then just clear it before using it. And we could hide old ABI implementation compilation behind WITH_FOO to avoid overhead of checking (ptr & 0x1) != 0. I think this is similar to what you've done below. > > The marker is defined as > #define THR_PSHARED_PTR \ > ((void *)(uintptr_t)((1ULL << (NBBY * sizeof(long) - 1)) | 1)) > The bit-pattern is 1000....00001. There are two tricks used: [ ... ] I also don't like to get the kernel involved if it isn't necessary. -- DE