From owner-freebsd-threads@freebsd.org Sun May 8 12:52:26 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 438E2B330FD for ; Sun, 8 May 2016 12:52:26 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 31C1B13B2 for ; Sun, 8 May 2016 12:52:26 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 2DADBB330FB; Sun, 8 May 2016 12:52:26 +0000 (UTC) Delivered-To: threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CE84B330F9; Sun, 8 May 2016 12:52:26 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B1F0213AE; Sun, 8 May 2016 12:52:25 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id 1FB7EB805C; Sun, 8 May 2016 14:52:22 +0200 (CEST) Received: by toad2.stack.nl (Postfix, from userid 1677) id E4E21892B2; Sun, 8 May 2016 14:52:22 +0200 (CEST) Date: Sun, 8 May 2016 14:52:22 +0200 From: Jilles Tjoelker To: Konstantin Belousov Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160508125222.GA48862@stack.nl> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160507165956.GC89104@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 May 2016 12:52:26 -0000 On Sat, May 07, 2016 at 07:59:56PM +0300, Konstantin Belousov wrote: > On Sat, May 07, 2016 at 01:30:11AM +0200, Jilles Tjoelker wrote: > > > 4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls > > > the lifetime of the shared mutex associated with a vnode' page. > > > Apparently, there is real code around which expects the following > > > to work: > > > - mmap a file, create a shared mutex in the mapping; > > > - the process exits; > > > - another process starts, mmaps the same file and expects that the > > > previously initialized mutex is still usable. > > > The knob changes the lifetime of such shared off-page from the > > > 'destroy on last unmap' to either 'until vnode is reclaimed' or > > > until 'pthread_mutex_destroy' called, whatever comes first. > > The 'until vnode is reclaimed' bit sounds like a recipe for hard to > > reproduce bugs. > > I do think it is related to the robust mutex patch, though. > > Without robust mutexes and assuming threads do not unmap the memory > > while having a mutex locked or while waiting on a condition variable, it > > is sufficient to create the off-page mutex/condvar automatically in its > > initial state when pthread_mutex_lock() or pthread_cond_*wait() are > > called and no off-page object exists. > > With robust mutexes, we need to store somewhere whether the next thread > > should receive [EOWNERDEAD] or not, and this should persist even if no > > processes have the memory mapped. This might be done by replacing > > THR_PSHARED_PTR with a different value in the pthread_mutex_t. I'm not > > sure I like that memory write being done from the kernel though. > In principle, I agree with this. Note that if we go with something > like THR_OWNERDEAD_PTR, the kernel write to set the value would be not > much different from the kernel write to unlock robust mutex with inlined > lock structures. > Still, I would prefer to to implement this now. For the local purposes, > the knob was enough, and default value will be 'disabled'. OK. The patch still initializes umtx_shm_vnobj_persistent to 1 though. There is also a leak if umtx_shm_vnobj_persistent is toggled from 1 to 0 while an unmapped object with an off-page is active. I forgot two issues in my analysis. Firstly, the automatic re-initialization (from r297185, in fact) allows the next thread to lock the mutex even if the previous thread terminated while holding the lock. Normally, one would expect that non-robust mutexes cause the lock attempt to hang (or fail if it's a trylock), except if a thread ID is reused (in which case the lock attempt may instead succeed recursively or fail). Secondly, the initialization attributes are lost. If the next thread after last unmap sees THR_PSHARED_PTR, it has no idea whether it should be robust or not, which type it should have and whether it should be PP/PI. A mutex that was originally initialized as non-robust should not be re-initialized as robust since the code will not handle the [EOWNERDEAD] and [ENOTRECOVERABLE] errors (in the worst case, errors are completely ignored and it is as if the mutex did not exist). Special-casing robust to store in the THR_PSHARED_PTR location seems strange since the other attributes are still lost. All this is POSIX-compliant since POSIX specifies that the state of synchronization objects becomes undefined on last unmap, and our implementation fundamentally depends on that possibility. Unfortunately, Linux and Solaris do not need the possibility. The automatic re-initialization and umtx_vnode_persistent are just hacks that make certain applications almost always work (but not always, and in such cases it will be hard to debug). Another issue with umtx_vnode_persistent is that it can hide high memory usage. Filling up a page with pthread_mutex_t will create many pages full of actual mutexes. This memory usage is only visible as long as it is still mapped somewhere. Apart from that, umtx_vnode_persistent can (at least conceptually) work fully reliably for shared memory objects and tmpfs files, which do not have persistent storage. > > The below is a review of https://kib.kiev.ua/kib/pshared/robust.2.patch > > > + if ((error2 == 0 || error2 == EOWNERDEAD) && cancel) > > > _thr_testcancel(curthread); > > I don't think [EOWNERDEAD] should be swept under the carpet like this. > > The cancellation cleanup handler will use the protected state and unlock > > the mutex without making the state consistent and the state will be > > unrecoverable. > So your argument there is to return EOWNERDEAD and not cancelling, > am I right ? I reused part of your text as the comment. Yes. > > > +void > > > +_mutex_leave_robust(struct pthread *curthread, struct pthread_mutex *m) > > > +{ > > > + > > > + if (!is_robust_mutex(m)) > > > + return; > > This accesses the mutex after writing a value to the lock > > word which allows other threads to lock it. A use after free may result, > > since it is valid for another thread to lock, unlock and destroy the > > mutex (assuming the mutex is not used otherwise later). > > Memory ordering may permit the load of m->m_lock.m_flags to be moved > > before the actual unlock, so this issue may not actually appear. > > Given that the overhead of a system call on every robust mutex unlock is > > not desired, the kernel's unlock of a terminated thread's mutexes will > > unavoidably have this use after free. However, for non-robust mutexes > > the previous guarantees should be kept. > I agree that this is a bug, and agree that the kernel accesses to the > curthread->inact_mtx are potentially unsafe. I also did not wanted to > issue a syscall for unlock of a robust mutex, as you noted. > I fixed the bug with the is_robust_mutex() test in _mutex_leave_robust() > by caching the robust status. > I was indeed worried by the kernel access issue you mentioned, but > kernel is immune to 'bad' memory accesses. What bothered me is the ill > ABA situation, where the lock memory is freed and repurposed, and then > the lock word is written with the one of two specific values which give > the same state as for locked mutex. This would cause kernel to 'unlock' > it (but not to follow the invalid m_rb_link). > But for this to happen, we must have a situation where a thread > is being terminated before mutex_unlock_common() reached the > _mutex_leave_robust() call. This is async thread termination, which > then must be either process termination (including execve()), or a call > to thr_exit() from a signal handler in our thread (including async > cancellation). > I am sure that the thr_exit() situation is non-conforming, so the only > concern is the process exit, and then, shared robust mutex, because for > private mutex, only the exiting process state is affected. I can verify > in umtx_handle_rb(), for instance, that for USYNC_PROCESS_SHARED object, > the underlying memory is backed by the umtx shm page. This would close > the race. > But this would interfere with libthr2, if that ever happen. Hmm, libthr2 or non-standard synchronization primitive implementations seem a good reason to not check for umtx shm page. However, the existing checks can be made stricter. The umtx_handle_rb() from robust.3.patch will use m_rb_lnk with no validation at all except that it is a valid pointer. However, if the UMUTEX_ROBUST flag is not set, the mutex should not have been in this list at all and it is probably safer to ignore m_rb_lnk. > > > @@ -37,7 +37,11 @@ struct umutex { > > > + __uintptr_t m_rb_lnk; /* Robust linkage */ > > Although Linux also stores the robust list nodes in the mutexes like > > this, I think it increases the chance of strange memory corruption. > > Making the robust list an array in the thread's memory would be more > > reliable. If the maximum number of owned robust mutexes can be small, > > this can have a fixed size; otherwise, it needs to grow as needed (which > > does add an allocation that may fail to the pthread_mutex_lock path, > > bad). > I gave this proposal some thought. > I very much dislike an idea of calling memory allocator on the lock, and > esp. the trylock, path. The later could need to obtain allocator locks > which (sometimes partially) defeat the trylock purpose. > I can use mmap(2) directly there, similarly how pthread_setspecific() > was changed recently, which would avoid the issue of calling userspace > allocator. Still, the problem of an addiitonal syscall, resulting > ENOMEM and also the time to copy the current robust owned list to grown > location are there (I do not see that using chunked allocations is > reasonable, since it would be the same list as current m_rb_lnk, but at > different level. > I prefer to keep the robust linked list for these reasons. There is a difference between chunked allocations and the current m_rb_lnk in that the list would reside in local memory, not vulnerable to other processes scribbling over it. This is probably not a major issue since sharing a mutex already allows threads to block each other indefinitely. The X server has used shared memory across privilege levels for years but not with anything like mutexes and condition variables. It uses X protocol messages and xshmfence (custom synchronization object that does not block the X server). > In fact, the deciding argument would be actual application usage of the > robustness. I thought, when writing the patch, when and how would I use > the feature, and did not see compelling arguments to even try to use it. > My stumbling block is the user data consistency recovery: for instance, > I tried to write a loop which would increment shared variable N times, > and I was not able to end up with any simple recovery mechanism from > the aborted iteration, except writing iteration in assembly and have a > parallel tick variable which enumerate each iteration action. One way of using this is to never call pthread_mutex_consistent() and only use it to report the problem and avoid hangs and crashes. A slightly extended version would tear down the whole structure including shared memory and all involved processes and rebuild it. When calling pthread_mutex_consistent(), some data loss will almost unavoidably occur. In some cases, it may be possible to re-initialize just the state protected by the mutex. Alternatively, a thread that modifies the state may make a copy of it before modifying it (some thread-local StoreStore fences are needed); the [EOWNERDEAD] handling can then put back the copy if it exists. This way, robust mutexes can be used to make updates transactional. Approaches based on message passing will be easier to get right and more reliable but they might be too slow. > Updated patch is at https://kib.kiev.ua/kib/pshared/robust.3.patch > I did not added the check for umtx shm into the umtx_handle_rb() yet, > waiting for your opinion. -- Jilles Tjoelker From owner-freebsd-threads@freebsd.org Sun May 8 14:32:53 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3889BB32112 for ; Sun, 8 May 2016 14:32:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D82A1893 for ; Sun, 8 May 2016 14:32:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u48EWpc1009002 for ; Sun, 8 May 2016 14:32:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Sun, 08 May 2016 14:32:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rblayzor@inoc.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 May 2016 14:32:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #108 from Robert Blayzor --- It's been longer than average now and I have not run into the processes terminating abnormally with the last patch installed against 10.3. HOWEVER,= I have noticed a new issue with the network stack that seems to be happening = at roughly the same interval. I'm not sure if the two are related or if fixing= one problem manifested into another. Basically now we're getting the severs filling up with TCP connections stuc= k in a "CLOSED" state. We'll end up getting thousands of them until connections = to the processes just time out. Sometimes we'll see kernel messages: sonewconn: pcb 0xfffff80001b36930: Listen queue overflow: 767 already in qu= eue awaiting acceptance (46 occurrences) sonewconn: pcb 0xfffff80001b36930: Listen queue overflow: 767 already in qu= eue awaiting acceptance (46 occurrences) sonewconn: pcb 0xfffff80001b36930: Listen queue overflow: 767 already in qu= eue awaiting acceptance (50 occurrences) sonewconn: pcb 0xfffff80001b36930: Listen queue overflow: 767 already in qu= eue awaiting acceptance (42 occurrences) sonewconn: pcb 0xfffff80001b36930: Listen queue overflow: 767 already in qu= eue awaiting acceptance (44 occurrences) sonewconn: pcb 0xfffff80001b36930: Listen queue overflow: 767 already in qu= eue awaiting acceptance (38 occurrences) sonewconn: pcb 0xfffff80001b36930: Listen queue overflow: 767 already in qu= eue awaiting acceptance (40 occurrences) [...] But not always... Currently I have a server in this state and I'll see: tcp6 0 0 mta1.imap mta-slb-1.alb1.i.20511 CLOSED tcp6 0 0 mta1.pop3 mta-slb-1.alb1.i.43879 CLOSED tcp6 0 0 mta1.imap mta-slb-0.alb1.i.5259 CLOSED tcp6 0 0 mta1.pop3 mta-slb-0.alb1.i.12519 CLOSED tcp6 0 0 mta1.imap mta-slb-1.alb1.i.1316 CLOSED tcp6 0 0 mta1.pop3 mta-slb-1.alb1.i.65343 CLOSED tcp6 0 0 mta1.imap mta-slb-0.alb1.i.16289 CLOSED tcp6 0 0 mta1.pop3 mta-slb-0.alb1.i.19215 CLOSED tcp6 32 0 mta1.sieve mta-slb-0.alb1.i.19549 CLOSED tcp6 0 0 mta1.imap mta-slb-1.alb1.i.49287 CLOSED tcp6 32 0 mta1.sieve mta-slb-1.alb1.i.54187 CLOSED tcp6 0 0 mta1.pop3 mta-slb-1.alb1.i.39767 CLOSED tcp6 0 0 mta1.imap mta-slb-0.alb1.i.54366 CLOSED tcp6 0 0 mta1.pop3 mta-slb-0.alb1.i.47579 CLOSED tcp6 0 0 mta1.imap mta-slb-1.alb1.i.48798 CLOSED tcp6 0 0 mta1.pop3 mta-slb-1.alb1.i.40190 CLOSED ... [ 1000's of lines truncated ] It's not just Dovecot either, we also will see several stuck in CLOSED from Exim as well. So it doesn't look like an application issue, in fact, sockst= at shows these stuck sockets not related to the process anymore... ie: ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:1:56602 ? ? ? ? tcp6 2607:f058:110:2::1:1:4190 2607:f058:110:2::f:0:32558 ? ? ? ? tcp6 2607:f058:110:2::1:1:110 2607:f058:110:2::f:1:53931 ? ? ? ? tcp6 2607:f058:110:2::1:1:110 2607:f058:110:2::f:0:58671 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:1:58788 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:0:30523 ? ? ? ? tcp6 2607:f058:110:2::1:1:110 2607:f058:10::10:32375 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:0:46131 ? ? ? ? tcp6 2607:f058:110:2::1:1:110 2607:f058:110:2::f:1:50671 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:1:4223 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:1:15773 ? ? ? ? tcp6 2607:f058:110:2::1:1:110 2607:f058:110:2::f:0:26610 ? ? ? ? tcp6 2607:f058:110:2::1:1:4190 2607:f058:110:2::f:0:38765 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:0:42310 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:1:5643 ? ? ? ? tcp6 2607:f058:110:2::1:1:143 2607:f058:110:2::f:0:37143 ? ? ? ? tcp6 2607:f058:110:2::1:1:4190 2607:f058:110:2::f:0:24906 [...] (agan, thousands of lines truncated) netstat -ans -p tcp tcp: 8497364 packets sent 3825626 data packets (484438498 bytes) 12 data packets (5560 bytes) retransmitted 1 data packet unnecessarily retransmitted 0 resends initiated by MTU discovery 4106401 ack-only packets (0 delayed) 0 URG only packets 0 window probe packets 313890 window update packets 251435 control packets 5525333 packets received 4126773 acks (for 483181992 bytes) 108333 duplicate acks 0 acks for unsent data 3787497 packets (1012422242 bytes) received in-sequence 151 completely duplicate packets (0 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 0 out-of-order packets (0 bytes) 0 packets (0 bytes) of data after window 0 window probes 7067 window update packets 6376 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 7546 connection requests 315652 connection accepts 0 bad connection attempts 0 listen queue overflows 73753 ignored RSTs in the windows 323196 connections established (including accepts) 323132 connections closed (including 1414 drops) 121869 connections updated cached RTT on close 121869 connections updated cached RTT variance on close 0 connections updated cached ssthresh on close 2 embryonic connections dropped 4126615 segments updated rtt (of 3726607 attempts) 12 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 14 Connections (fin_wait_2) dropped because of timeout 19343 keepalive timeouts 19298 keepalive probes sent 45 connections dropped by keepalive 240267 correct ACK header predictions 792401 correct data packet header predictions 315653 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 315652 completed 0 bucket overflow 0 cache overflow 1 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 315653 cookies sent 0 cookies received 164 hostcache entries added 0 bucket overflow 0 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 0 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window 0 packets with valid tcp-md5 signature received 0 packets with invalid tcp-md5 signature received 0 packets with tcp-md5 signature mismatch 0 packets with unexpected tcp-md5 signature received 0 packets without expected tcp-md5 signature received If I attempt to kill and restart the processes, sometimes it works, sometim= es it doesn't and I have to end up rebooting the server. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Mon May 9 02:19:53 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C5E27B32B0A for ; Mon, 9 May 2016 02:19:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B6D121AB3 for ; Mon, 9 May 2016 02:19:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u492Jq1A000155 for ; Mon, 9 May 2016 02:19:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 204426] Processes terminating cannot access memory Date: Mon, 09 May 2016 02:19:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 May 2016 02:19:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204426 --- Comment #109 from Konstantin Belousov --- (In reply to Robert Blayzor from comment #108) Thank you for the testing. I am confident that your SIGSEGV issue is resol= ved. Also, I am sure that the network issue you described is unrelated. Please track it in the different bug report. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Mon May 9 02:51:14 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D963B3331E for ; Mon, 9 May 2016 02:51:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 57D191844 for ; Mon, 9 May 2016 02:51:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 5058CB3331C; Mon, 9 May 2016 02:51:14 +0000 (UTC) Delivered-To: threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FAE7B3331A; Mon, 9 May 2016 02:51:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D2C4E1840; Mon, 9 May 2016 02:51:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u492p7u1027902 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 9 May 2016 05:51:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u492p7u1027902 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u492p7Gr027895; Mon, 9 May 2016 05:51:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 9 May 2016 05:51:07 +0300 From: Konstantin Belousov To: Jilles Tjoelker Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160509025107.GN89104@kib.kiev.ua> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160508125222.GA48862@stack.nl> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 May 2016 02:51:14 -0000 On Sun, May 08, 2016 at 02:52:22PM +0200, Jilles Tjoelker wrote: > OK. The patch still initializes umtx_shm_vnobj_persistent to 1 though. > There is also a leak if umtx_shm_vnobj_persistent is toggled from 1 to 0 > while an unmapped object with an off-page is active. > > I forgot two issues in my analysis. > > Firstly, the automatic re-initialization (from r297185, in fact) allows > the next thread to lock the mutex even if the previous thread terminated > while holding the lock. Normally, one would expect that non-robust > mutexes cause the lock attempt to hang (or fail if it's a trylock), > except if a thread ID is reused (in which case the lock attempt may > instead succeed recursively or fail). > > Secondly, the initialization attributes are lost. If the next thread > after last unmap sees THR_PSHARED_PTR, it has no idea whether it should > be robust or not, which type it should have and whether it should be > PP/PI. A mutex that was originally initialized as non-robust should not > be re-initialized as robust since the code will not handle the > [EOWNERDEAD] and [ENOTRECOVERABLE] errors (in the worst case, errors are > completely ignored and it is as if the mutex did not exist). > > Special-casing robust to store in the THR_PSHARED_PTR location seems > strange since the other attributes are still lost. Yes, these are all good points. > > All this is POSIX-compliant since POSIX specifies that the state of > synchronization objects becomes undefined on last unmap, and our > implementation fundamentally depends on that possibility. Unfortunately, Could you, please, point me to the exact place in the standard where this is allowed ? > Linux and Solaris do not need the possibility. The automatic > re-initialization and umtx_vnode_persistent are just hacks that make > certain applications almost always work (but not always, and in such > cases it will be hard to debug). > > Another issue with umtx_vnode_persistent is that it can hide high memory > usage. Filling up a page with pthread_mutex_t will create many pages > full of actual mutexes. This memory usage is only visible as long as it > is still mapped somewhere. There is already a resource limit for the number of pshared locks per uid, RLIMIT_UMTXP. When exceeded, user would get somewhat obscure failure mode, but excessive memory consumption is not allowed. And I think that vmstat -o would give enough info to diagnose, except that users must know about it and be quialified enough to interpret the output. > > Apart from that, umtx_vnode_persistent can (at least conceptually) work > fully reliably for shared memory objects and tmpfs files, which do not > have persistent storage. I changed defaults for the umtx_vnode_persistent to 0 in the published patch. > Hmm, libthr2 or non-standard synchronization primitive implementations > seem a good reason to not check for umtx shm page. > > However, the existing checks can be made stricter. The umtx_handle_rb() > from robust.3.patch will use m_rb_lnk with no validation at all except > that it is a valid pointer. However, if the UMUTEX_ROBUST flag is not > set, the mutex should not have been in this list at all and it is > probably safer to ignore m_rb_lnk. Ok, I changed the code to consider lack of UMUTEX_ROBUST as a stopper for the list walk. Also, I stop the walk if mutex is not owned by the current thread, except when the mutex was stored in inact slot. The same piece of changes hopefully fixes list walk for COMPAT32 on big-endian machines. > > There is a difference between chunked allocations and the current > m_rb_lnk in that the list would reside in local memory, not vulnerable > to other processes scribbling over it. This is probably not a major > issue since sharing a mutex already allows threads to block each other > indefinitely. I would easily deletegate the chunked array to some future reimplementation if not the ABI issue. Still, I do not like it. Current updates to the patch https://kib.kiev.ua/kib/pshared/robust.4.patch From owner-freebsd-threads@freebsd.org Thu May 12 06:53:36 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C2F3B38527 for ; Thu, 12 May 2016 06:53:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5CB3514FD for ; Thu, 12 May 2016 06:53:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u4C6ra5Q084309 for ; Thu, 12 May 2016 06:53:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-threads@FreeBSD.org Subject: [Bug 209233] [patch] pthread_suspend_all_np races with check_suspend Date: Thu, 12 May 2016 06:53:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: threads X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-threads@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 06:53:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D209233 --- Comment #11 from commit-hook@freebsd.org --- A commit references this bug: Author: kib Date: Thu May 12 06:53:23 UTC 2016 New revision: 299521 URL: https://svnweb.freebsd.org/changeset/base/299521 Log: MFC r299114: Do not leak THR_FLAGS_SUSPENDED from the previous suspend/resume cycle. PR: 209233 Changes: _U stable/10/ stable/10/lib/libthr/thread/thr_resume_np.c stable/10/lib/libthr/thread/thr_sig.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-threads@freebsd.org Fri May 13 15:37:19 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC2B4B39DF3 for ; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AA56C1C7B for ; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id A33DFB39DF1; Fri, 13 May 2016 15:37:19 +0000 (UTC) Delivered-To: threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2796B39DEF; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3AD101C70; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id 0A28E358C68; Fri, 13 May 2016 17:37:16 +0200 (CEST) Received: by toad2.stack.nl (Postfix, from userid 1677) id 7228F892FF; Fri, 13 May 2016 17:37:16 +0200 (CEST) Date: Fri, 13 May 2016 17:37:16 +0200 From: Jilles Tjoelker To: Konstantin Belousov Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160513153716.GA30576@stack.nl> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160509025107.GN89104@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2016 15:37:19 -0000 On Mon, May 09, 2016 at 05:51:07AM +0300, Konstantin Belousov wrote: > On Sun, May 08, 2016 at 02:52:22PM +0200, Jilles Tjoelker wrote: > > OK. The patch still initializes umtx_shm_vnobj_persistent to 1 though. > > There is also a leak if umtx_shm_vnobj_persistent is toggled from 1 to 0 > > while an unmapped object with an off-page is active. > [snip] > > All this is POSIX-compliant since POSIX specifies that the state of > > synchronization objects becomes undefined on last unmap, and our > > implementation fundamentally depends on that possibility. Unfortunately, > Could you, please, point me to the exact place in the standard where > this is allowed ? The mmap() page in POSIX.1-2008tc1 XSH 3 has: ] The state of synchronization objects such as mutexes, semaphores, ] barriers, and conditional variables placed in shared memory mapped ] with MAP_SHARED becomes undefined when the last region in any process ] containing the synchronization object is unmapped. This is new in issue 7 (SUSv4): ] Austin Group Interpretations 1003.1-2001 #078 and #079 are applied, ] clarifying page alignment requirements and adding a note about the ] state of synchronization objects becoming undefined when a shared ] region is unmapped. > > Linux and Solaris do not need the possibility. The automatic > > re-initialization and umtx_vnode_persistent are just hacks that make > > certain applications almost always work (but not always, and in such > > cases it will be hard to debug). > > Another issue with umtx_vnode_persistent is that it can hide high memory > > usage. Filling up a page with pthread_mutex_t will create many pages > > full of actual mutexes. This memory usage is only visible as long as it > > is still mapped somewhere. > There is already a resource limit for the number of pshared locks per > uid, RLIMIT_UMTXP. When exceeded, user would get somewhat obscure > failure mode, but excessive memory consumption is not allowed. And I > think that vmstat -o would give enough info to diagnose, except that > users must know about it and be quialified enough to interpret the > output. Hmm, OK. > > Apart from that, umtx_vnode_persistent can (at least conceptually) work > > fully reliably for shared memory objects and tmpfs files, which do not > > have persistent storage. > I changed defaults for the umtx_vnode_persistent to 0 in the published > patch. OK. > > Hmm, libthr2 or non-standard synchronization primitive implementations > > seem a good reason to not check for umtx shm page. > > However, the existing checks can be made stricter. The umtx_handle_rb() > > from robust.3.patch will use m_rb_lnk with no validation at all except > > that it is a valid pointer. However, if the UMUTEX_ROBUST flag is not > > set, the mutex should not have been in this list at all and it is > > probably safer to ignore m_rb_lnk. > Ok, I changed the code to consider lack of UMUTEX_ROBUST as a stopper > for the list walk. Also, I stop the walk if mutex is not owned by > the current thread, except when the mutex was stored in inact slot. > The same piece of changes hopefully fixes list walk for COMPAT32 on > big-endian machines. OK. > > There is a difference between chunked allocations and the current > > m_rb_lnk in that the list would reside in local memory, not vulnerable > > to other processes scribbling over it. This is probably not a major > > issue since sharing a mutex already allows threads to block each other > > indefinitely. > I would easily deletegate the chunked array to some future reimplementation > if not the ABI issue. Still, I do not like it. An array only works well for this if you know beforehand how long it needs to be, and I don't think we can do this since Linux's limit is so high that an array would waste a lot of memory. The existence of some limit is, however, unavoidable and it could be considered a bug that pthread_mutex_lock() for a robust mutex returns success even if it will not fulfill its promise to do the EOWNERDEAD thing. > Current updates to the patch https://kib.kiev.ua/kib/pshared/robust.4.patch -- Jilles Tjoelker From owner-freebsd-threads@freebsd.org Fri May 13 20:19:23 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02F25B3AF5A for ; Fri, 13 May 2016 20:19:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E0E7D1B52 for ; Fri, 13 May 2016 20:19:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id D94F0B3AF57; Fri, 13 May 2016 20:19:22 +0000 (UTC) Delivered-To: threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8BC4B3AF55; Fri, 13 May 2016 20:19:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A7131B4B; Fri, 13 May 2016 20:19:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u4DKJGNV052208 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 13 May 2016 23:19:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u4DKJGNV052208 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u4DKJGQj052207; Fri, 13 May 2016 23:19:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 13 May 2016 23:19:16 +0300 From: Konstantin Belousov To: Jilles Tjoelker Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160513201916.GF89104@kib.kiev.ua> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> <20160513153716.GA30576@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160513153716.GA30576@stack.nl> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2016 20:19:23 -0000 On Fri, May 13, 2016 at 05:37:16PM +0200, Jilles Tjoelker wrote: > On Mon, May 09, 2016 at 05:51:07AM +0300, Konstantin Belousov wrote: > The mmap() page in POSIX.1-2008tc1 XSH 3 has: > > ] The state of synchronization objects such as mutexes, semaphores, > ] barriers, and conditional variables placed in shared memory mapped > ] with MAP_SHARED becomes undefined when the last region in any process > ] containing the synchronization object is unmapped. > > This is new in issue 7 (SUSv4): > > ] Austin Group Interpretations 1003.1-2001 #078 and #079 are applied, > ] clarifying page alignment requirements and adding a note about the > ] state of synchronization objects becoming undefined when a shared > ] region is unmapped. Very interesting, thanks. BTW, is there a chance of Austin Group get notified of, and possibly adopting, MAP_EXCL flag ? > > > Current updates to the patch https://kib.kiev.ua/kib/pshared/robust.4.patch > Do you have any further notes, or do you want to give the patch more time ? If not, are you fine with 'Reviewed by' attribution ? Thanks. From owner-freebsd-threads@freebsd.org Fri May 13 22:34:38 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38B69B3A198 for ; Fri, 13 May 2016 22:34:38 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 22E991EED for ; Fri, 13 May 2016 22:34:38 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 1ABD0B3A196; Fri, 13 May 2016 22:34:38 +0000 (UTC) Delivered-To: threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A289B3A194; Fri, 13 May 2016 22:34:38 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DA77E1EEB; Fri, 13 May 2016 22:34:37 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id 82C52358C5C; Sat, 14 May 2016 00:34:34 +0200 (CEST) Received: by toad2.stack.nl (Postfix, from userid 1677) id E9300892FF; Sat, 14 May 2016 00:34:34 +0200 (CEST) Date: Sat, 14 May 2016 00:34:34 +0200 From: Jilles Tjoelker To: Konstantin Belousov Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160513223434.GA47987@stack.nl> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> <20160513153716.GA30576@stack.nl> <20160513201916.GF89104@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160513201916.GF89104@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2016 22:34:38 -0000 On Fri, May 13, 2016 at 11:19:16PM +0300, Konstantin Belousov wrote: > On Fri, May 13, 2016 at 05:37:16PM +0200, Jilles Tjoelker wrote: > > On Mon, May 09, 2016 at 05:51:07AM +0300, Konstantin Belousov wrote: > > The mmap() page in POSIX.1-2008tc1 XSH 3 has: > > ] The state of synchronization objects such as mutexes, semaphores, > > ] barriers, and conditional variables placed in shared memory mapped > > ] with MAP_SHARED becomes undefined when the last region in any process > > ] containing the synchronization object is unmapped. > > This is new in issue 7 (SUSv4): > > ] Austin Group Interpretations 1003.1-2001 #078 and #079 are applied, > > ] clarifying page alignment requirements and adding a note about the > > ] state of synchronization objects becoming undefined when a shared > > ] region is unmapped. > Very interesting, thanks. > BTW, is there a chance of Austin Group get notified of, and possibly > adopting, MAP_EXCL flag ? You could send a message to the mailing list. MAP_EXCL is not used in the FreeBSD base but Debian code search shows Chromium and Hurd ftpfs mentioning it in comments. > > > Current updates to the patch > > > https://kib.kiev.ua/kib/pshared/robust.4.patch > Do you have any further notes, or do you want to give the patch more time ? > If not, are you fine with 'Reviewed by' attribution ? I don't trust the if (umtx_shm_vnobj_persistent) in sys/vm/vnode_pager.c: + if (umtx_shm_vnobj_persistent) + umtx_shm_object_terminated(obj); If umtx_shm_vnobj_persistent is turned off via sysctl between the check in sys/vm/vm_object.c and this one, don't we have a leak? 'Reviewed by: jilles' is fine otherwise. -- Jilles Tjoelker From owner-freebsd-threads@freebsd.org Sat May 14 10:10:50 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83D44B3A892 for ; Sat, 14 May 2016 10:10:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6D1A41B7D for ; Sat, 14 May 2016 10:10:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 6C541B3A890; Sat, 14 May 2016 10:10:50 +0000 (UTC) Delivered-To: threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6BDF0B3A88F; Sat, 14 May 2016 10:10:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E25271B7A; Sat, 14 May 2016 10:10:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u4EAAiVJ060181 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 14 May 2016 13:10:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u4EAAiVJ060181 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u4EAAiVg060178; Sat, 14 May 2016 13:10:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 14 May 2016 13:10:44 +0300 From: Konstantin Belousov To: Jilles Tjoelker Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160514101044.GM89104@kib.kiev.ua> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> <20160513153716.GA30576@stack.nl> <20160513201916.GF89104@kib.kiev.ua> <20160513223434.GA47987@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160513223434.GA47987@stack.nl> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 May 2016 10:10:50 -0000 On Sat, May 14, 2016 at 12:34:34AM +0200, Jilles Tjoelker wrote: > On Fri, May 13, 2016 at 11:19:16PM +0300, Konstantin Belousov wrote: > > BTW, is there a chance of Austin Group get notified of, and possibly > > adopting, MAP_EXCL flag ? > > You could send a message to the mailing list. MAP_EXCL is not used in > the FreeBSD base but Debian code search shows Chromium and Hurd ftpfs > mentioning it in comments. It is not clear if they reference FreeBSD flag, or just come to this semi-obvious idea independently. Anway, it is not used because feature is not present anywhere (except us). I do not want to present this on ag list as a stranger with bad language. > I don't trust the if (umtx_shm_vnobj_persistent) in > sys/vm/vnode_pager.c: > > + if (umtx_shm_vnobj_persistent) > + umtx_shm_object_terminated(obj); > > If umtx_shm_vnobj_persistent is turned off via sysctl between the check > in sys/vm/vm_object.c and this one, don't we have a leak? Removed the test in the vnode_destroy_vobject(), doing the umtx termination unconditionally on reclaim. > > 'Reviewed by: jilles' is fine otherwise. Updated patch is at https://www.kib.kiev.ua/kib/pshared/robust.5.patch P.S. I will document UMUTEX_ROBUST/UMUTEX_RB_* and UMTX_OP_ROBUST_LISTS after the feature is committed.