From owner-freebsd-arch@freebsd.org Sun May 8 12:52:26 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F727B330FC for ; Sun, 8 May 2016 12:52:26 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 2E06C13B1 for ; Sun, 8 May 2016 12:52:26 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 2D608B330FA; Sun, 8 May 2016 12:52:26 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CE84B330F9; Sun, 8 May 2016 12:52:26 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B1F0213AE; Sun, 8 May 2016 12:52:25 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id 1FB7EB805C; Sun, 8 May 2016 14:52:22 +0200 (CEST) Received: by toad2.stack.nl (Postfix, from userid 1677) id E4E21892B2; Sun, 8 May 2016 14:52:22 +0200 (CEST) Date: Sun, 8 May 2016 14:52:22 +0200 From: Jilles Tjoelker To: Konstantin Belousov Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160508125222.GA48862@stack.nl> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160507165956.GC89104@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 May 2016 12:52:26 -0000 On Sat, May 07, 2016 at 07:59:56PM +0300, Konstantin Belousov wrote: > On Sat, May 07, 2016 at 01:30:11AM +0200, Jilles Tjoelker wrote: > > > 4. The sysctl kern.ipc.umtx_vnode_persistent is added, which controls > > > the lifetime of the shared mutex associated with a vnode' page. > > > Apparently, there is real code around which expects the following > > > to work: > > > - mmap a file, create a shared mutex in the mapping; > > > - the process exits; > > > - another process starts, mmaps the same file and expects that the > > > previously initialized mutex is still usable. > > > The knob changes the lifetime of such shared off-page from the > > > 'destroy on last unmap' to either 'until vnode is reclaimed' or > > > until 'pthread_mutex_destroy' called, whatever comes first. > > The 'until vnode is reclaimed' bit sounds like a recipe for hard to > > reproduce bugs. > > I do think it is related to the robust mutex patch, though. > > Without robust mutexes and assuming threads do not unmap the memory > > while having a mutex locked or while waiting on a condition variable, it > > is sufficient to create the off-page mutex/condvar automatically in its > > initial state when pthread_mutex_lock() or pthread_cond_*wait() are > > called and no off-page object exists. > > With robust mutexes, we need to store somewhere whether the next thread > > should receive [EOWNERDEAD] or not, and this should persist even if no > > processes have the memory mapped. This might be done by replacing > > THR_PSHARED_PTR with a different value in the pthread_mutex_t. I'm not > > sure I like that memory write being done from the kernel though. > In principle, I agree with this. Note that if we go with something > like THR_OWNERDEAD_PTR, the kernel write to set the value would be not > much different from the kernel write to unlock robust mutex with inlined > lock structures. > Still, I would prefer to to implement this now. For the local purposes, > the knob was enough, and default value will be 'disabled'. OK. The patch still initializes umtx_shm_vnobj_persistent to 1 though. There is also a leak if umtx_shm_vnobj_persistent is toggled from 1 to 0 while an unmapped object with an off-page is active. I forgot two issues in my analysis. Firstly, the automatic re-initialization (from r297185, in fact) allows the next thread to lock the mutex even if the previous thread terminated while holding the lock. Normally, one would expect that non-robust mutexes cause the lock attempt to hang (or fail if it's a trylock), except if a thread ID is reused (in which case the lock attempt may instead succeed recursively or fail). Secondly, the initialization attributes are lost. If the next thread after last unmap sees THR_PSHARED_PTR, it has no idea whether it should be robust or not, which type it should have and whether it should be PP/PI. A mutex that was originally initialized as non-robust should not be re-initialized as robust since the code will not handle the [EOWNERDEAD] and [ENOTRECOVERABLE] errors (in the worst case, errors are completely ignored and it is as if the mutex did not exist). Special-casing robust to store in the THR_PSHARED_PTR location seems strange since the other attributes are still lost. All this is POSIX-compliant since POSIX specifies that the state of synchronization objects becomes undefined on last unmap, and our implementation fundamentally depends on that possibility. Unfortunately, Linux and Solaris do not need the possibility. The automatic re-initialization and umtx_vnode_persistent are just hacks that make certain applications almost always work (but not always, and in such cases it will be hard to debug). Another issue with umtx_vnode_persistent is that it can hide high memory usage. Filling up a page with pthread_mutex_t will create many pages full of actual mutexes. This memory usage is only visible as long as it is still mapped somewhere. Apart from that, umtx_vnode_persistent can (at least conceptually) work fully reliably for shared memory objects and tmpfs files, which do not have persistent storage. > > The below is a review of https://kib.kiev.ua/kib/pshared/robust.2.patch > > > + if ((error2 == 0 || error2 == EOWNERDEAD) && cancel) > > > _thr_testcancel(curthread); > > I don't think [EOWNERDEAD] should be swept under the carpet like this. > > The cancellation cleanup handler will use the protected state and unlock > > the mutex without making the state consistent and the state will be > > unrecoverable. > So your argument there is to return EOWNERDEAD and not cancelling, > am I right ? I reused part of your text as the comment. Yes. > > > +void > > > +_mutex_leave_robust(struct pthread *curthread, struct pthread_mutex *m) > > > +{ > > > + > > > + if (!is_robust_mutex(m)) > > > + return; > > This accesses the mutex after writing a value to the lock > > word which allows other threads to lock it. A use after free may result, > > since it is valid for another thread to lock, unlock and destroy the > > mutex (assuming the mutex is not used otherwise later). > > Memory ordering may permit the load of m->m_lock.m_flags to be moved > > before the actual unlock, so this issue may not actually appear. > > Given that the overhead of a system call on every robust mutex unlock is > > not desired, the kernel's unlock of a terminated thread's mutexes will > > unavoidably have this use after free. However, for non-robust mutexes > > the previous guarantees should be kept. > I agree that this is a bug, and agree that the kernel accesses to the > curthread->inact_mtx are potentially unsafe. I also did not wanted to > issue a syscall for unlock of a robust mutex, as you noted. > I fixed the bug with the is_robust_mutex() test in _mutex_leave_robust() > by caching the robust status. > I was indeed worried by the kernel access issue you mentioned, but > kernel is immune to 'bad' memory accesses. What bothered me is the ill > ABA situation, where the lock memory is freed and repurposed, and then > the lock word is written with the one of two specific values which give > the same state as for locked mutex. This would cause kernel to 'unlock' > it (but not to follow the invalid m_rb_link). > But for this to happen, we must have a situation where a thread > is being terminated before mutex_unlock_common() reached the > _mutex_leave_robust() call. This is async thread termination, which > then must be either process termination (including execve()), or a call > to thr_exit() from a signal handler in our thread (including async > cancellation). > I am sure that the thr_exit() situation is non-conforming, so the only > concern is the process exit, and then, shared robust mutex, because for > private mutex, only the exiting process state is affected. I can verify > in umtx_handle_rb(), for instance, that for USYNC_PROCESS_SHARED object, > the underlying memory is backed by the umtx shm page. This would close > the race. > But this would interfere with libthr2, if that ever happen. Hmm, libthr2 or non-standard synchronization primitive implementations seem a good reason to not check for umtx shm page. However, the existing checks can be made stricter. The umtx_handle_rb() from robust.3.patch will use m_rb_lnk with no validation at all except that it is a valid pointer. However, if the UMUTEX_ROBUST flag is not set, the mutex should not have been in this list at all and it is probably safer to ignore m_rb_lnk. > > > @@ -37,7 +37,11 @@ struct umutex { > > > + __uintptr_t m_rb_lnk; /* Robust linkage */ > > Although Linux also stores the robust list nodes in the mutexes like > > this, I think it increases the chance of strange memory corruption. > > Making the robust list an array in the thread's memory would be more > > reliable. If the maximum number of owned robust mutexes can be small, > > this can have a fixed size; otherwise, it needs to grow as needed (which > > does add an allocation that may fail to the pthread_mutex_lock path, > > bad). > I gave this proposal some thought. > I very much dislike an idea of calling memory allocator on the lock, and > esp. the trylock, path. The later could need to obtain allocator locks > which (sometimes partially) defeat the trylock purpose. > I can use mmap(2) directly there, similarly how pthread_setspecific() > was changed recently, which would avoid the issue of calling userspace > allocator. Still, the problem of an addiitonal syscall, resulting > ENOMEM and also the time to copy the current robust owned list to grown > location are there (I do not see that using chunked allocations is > reasonable, since it would be the same list as current m_rb_lnk, but at > different level. > I prefer to keep the robust linked list for these reasons. There is a difference between chunked allocations and the current m_rb_lnk in that the list would reside in local memory, not vulnerable to other processes scribbling over it. This is probably not a major issue since sharing a mutex already allows threads to block each other indefinitely. The X server has used shared memory across privilege levels for years but not with anything like mutexes and condition variables. It uses X protocol messages and xshmfence (custom synchronization object that does not block the X server). > In fact, the deciding argument would be actual application usage of the > robustness. I thought, when writing the patch, when and how would I use > the feature, and did not see compelling arguments to even try to use it. > My stumbling block is the user data consistency recovery: for instance, > I tried to write a loop which would increment shared variable N times, > and I was not able to end up with any simple recovery mechanism from > the aborted iteration, except writing iteration in assembly and have a > parallel tick variable which enumerate each iteration action. One way of using this is to never call pthread_mutex_consistent() and only use it to report the problem and avoid hangs and crashes. A slightly extended version would tear down the whole structure including shared memory and all involved processes and rebuild it. When calling pthread_mutex_consistent(), some data loss will almost unavoidably occur. In some cases, it may be possible to re-initialize just the state protected by the mutex. Alternatively, a thread that modifies the state may make a copy of it before modifying it (some thread-local StoreStore fences are needed); the [EOWNERDEAD] handling can then put back the copy if it exists. This way, robust mutexes can be used to make updates transactional. Approaches based on message passing will be easier to get right and more reliable but they might be too slow. > Updated patch is at https://kib.kiev.ua/kib/pshared/robust.3.patch > I did not added the check for umtx shm into the umtx_handle_rb() yet, > waiting for your opinion. -- Jilles Tjoelker From owner-freebsd-arch@freebsd.org Sun May 8 22:27:13 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25506B33369 for ; Sun, 8 May 2016 22:27:13 +0000 (UTC) (envelope-from bounces+2732470-a481-freebsd-arch=freebsd.org@sg.wixshoutout.com) Received: from o4.sg.wixshoutout.com (o4.sg.wixshoutout.com [167.89.77.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E16771EEF for ; Sun, 8 May 2016 22:27:12 +0000 (UTC) (envelope-from bounces+2732470-a481-freebsd-arch=freebsd.org@sg.wixshoutout.com) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=wixshoutout.com; h=content-type:from:list-unsubscribe:mime-version:subject:to; s=s1; bh=syHAwyr67Zj1XVd7tmK7h/i7iJc=; b=jZZCSNhJ+aKKGZY1wBgpGRv x1nIca72c6oqO7L2vkFGSNqpopFxHfZ59XRRwjhmUd+0as2QB6vuUmW1aoAmukCW YMh3GXl6i1fsp1L8ANazSEDF5wQP9oPePDp0Tq9azPSWt2TuKuFZS8TmhHmoaIie WZxIWInQyQfFeNIAgPVQ= Received: by filter0173p1las1.sendgrid.net with SMTP id filter0173p1las1.24630.572FBD4010 2016-05-08 22:27:12.365903749 +0000 UTC Received: from MjczMjQ3MA (216-139-213-200.aus.us.siteprotect.com [216.139.213.200]) by ismtpd0008p1las1.sendgrid.net (SG) with HTTP id aRoA45fLRuSkOFwcuO6Pgg for ; Sun, 08 May 2016 22:27:12.516 +0000 (UTC) Date: Sun, 08 May 2016 22:27:12 +0000 From: "PriceHouse Accounting" Mime-Version: 1.0 Precedence: Bulk Subject: =?UTF-8?Q?=C2=A0__Business_support_in_Estonia?= To: freebsd-arch@freebsd.org Message-ID: X-SG-EID: t2fXfoZHCw6vGsGKHqKxJ1vvZFmqZoRBNxUXljRrVNYYYYg9ITzmM5+NPyGwdp5HxMFAWIqCzykVXg mcsP5toimxLlknXMnytDCK5HTl0y+F/bp0Gnhy/pubu7ttRtsKu7BhGb9uE9ivW9a6Zxu70KlXCqFT PJZtPiKizmkM/3hkO8tgQaHNrWU+w4NM6ofASAAUfLO+ipp8AGXv95RJDzwI7Qahg7uY0QeDatJra+ 3/eP+SP3KMRU4KRD/HGAuY X-SG-ID: NrBj8y1E2RHVMZO8PbtvIYehR6O3HfSMlq8ddug5Wi2jLMnOd1x7r+8lUDECUs9IDoa/GhipqR+yB5 FcZknJHOjn6mqM3t2eenbStku/p2GYAtqH99ox1YxGY+bEoaFJnmOGmYvE5oOE8PFBGhF1IhJDTFXU eztkGv1o4RX2FTglbN1bUayd2sveBJZ5hG3UP9sonFBItRf926UPWjGcr2IRRtpYJNgVPiI/Lr59J8 ZAVUmJsjPTQXffDF3gnxKCqpzRjLvL7vjqYAkhs1+t+tQbQhhVXiuLXc5+Zc4J5fwHNvCUF+CIfI3x mGGlYh47ECWQHthom67LrIzuO5Q3qQ== Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 May 2016 22:27:13 -0000 =C2=A0 Business support in Estonia New shelf company in Estonia - from 750=E2=82=AC Old shelf company in Eston= ia - from 1200=E2=82=AC VAT registered shelf company in Estonia - from 1900= =E2=82=AC Company liquidation in EU - from 2500=E2=82=AC VAT registered she= lf company in Latvia - from 1900=E2=82=AC VAT registered shelf company in L= ithuania - from 2500=E2=82=AC VAT registered shelf company in UK - from 190= 0=E2=82=AC VAT registered shelf company in Poland - from 2500=E2=82=AC Acco= unting and payroll in Estonia - from 150=E2=82=AC Address services and mail= forwarding - from 30=E2=82=AC per month Legal services in Estonia - from 1= 50=E2=82=AC Company everyday management ( managing director service ) in Es= tonia - from 3000=E2=82=AC per month phone +372-5089-565 eng-rus-est-fi= n WhatsApp 372-5089-565, Viber 372-5089-565 Skype:line7775 Email: valmisfir= ma@gmail.com www.accounting100.com =C2=A0 =C2=A0 =C2=A0 Click on the link below to open the message in a browser: http://accounting100.com/so/1LIIPkkM?cid=3D33516468-206b-4045-8bd2-cc5e77e0= adbd You've received this email because you are a subscriber of this site http://www.accounting100.com/ If you feel you received it by mistake or wish to unsubscribe, click here http://www.wix.com/my-account/contacts/unsubscribe?metaSiteId=3D9a60d7b9-0b= ce-4a3d-87bc-ba265003c010&unsubscribeToken=3Dabcefd4d5885911e7f4ea863ecce18= 0388e169a39e4c714f1a99cb3376c2b706d52930165695c96b2236c897c392cd794113a98e1= 2684e2b874fe4c7543984680db954d276628467514fc6825f86d5d3445fe65559299dd1ed48= d4aea7087f26e2f327080a8f9661595d8b8733bbde509e7b1fccddfb045756bc9b914217b3c= 626975a9f02ca428004db106c4753140b81e83d635d95b74702c57c816559c563b810b5e390= 08180faf590f5a921b43b30fdaf3db1b7b0889a2473c0fb1b229b2e006d287cce8d36931401= 337910a0e4e8f1e7ad6bf0a2e76450249099195b2a35cbf4647944a399d6c6200af9924de29= 588fd3dc0d403ecb4dc08e12cd1f5cbd65e4019d8103581b064c7c1eb9b85b6a09d2dbc7364= 5c20d0df26936c847c4a1c7ac9c12ac8c179afe95c1ab2e952b8d07750b9d6b367ee67ed3d6= da555959d541e8c3f6ab40538468cb23542e8104e5b39af1aed946df9a7c5b9473a449d9147= cb6d46a4db5dbb3121276d43530fc7e32443ca4808a575209093ec5392f4dc9df1aee9a6ace= 2926379566c81e2650d7&siteName=3Daccounting-estonia= From owner-freebsd-arch@freebsd.org Mon May 9 02:51:14 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69F46B3331D for ; Mon, 9 May 2016 02:51:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 545B11843 for ; Mon, 9 May 2016 02:51:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 50308B3331B; Mon, 9 May 2016 02:51:14 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FAE7B3331A; Mon, 9 May 2016 02:51:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D2C4E1840; Mon, 9 May 2016 02:51:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u492p7u1027902 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 9 May 2016 05:51:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u492p7u1027902 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u492p7Gr027895; Mon, 9 May 2016 05:51:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 9 May 2016 05:51:07 +0300 From: Konstantin Belousov To: Jilles Tjoelker Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160509025107.GN89104@kib.kiev.ua> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160508125222.GA48862@stack.nl> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 May 2016 02:51:14 -0000 On Sun, May 08, 2016 at 02:52:22PM +0200, Jilles Tjoelker wrote: > OK. The patch still initializes umtx_shm_vnobj_persistent to 1 though. > There is also a leak if umtx_shm_vnobj_persistent is toggled from 1 to 0 > while an unmapped object with an off-page is active. > > I forgot two issues in my analysis. > > Firstly, the automatic re-initialization (from r297185, in fact) allows > the next thread to lock the mutex even if the previous thread terminated > while holding the lock. Normally, one would expect that non-robust > mutexes cause the lock attempt to hang (or fail if it's a trylock), > except if a thread ID is reused (in which case the lock attempt may > instead succeed recursively or fail). > > Secondly, the initialization attributes are lost. If the next thread > after last unmap sees THR_PSHARED_PTR, it has no idea whether it should > be robust or not, which type it should have and whether it should be > PP/PI. A mutex that was originally initialized as non-robust should not > be re-initialized as robust since the code will not handle the > [EOWNERDEAD] and [ENOTRECOVERABLE] errors (in the worst case, errors are > completely ignored and it is as if the mutex did not exist). > > Special-casing robust to store in the THR_PSHARED_PTR location seems > strange since the other attributes are still lost. Yes, these are all good points. > > All this is POSIX-compliant since POSIX specifies that the state of > synchronization objects becomes undefined on last unmap, and our > implementation fundamentally depends on that possibility. Unfortunately, Could you, please, point me to the exact place in the standard where this is allowed ? > Linux and Solaris do not need the possibility. The automatic > re-initialization and umtx_vnode_persistent are just hacks that make > certain applications almost always work (but not always, and in such > cases it will be hard to debug). > > Another issue with umtx_vnode_persistent is that it can hide high memory > usage. Filling up a page with pthread_mutex_t will create many pages > full of actual mutexes. This memory usage is only visible as long as it > is still mapped somewhere. There is already a resource limit for the number of pshared locks per uid, RLIMIT_UMTXP. When exceeded, user would get somewhat obscure failure mode, but excessive memory consumption is not allowed. And I think that vmstat -o would give enough info to diagnose, except that users must know about it and be quialified enough to interpret the output. > > Apart from that, umtx_vnode_persistent can (at least conceptually) work > fully reliably for shared memory objects and tmpfs files, which do not > have persistent storage. I changed defaults for the umtx_vnode_persistent to 0 in the published patch. > Hmm, libthr2 or non-standard synchronization primitive implementations > seem a good reason to not check for umtx shm page. > > However, the existing checks can be made stricter. The umtx_handle_rb() > from robust.3.patch will use m_rb_lnk with no validation at all except > that it is a valid pointer. However, if the UMUTEX_ROBUST flag is not > set, the mutex should not have been in this list at all and it is > probably safer to ignore m_rb_lnk. Ok, I changed the code to consider lack of UMUTEX_ROBUST as a stopper for the list walk. Also, I stop the walk if mutex is not owned by the current thread, except when the mutex was stored in inact slot. The same piece of changes hopefully fixes list walk for COMPAT32 on big-endian machines. > > There is a difference between chunked allocations and the current > m_rb_lnk in that the list would reside in local memory, not vulnerable > to other processes scribbling over it. This is probably not a major > issue since sharing a mutex already allows threads to block each other > indefinitely. I would easily deletegate the chunked array to some future reimplementation if not the ABI issue. Still, I do not like it. Current updates to the patch https://kib.kiev.ua/kib/pshared/robust.4.patch From owner-freebsd-arch@freebsd.org Wed May 11 01:24:28 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4EA1FB36639 for ; Wed, 11 May 2016 01:24:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 334651822 for ; Wed, 11 May 2016 01:24:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 2F316B36638; Wed, 11 May 2016 01:24:28 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CA95B36636 for ; Wed, 11 May 2016 01:24:28 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-oi0-f44.google.com (mail-oi0-f44.google.com [209.85.218.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6A921820; Wed, 11 May 2016 01:24:27 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-oi0-f44.google.com with SMTP id x201so44591566oif.3; Tue, 10 May 2016 18:24:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:date:message-id:subject :from:to:cc; bh=gysBeLLIBl7o8s9x6cV3t7g33eYMcXQskf18XVDDAvE=; b=HAHuRBMPutKgIXeb/rRM7DOJkBJOZtt+n45U8rXx7LT7jcGIPYrD+s9dN76fUJkr/b O2vKK1aJTJ4SxS4QEN+M9ApMt1Spdupyb/H2T+8JKK5XANC5dwDOVZVi5Ar568F9TnV7 QjBYofMPpUJbSc4tEahSr+8hS4NwwQLKR1ZuKBEKy1spdkwvOx3OlXUUfv6tLM0Ez4R+ KO91Oy+Gh7T8zlo8SXzJVfIeJJXuaDjW8bV1huhONeMUNMo0SZIh0WSnj1ry7NtbbSmE X48Y2vh4KdJOytHxUB/gRT6TQNTGQcOF/QypZ1Ph1MXp+ej0zIHXjj5ZhAaOJQ71SUXE 75bw== X-Gm-Message-State: AOPr4FUo+3iCDogVLsqowohosdmwHPeCJgzAtJOIXVVdO+aEjprGd+IFz2hN6RT5kPgz1g== X-Received: by 10.157.12.241 with SMTP id o46mr354409otd.16.1462929861267; Tue, 10 May 2016 18:24:21 -0700 (PDT) Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com. [209.85.218.46]) by smtp.gmail.com with ESMTPSA id u11sm1363398oie.29.2016.05.10.18.24.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 May 2016 18:24:21 -0700 (PDT) Received: by mail-oi0-f46.google.com with SMTP id v145so44535926oie.0; Tue, 10 May 2016 18:24:20 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.202.102.222 with SMTP id m91mr309517oik.79.1462929860761; Tue, 10 May 2016 18:24:20 -0700 (PDT) Reply-To: cem@FreeBSD.org Received: by 10.157.6.111 with HTTP; Tue, 10 May 2016 18:24:20 -0700 (PDT) Date: Tue, 10 May 2016 18:24:20 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: KASSERT: always assert; KWARN From: Conrad Meyer To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 01:24:28 -0000 I'd like to logically revert r243980 and r244105, such that KASSERT uses the __dead2-annotated panic(9). Going back to the old behavior enables Coverity and other static analyzers to reason about KASSERT invariants via the __dead2 panic(9) path. This proposal is in https://reviews.freebsd.org/D6117 . As a follow-up, to match the assumed intent of the r243980 changes, I propose a KWARN facility which may be muted, rate limited, or even cause panic. Generally, KASSERTs should not be KWARNs. That proposal is here: https://reviews.freebsd.org/D6134 Finally, I am looking for suggestions of things it *does* make sense to KWARN about. One suggestion was witness_warn; however, it doesn't seem like a great fit (without adding allocating sbufs in, anyway). A sketch of that is in https://reviews.freebsd.org/D6306 . Thoughts or objections? Does anyone like the ability to opt out of invariants asserts? Best, Conrad From owner-freebsd-arch@freebsd.org Wed May 11 03:39:33 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44D74B36D82 for ; Wed, 11 May 2016 03:39:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 283FA1CB3 for ; Wed, 11 May 2016 03:39:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 279DCB36D80; Wed, 11 May 2016 03:39:33 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27478B36D7D for ; Wed, 11 May 2016 03:39:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E8EE21CB2; Wed, 11 May 2016 03:39:32 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x22c.google.com with SMTP id i75so34342265ioa.3; Tue, 10 May 2016 20:39:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=C8FhrsoxuUdkMPW/sOzwB7tCw1DTzwcxVs/MyUtYqNo=; b=A7Z1kkjKlUF9PWJwzjgyTGMCqjhdZ0xPwriID+tZOHhnJFa9Navf5LgeyfmedlAkIL 0VYEU15D4QeSw9Gq6zuXaLRRJxsigE3DToH+phdQoOFcc7cA7dicb7NoyBVwcFPXkavD NBdNUSEoB+FS+VBUThR8Ie71Hwgas904EmfHQadLx7d2TviGs71Gz8qjIDrBX8x0YO43 TnUQnz6bOVb4JOPzzDLI71Z1UGjXs4x/dZo1euTzjn53RyAkduSyPgLm9HMoY5ad/ELr /ObWfnLmDCRIXZASqhcuGOn3Bf35no0T0y0fewwU1dEHiFcxVIGcXxYpbNYLkDBVySut 3hzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=C8FhrsoxuUdkMPW/sOzwB7tCw1DTzwcxVs/MyUtYqNo=; b=jMCYPdUWYfZXApDCLqqHkv0OBhaEWAtkOHylysN5K4chwD/V+NDNghivzeIULkDefT D1viJH4yi9tpAyoY3iAEIy0YRlYp7QmWwOxpHnGzQob5DC00VwK9c5nM8QRS5pK5guYu +aeFQB2dsZnWgWFXQjRNW+B7v7zdBO4CWhhSbJBNCdMheCwmSxb3E1ROiXNByYaMT8Cc 65qsJBC3EmODFi9QOopiBR/XqTsUkJXZiEn9dz6Riu/RQO90V6OkDv52hO4XMxYgQ4FH vXbIlzh7XMinBFR/DNittlBriBa5MRFFOfJgEksrx9lyt2n4Th36r/bBQk35JUB3+rB+ PuJg== X-Gm-Message-State: AOPr4FXFSrM5jA32fBvlIsOc9JsdNHXDcLPLWAhBohmexcDdLuIVq5bQpU1KYVBAro9U2LdL6EmXxJTFnzOnQw== MIME-Version: 1.0 X-Received: by 10.107.144.135 with SMTP id s129mr1257211iod.165.1462937972398; Tue, 10 May 2016 20:39:32 -0700 (PDT) Received: by 10.36.113.3 with HTTP; Tue, 10 May 2016 20:39:32 -0700 (PDT) In-Reply-To: References: Date: Tue, 10 May 2016 20:39:32 -0700 Message-ID: Subject: Re: KASSERT: always assert; KWARN From: Adrian Chadd To: cem@freebsd.org Cc: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 03:39:33 -0000 i found it very useful to get asserts to print, rather than panic. -a On 10 May 2016 at 18:24, Conrad Meyer wrote: > I'd like to logically revert r243980 and r244105, such that KASSERT > uses the __dead2-annotated panic(9). > > Going back to the old behavior enables Coverity and other static > analyzers to reason about KASSERT invariants via the __dead2 panic(9) > path. > > This proposal is in https://reviews.freebsd.org/D6117 . > > As a follow-up, to match the assumed intent of the r243980 changes, I > propose a KWARN facility which may be muted, rate limited, or even > cause panic. Generally, KASSERTs should not be KWARNs. That proposal > is here: https://reviews.freebsd.org/D6134 > > Finally, I am looking for suggestions of things it *does* make sense > to KWARN about. One suggestion was witness_warn; however, it doesn't > seem like a great fit (without adding allocating sbufs in, anyway). A > sketch of that is in https://reviews.freebsd.org/D6306 . > > Thoughts or objections? Does anyone like the ability to opt out of > invariants asserts? > > Best, > Conrad > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@freebsd.org Wed May 11 07:14:43 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C45BB36DEC for ; Wed, 11 May 2016 07:14:43 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 2C08D11ED for ; Wed, 11 May 2016 07:14:43 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 2B629B36DEB; Wed, 11 May 2016 07:14:43 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B080B36DEA for ; Wed, 11 May 2016 07:14:43 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E7D1E11EC; Wed, 11 May 2016 07:14:42 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mail-io0-x230.google.com with SMTP id f89so43635283ioi.0; Wed, 11 May 2016 00:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc; bh=MmssOIe03nrNbsUOgQe4MelpjqjdIbNS/7/gILsudsw=; b=Ba2+jlsVCe4qV2vQj3Fh5jef6p/O4AkpewAaEJJwmAGs9GeiGDTr09sCJwAE6JJ85D Uazen8immMi4pN4AZtwwr5iO0Mp6EZhyLmJDnWNhhIXwYlfO7XVsHZKAGEYUD1ingMbl LDmi395PXoQwhtZavsoQM9k3P8y2Hq9F9OVyQqbOvccZ2HfwazXcqABOvyGbWx6EgIHn fXe2u4SXEFUCbOHfx9aU0CZBJVuaDceobKqNV7mRD6kvZLGxW6tBFBkcQ0Yt1B35XN19 MSZmd+2uWi8sM4HSlRIKrqe4GERR52fOq1tRrEkRQh9shKka6uma5pG9/cBDT3n8vnRJ YVkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc; bh=MmssOIe03nrNbsUOgQe4MelpjqjdIbNS/7/gILsudsw=; b=Y+Vs8YcI/LeWXUErw3/wrZRPAFw0JpZiRJIVeEi0fs2zzvxB37ty2jkb7ieaSKqJFC GyocdfHwHAGKzDMoAngzGd9nkoMjzt7xhm6gqOYJZkVkuCktCcSVmmEkmHSs1hDRxbXc sZ/n0KYOZMRgsqExTq/k8hN4RZFF9JsY5/fbTdL27JUMUyqAPupoztkkbTjT0RZQMHs3 wIMFJLanO7Q5Lj/C6MLNzjqoeU1/earFtA9J7VocZc0Ezuxo2ygFtND/Ql046onjFs2i YFb6OT5PJKUmv+pPuO883xxWyVJDokV8n8Ho5U77lN85YV+jh5o76eYU2Owh2S4EqHKr SfQA== X-Gm-Message-State: AOPr4FXotNnD9cF43uUqNw9nNpSd7E6r28Hof7dANdOcQm241QSEDSsS1SFjmRnCRrnFN1hEqW8Bkf8hoGFr6Q== MIME-Version: 1.0 X-Received: by 10.107.7.170 with SMTP id g42mr1942264ioi.81.1462950882415; Wed, 11 May 2016 00:14:42 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.107.140.8 with HTTP; Wed, 11 May 2016 00:14:42 -0700 (PDT) In-Reply-To: References: Date: Wed, 11 May 2016 00:14:42 -0700 X-Google-Sender-Auth: nfxCSnwJckWrtA34bxXu9PTwvIM Message-ID: Subject: Re: KASSERT: always assert; KWARN From: "K. Macy" To: Adrian Chadd Cc: "cem@freebsd.org" , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 07:14:43 -0000 Ugh. Et tu Brutus. Then they either shouldn't be asserts or you need a new mode for Adrian and friends. Either way, if people don't want to hit panics from asserts they should not run with invariants enabled. No one other than manic depressive developers look at logs in the common case. On Tuesday, May 10, 2016, Adrian Chadd wrote: > i found it very useful to get asserts to print, rather than panic. > > > > -a > > > On 10 May 2016 at 18:24, Conrad Meyer > > wrote: > > I'd like to logically revert r243980 and r244105, such that KASSERT > > uses the __dead2-annotated panic(9). > > > > Going back to the old behavior enables Coverity and other static > > analyzers to reason about KASSERT invariants via the __dead2 panic(9) > > path. > > > > This proposal is in https://reviews.freebsd.org/D6117 . > > > > As a follow-up, to match the assumed intent of the r243980 changes, I > > propose a KWARN facility which may be muted, rate limited, or even > > cause panic. Generally, KASSERTs should not be KWARNs. That proposal > > is here: https://reviews.freebsd.org/D6134 > > > > Finally, I am looking for suggestions of things it *does* make sense > > to KWARN about. One suggestion was witness_warn; however, it doesn't > > seem like a great fit (without adding allocating sbufs in, anyway). A > > sketch of that is in https://reviews.freebsd.org/D6306 . > > > > Thoughts or objections? Does anyone like the ability to opt out of > > invariants asserts? > > > > Best, > > Conrad > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org > " > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org > " > From owner-freebsd-arch@freebsd.org Wed May 11 16:04:06 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FD4DB37334 for ; Wed, 11 May 2016 16:04:06 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 836241695 for ; Wed, 11 May 2016 16:04:06 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from Alfreds-MacBook-Pro-2.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 1D59C346DE31 for ; Wed, 11 May 2016 09:04:00 -0700 (PDT) Subject: Re: KASSERT: always assert; KWARN To: freebsd-arch@freebsd.org References: From: Alfred Perlstein Organization: FreeBSD Message-ID: Date: Wed, 11 May 2016 09:04:41 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 16:04:06 -0000 On 5/10/16 6:24 PM, Conrad Meyer wrote: > Thoughts or objections? Does anyone like the ability to opt out of > invariants asserts? Yes. During my time at iXsystems we used this facility several times to get a log from a customer site with a number of "kasserts". The reason we did this was multiple reasons: 1) We needed to ship a kernel with asserts enabled. 2) When we did, the first assert hit was: a) In an unrelated module and not relevant. b) Not enough information came back from just the first assert. 3) We found it more useful to get multiple errors back from a customer in one trip rather than one fix at a time. Unfortunately one fix at a time would have had us lose the customer. The KASSERT/assert system is very, very, very useful. However if you are at a last resort sending a debug kernel (with Kassert enabled) and do not get enough information back then you will lose that customer. I understand that a few vocal folks are upset, like seriously, seriously upset, however at the time this was the only way we could effectively debug a customer problem and my hope was that others could make use of it as well. Linux has had a similar functionality for many years. In Linux there is the kernel "oops()" which does nearly the same thing. Initially I mocked Linux's "oops" for being silly and "wrong", using the exact same reasons that many have used to dislike "kassert_warn". However once I was responsible for an extremely pissed off customer who was paying us quite a sum of money AND I was not getting enough information back, I changed my mind. https://en.wikipedia.org/wiki/Linux_kernel_oops -Alfred From owner-freebsd-arch@freebsd.org Wed May 11 16:30:54 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 290A5B37B78 for ; Wed, 11 May 2016 16:30:54 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-oi0-f66.google.com (mail-oi0-f66.google.com [209.85.218.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EDB921969 for ; Wed, 11 May 2016 16:30:53 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-oi0-f66.google.com with SMTP id w198so7954662oiw.2 for ; Wed, 11 May 2016 09:30:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :date:message-id:subject:from:to; bh=roD8tkemRcnvYjcMIhSjeRSsQq1EHVQdP78PawIaQpQ=; b=WdX/T8pmMhWsz89BidAiJIgWDx6EdEp6u/WZEu8iTf/WC/uawkJDckfFrhFEuLVkZ+ F7Adhgco+72RvRupINyxvbYXSNr1VcyXq+r07//kH0FmwV31vs4rUWtIDXSLTTU5qP9H V6otbxqkJDc2lyvidrBK4+wiqKxP6bTfs8rE23yjvDURMZg9C/+8LFXfgAdoOcnmcA7S bnSiKWu43Dz+feHP3ZRgWBZGAjT65ufOcRl8+wcUR0GfyQqLPeHCdr7L0BJnjA1QDCyM 417M610eEt7iZRkQUePe03h9zmD65b2hdcjNx+8vzf5VrI5VAI3kOjuHKpe2pyLpmhkD vpyA== X-Gm-Message-State: AOPr4FX3kpg5rZ3UWN7E8UkUalbmq7HnxQsOGVXbMlHxfIVTXRlzO2qzw/mh6lvp65o4Rw== X-Received: by 10.202.220.87 with SMTP id t84mr2319242oig.57.1462984247304; Wed, 11 May 2016 09:30:47 -0700 (PDT) Received: from mail-oi0-f48.google.com (mail-oi0-f48.google.com. [209.85.218.48]) by smtp.gmail.com with ESMTPSA id ry2sm2319694oeb.4.2016.05.11.09.30.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 May 2016 09:30:47 -0700 (PDT) Received: by mail-oi0-f48.google.com with SMTP id x201so76532701oif.3 for ; Wed, 11 May 2016 09:30:46 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.202.215.86 with SMTP id o83mr2349462oig.55.1462984246638; Wed, 11 May 2016 09:30:46 -0700 (PDT) Reply-To: cem@FreeBSD.org Received: by 10.157.19.20 with HTTP; Wed, 11 May 2016 09:30:46 -0700 (PDT) In-Reply-To: References: Date: Wed, 11 May 2016 09:30:46 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: KASSERT: always assert; KWARN From: Conrad Meyer To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 16:30:54 -0000 On Wed, May 11, 2016 at 9:04 AM, Alfred Perlstein wrote: > On 5/10/16 6:24 PM, Conrad Meyer wrote: > >> Thoughts or objections? Does anyone like the ability to opt out of >> invariants asserts? > > > Yes. > > During my time at iXsystems we used this facility several times to get a log > from a customer site with a number of "kasserts". > > The reason we did this was multiple reasons: > 1) We needed to ship a kernel with asserts enabled. > 2) When we did, the first assert hit was: > a) In an unrelated module and not relevant. > b) Not enough information came back from just the first assert. > 3) We found it more useful to get multiple errors back from a customer in > one trip rather than one fix at a time. Unfortunately one fix at a time > would have had us lose the customer. > > The KASSERT/assert system is very, very, very useful. However if you are at > a last resort sending a debug kernel (with Kassert enabled) and do not get > enough information back then you will lose that customer. > > I understand that a few vocal folks are upset, like seriously, seriously > upset, however at the time this was the only way we could effectively debug > a customer problem and my hope was that others could make use of it as well. > > Linux has had a similar functionality for many years. In Linux there is the > kernel "oops()" which does nearly the same thing. > > Initially I mocked Linux's "oops" for being silly and "wrong", using the > exact same reasons that many have used to dislike "kassert_warn". However > once I was responsible for an extremely pissed off customer who was paying > us quite a sum of money AND I was not getting enough information back, I > changed my mind. > > https://en.wikipedia.org/wiki/Linux_kernel_oops Hi, Here's my follow-up from the Phabricator review. (Alfred, you've already seen it. But, for everyone else:) Here's another proposal: Add a mode between INVARIANTS off and INVARIANTS on. Call it INVARIANTS_OPTIONAL. * In !INVARIANTS mode, you don't have KASSERTs at all (like today). * In INVARIANTS && INVARIANTS_OPTIONAL mode, you get the all the print-and-continue KASSERT knobs you have today. (So, same default of panicing, but optionally they can be disabled and turned into logs.) * In INVARIANTS && !INVARIANTS_OPTIONAL mode, KASSERT always panics. I would suggest GENERIC move from the current mode, effectively INVARIANTS_OPTIONAL, to !INVARIANTS_OPTIONAL. But if you want to ship a kernel with pass-through assertions enabled, you can still do that by enabling INVARIANTS_OPTIONAL. This gives the expected KASSERT behavior for Coverity modeling, and still enables the KASSERT-lite use case. (It would just be kicked out of GENERIC.) Adrian, would that meet your needs? Thanks, Conrad From owner-freebsd-arch@freebsd.org Wed May 11 17:40:59 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B58A2B37F90 for ; Wed, 11 May 2016 17:40:59 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F0DE1EB8; Wed, 11 May 2016 17:40:59 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 38AB4B98F; Wed, 11 May 2016 13:40:58 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org, cem@freebsd.org Subject: Re: KASSERT: always assert; KWARN Date: Wed, 11 May 2016 09:48:56 -0700 Message-ID: <31200026.OetD7h0dHc@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 11 May 2016 13:40:58 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 17:40:59 -0000 On Wednesday, May 11, 2016 09:30:46 AM Conrad Meyer wrote: > On Wed, May 11, 2016 at 9:04 AM, Alfred Perlstein wrote: > > On 5/10/16 6:24 PM, Conrad Meyer wrote: > > > >> Thoughts or objections? Does anyone like the ability to opt out of > >> invariants asserts? > > > > > > Yes. > > > > During my time at iXsystems we used this facility several times to get a log > > from a customer site with a number of "kasserts". > > > > The reason we did this was multiple reasons: > > 1) We needed to ship a kernel with asserts enabled. > > 2) When we did, the first assert hit was: > > a) In an unrelated module and not relevant. > > b) Not enough information came back from just the first assert. > > 3) We found it more useful to get multiple errors back from a customer in > > one trip rather than one fix at a time. Unfortunately one fix at a time > > would have had us lose the customer. > > > > The KASSERT/assert system is very, very, very useful. However if you are at > > a last resort sending a debug kernel (with Kassert enabled) and do not get > > enough information back then you will lose that customer. > > > > I understand that a few vocal folks are upset, like seriously, seriously > > upset, however at the time this was the only way we could effectively debug > > a customer problem and my hope was that others could make use of it as well. > > > > Linux has had a similar functionality for many years. In Linux there is the > > kernel "oops()" which does nearly the same thing. > > > > Initially I mocked Linux's "oops" for being silly and "wrong", using the > > exact same reasons that many have used to dislike "kassert_warn". However > > once I was responsible for an extremely pissed off customer who was paying > > us quite a sum of money AND I was not getting enough information back, I > > changed my mind. > > > > https://en.wikipedia.org/wiki/Linux_kernel_oops > > Hi, > > Here's my follow-up from the Phabricator review. (Alfred, you've > already seen it. But, for everyone else:) > > Here's another proposal: > > Add a mode between INVARIANTS off and INVARIANTS on. Call it > INVARIANTS_OPTIONAL. > > * In !INVARIANTS mode, you don't have KASSERTs at all (like today). > > * In INVARIANTS && INVARIANTS_OPTIONAL mode, you get the all the > print-and-continue KASSERT knobs you have today. (So, same default of > panicing, but optionally they can be disabled and turned into logs.) > > * In INVARIANTS && !INVARIANTS_OPTIONAL mode, KASSERT always panics. > > I would suggest GENERIC move from the current mode, effectively > INVARIANTS_OPTIONAL, to !INVARIANTS_OPTIONAL. But if you want to ship > a kernel with pass-through assertions enabled, you can still do that > by enabling INVARIANTS_OPTIONAL. > > This gives the expected KASSERT behavior for Coverity modeling, and > still enables the KASSERT-lite use case. (It would just be kicked out > of GENERIC.) > > Adrian, would that meet your needs? Eh, if you keep going past many of the assertions the original code enabled, you will get _more_ bogus assertions as fallout. For example, if you aren't holding the correct locks (or you try to unlock a lock locked in a different mode), then you are going to start corrupting data resulting in false positives. This is like getting 20 compiler errors due to missing a semicolon earlier in the file. The first error is real, the rest are noise. Sometimes there are actual errors in the noise, but you have to sort through a lot of noise. However, the compiler doesn't normally mangle the rest of your source code after your first error, but int the case of assertions the kernel often _is_ going to start mangling your data after your first failure. If you have things that aren't actual errors but for which you _handle_ the unexpected case instead of just blindly corrupting data, then you can use KWARN for those cases. However, you have to actually handle the condition for that to be safe. In your case of the cdrom driver, if you weren't using it, then turn it off (i.e. take it out of the kernel) if you don't have time to debug it. -- John Baldwin From owner-freebsd-arch@freebsd.org Wed May 11 23:50:08 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64DFFB37394 for ; Wed, 11 May 2016 23:50:08 +0000 (UTC) (envelope-from rpaulo@me.com) Received: from mr11p00im-asmtp002.me.com (mr11p00im-asmtp002.me.com [17.110.69.253]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D3FA1507; Wed, 11 May 2016 23:50:08 +0000 (UTC) (envelope-from rpaulo@me.com) Received: from process-dkim-sign-daemon.mr11p00im-asmtp002.me.com by mr11p00im-asmtp002.me.com (Oracle Communications Messaging Server 7.0.5.36.0 64bit (built Sep 8 2015)) id <0O7100O00DZ9PB00@mr11p00im-asmtp002.me.com>; Wed, 11 May 2016 23:50:02 +0000 (GMT) MIME-version: 1.0 Content-transfer-encoding: 8BIT Content-type: text/plain; charset=UTF-8 Received: from akita.hsd1.ca.comcast.net (c-73-222-51-139.hsd1.ca.comcast.net [73.222.51.139]) by mr11p00im-asmtp002.me.com (Oracle Communications Messaging Server 7.0.5.36.0 64bit (built Sep 8 2015)) with ESMTPSA id <0O7100MLIE7CA010@mr11p00im-asmtp002.me.com>; Wed, 11 May 2016 23:50:01 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-05-11_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1510270003 definitions=main-1605110311 Message-id: <1463010600.7806.10.camel@me.com> Subject: Re: KASSERT: always assert; KWARN From: Rui Paulo To: John Baldwin , freebsd-arch@freebsd.org, cem@freebsd.org Date: Wed, 11 May 2016 16:50:00 -0700 In-reply-to: <31200026.OetD7h0dHc@ralph.baldwin.cx> References: <31200026.OetD7h0dHc@ralph.baldwin.cx> X-Mailer: Evolution 3.18.5.1-1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=me.com; s=4d515a; t=1463010602; bh=YWEQtq+pDZCRgw2FOSUSiUGVgwJuBS5gz+kaVlFxPUo=; h=MIME-version:Content-type:Message-id:Subject:From:To:Date; b=O4sMn00Sl3m0eviU6faIu6ADdlEA0pOBuMlRhk5Hhl7HKcLfgFD1BWsfMlyjDW9X9 umlTiN5qVJt2AQQ2jYNJIHDfM3PiBAEmqYxtzxRH8XVf23oBsWkA820u9+tdfqw5y2 7E8bEz96okj9H7Zxnq7c8UMZs2h4yZLQ13gbD7YynP/ZLa9id8VeEHsEubGlQX9wye 0vrmAHNcgUgibTX8I8In5K62mWnOf5iWabc/fx6Igv4Mlh5FfwBNOZmaoXFV/ESr3j ZFJcPo8TFTA57OTUVCxRZb+nvX8zYp04Xlb+dowV2xlZjmWWbuHPWwp/N4s9GU8hnP stGr70wlOHrSw== X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 23:50:08 -0000 On Wed, 2016-05-11 at 09:48 -0700, John Baldwin wrote: > Eh, if you keep going past many of the assertions the original code > enabled, you will get _more_ bogus assertions as fallout. I agree with John.  We really shouldn't be fiddling with this. There's just one case where this could be useful: you just imported a bunch of code that you don't understand and you're using the kassert's as a way to create a mental model of the code.  However, this is a very specific edge case. -- Rui Paulo From owner-freebsd-arch@freebsd.org Thu May 12 02:33:35 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A400B363C8 for ; Thu, 12 May 2016 02:33:35 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 8B16F1DE7; Thu, 12 May 2016 02:33:35 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from Alfreds-MacBook-Pro-2.local (unknown [IPv6:2601:645:8003:a4d6:5813:8fd5:ad51:daf5]) by elvis.mu.org (Postfix) with ESMTPSA id 84C87346DE31; Wed, 11 May 2016 19:33:34 -0700 (PDT) Subject: Re: KASSERT: always assert; KWARN To: Rui Paulo , John Baldwin , freebsd-arch@freebsd.org, cem@freebsd.org References: <31200026.OetD7h0dHc@ralph.baldwin.cx> <1463010600.7806.10.camel@me.com> From: Alfred Perlstein Organization: FreeBSD Message-ID: Date: Wed, 11 May 2016 19:34:17 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <1463010600.7806.10.camel@me.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 02:33:35 -0000 On 5/11/16 4:50 PM, Rui Paulo wrote: > On Wed, 2016-05-11 at 09:48 -0700, John Baldwin wrote: >> Eh, if you keep going past many of the assertions the original code >> enabled, you will get _more_ bogus assertions as fallout. > I agree with John. We really shouldn't be fiddling with this. > > There's just one case where this could be useful: you just imported a > bunch of code that you don't understand and you're using the kassert's > as a way to create a mental model of the code. However, this is a very > specific edge case. > I'm not sure what you and John are suggesting here. Are you suggesting to revert the explicit panics in the witness code, or to revert the entire kassert_warn subsystem? -Alfred From owner-freebsd-arch@freebsd.org Thu May 12 03:37:05 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9BA20B2D512 for ; Thu, 12 May 2016 03:37:05 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7EF7A1E08 for ; Thu, 12 May 2016 03:37:05 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 7E4EDB2D511; Thu, 12 May 2016 03:37:05 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DF80B2D510 for ; Thu, 12 May 2016 03:37:05 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-io0-x231.google.com (mail-io0-x231.google.com [IPv6:2607:f8b0:4001:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BAC21E07; Thu, 12 May 2016 03:37:05 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mail-io0-x231.google.com with SMTP id d62so79756517iof.2; Wed, 11 May 2016 20:37:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc; bh=0otaq8NOzQdvfV7AalhloiC9jdHs5GzJKw8l1NaBfwk=; b=h04uSLnegpPd2fUMcWmwYIvc+5DmUcirawqt0RZBzaH9iE+F9fhCOps4ECDYkUwXPc VpGMIJya1pdrXIL60eZ05MWZQ7HjtYTwps5Ka+iP0ztW4DBovRsNQeqqdrrHQ1qBE6rr 6IoUQUo6nW4cyL1aYJOMoeBCYp4I68iVYyjqpGYjF30UAs6eJlhjAJDe29nobCGSSC/j enqsMp8ktK24wQrmUo7rKyKrwUKF2maAMzAZiuvtbHKIHLHl4mHgikoEyF3G4R/vj8Vn 38aze9ZIhucNzi/A8/nOPwXU8ym7O9u1UJIl++sEPNG4YrkmnVPA/VbCKfpnmRJgzZR+ JYew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:date:message-id:subject:from :to:cc; bh=0otaq8NOzQdvfV7AalhloiC9jdHs5GzJKw8l1NaBfwk=; b=NsGElI+kEPPIJQAIHpQftfStaOP1H8ZeTr3sqPpaXUV+FYhupCQb5wRU+azMbfS9EO xQ1hiZUmesPszawL+Ev3FJTG5UVRnyNP9q0bmlQtW8+lsCpMIaFnHOvVqz9+Z78WKt5v 3jCsohMWZffazq7xzPHk2B5U9AMRn0nSEN8V1FvAyvfr1EU4Zs8SygmbKYQsQw/05q6N KDls5HSdtBKCBoX2vcrN/k0b+Du7m7K0dNy6q5mSKfJZ/A0tgEEAra3UWpZkOlE/n1Ml JT5ex7SRGqYQM3Sbp9FHGsQdyU8cTwiZ0v0C69AFPq4TkM9Xst2whoBunMaOzd/usB/i m2uQ== X-Gm-Message-State: AOPr4FXhMR8Mk5pA9ttsxgzNflI/O3UyzMecPdhB6Iw4G1UrewJWmywUYzNd3vko7DTT9hYDJ8Q7zCzPxxHCmQ== MIME-Version: 1.0 X-Received: by 10.36.29.13 with SMTP id 13mr6803857itj.99.1463024224307; Wed, 11 May 2016 20:37:04 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.107.140.8 with HTTP; Wed, 11 May 2016 20:37:04 -0700 (PDT) Date: Wed, 11 May 2016 20:37:04 -0700 X-Google-Sender-Auth: SXO59ZbbqQ3fS7svlPKQGenw87A Message-ID: Subject: MMU Notifiers and the quest for graphics feature parity From: "K. Macy" To: "freebsd-arch@freebsd.org" Cc: Ed Maste , Mark Johnston Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 03:37:05 -0000 My motivation for writing this email stems from my recent work extending the linuxkpi to facilitate updating graphics support for FreeBSD. I've made reasonable progress limiting my changes to the linuxkpi so far. I have 3D HW acceleration working on an essentially unmodified 4.6-rc5 drm and i915 driver. The 4.6 i915 driver supports all of Haswell (3.8 doesn't), Broadwall, Skylake, and the, as of yet, unreleased Kaby Lake processors. Additionally, I am expecting a Thinkpad e565 (AMD based) to arrive on Friday which will allow me to bring up amdgpu (the Radeon driver for post 2013 AMD GPU hardware) this weekend. I find this exciting as it means that FreeBSD will be able to run on _all_ x86 graphics hardware. Unfortunately, most if not all ARM GPU drivers are closed source. This isn't quite everything though. There is a new "userptr" ioctl in graphics drivers that allows applications to safely register user memory with the driver. This works because if pages are evicted or the process goes away, the driver will be notified first. This notification takes place through the "mmu notifier" facility. MMU Notifiers are essentially just VM callbacks (or in FreeBSD parlance eventhandlers). They were first added to Linux to enable KVM to better interoperate with the native VM's resident set management: https://lwn.net/Articles/266320/ They're now also used by IB, IOMMUs, MPSS (Xeon Phi), and graphics drivers. The API is well enough documented in the header file: http://lxr.free-electrons.com/source/include/linux/mmu_notifier.h I don't have strong feelings for what these are called or their precise location. I do, however, need to be able to support roughly equivalent semantics when I register these eventhandlers from the notifer calls in the linuxkpi. I haven't worked on the VM in a year or two so don't know quite how to proceed. I'm happy to produce patches for review, but fear that they might be rejected out of hand for one reason or another. I'd like feedback on the approach itself and what I might do to make any proposed patches more palatable. Thanks in advance. From owner-freebsd-arch@freebsd.org Thu May 12 10:30:42 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B023EB388D4 for ; Thu, 12 May 2016 10:30:42 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 912191BA1 for ; Thu, 12 May 2016 10:30:42 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 8CBD9B388D3; Thu, 12 May 2016 10:30:42 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C5D9B388D2 for ; Thu, 12 May 2016 10:30:42 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: from mail-oi0-x232.google.com (mail-oi0-x232.google.com [IPv6:2607:f8b0:4003:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CE7C1BA0; Thu, 12 May 2016 10:30:42 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: by mail-oi0-x232.google.com with SMTP id k142so112411929oib.1; Thu, 12 May 2016 03:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-transfer-encoding; bh=/Vofah/vBNqFSIh9a1ISInlH5GhjKBfmnfd2gjanGag=; b=Ji61iHqtby4WhUNL8p1Q/nHVZHHMoDnom/FU+WpDsOSG34zKtmqEJuP/23ozR4+FQN M2kAzIrtSdKa73PxVD0yKCr/AlzT61DIkkkTggeotGZ/wsshxtag/OGB2RG1yZ5tWgwK QojvaiU/c8vWswqD1Xs9NhiqnE692x0otLiluh8J55+tsjLZVoNhyITK8UUOM2BqjLO+ T2m7bYUtifPeTc8w0Es+ldhW/yNm1CfV7M89tFUSEZnfnZiXGzWYmE/uwqQj9kQQ30+z Rx38yH4XWzz2rL6PUWm+R3bPun0BKUXA020J9pJbVJUhm6QgiHkgyMaG9qg/iIOUzBGj 3hTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-transfer-encoding; bh=/Vofah/vBNqFSIh9a1ISInlH5GhjKBfmnfd2gjanGag=; b=aMT0tJU4i3Jagqiuz58rxO0azK8er4Ne58dnVaMoXHtQlkhy580Bje/lFv8rzkiGFY S+MWKtPrToo9yh5wvyputISXo+FDTxZb4Lt+Wxz7NwruEmLr5D1B45fgREWSM4maqEQD JvAy7uEf7wxTkAa/eIpTRUaDznwmToZlBlXYN8JFw1wblnKbI61wQ6cumOCRReUE2Cca PgdGEivyqg8rrMGBrO9M+18+WtKOACjy5xCea6KelnxmlJSu381Vf0MdYVX3nQDgAPS0 WdvpP8yqGA/26TrxsucBhqPeMsu23jsNPnwA0QGyW5Y5MU1MWtsr6+SNxf3RtmUm0jMf xtHQ== X-Gm-Message-State: AOPr4FWxo2Ts65eWvJqAd59XQPJS31GdvAa2qFLBHolXfN4r41Q8bS6RLj5EMvaKphTHPt2OloXoakDn6VD6ug== MIME-Version: 1.0 X-Received: by 10.202.60.131 with SMTP id j125mr4378926oia.42.1463049041285; Thu, 12 May 2016 03:30:41 -0700 (PDT) Received: by 10.60.9.129 with HTTP; Thu, 12 May 2016 03:30:41 -0700 (PDT) In-Reply-To: References: Date: Thu, 12 May 2016 12:30:41 +0200 Message-ID: Subject: Re: MMU Notifiers and the quest for graphics feature parity From: =?UTF-8?Q?Ren=C3=A9_Ladan?= To: "K. Macy" Cc: "freebsd-arch@freebsd.org" , Mark Johnston , Ed Maste Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 10:30:42 -0000 2016-05-12 5:37 GMT+02:00 K. Macy : > My motivation for writing this email stems from my recent work extending = the > linuxkpi to facilitate updating graphics support for FreeBSD. I've made > reasonable progress limiting my changes to the linuxkpi so far. I have 3D= HW > acceleration working on an essentially unmodified 4.6-rc5 drm and i915 dr= iver. > The 4.6 i915 driver supports all of Haswell (3.8 doesn't), Broadwall, Sky= lake, > and the, as of yet, unreleased Kaby Lake processors. Additionally, I am > expecting a Thinkpad e565 (AMD based) to arrive on Friday which will allo= w me > to bring up amdgpu (the Radeon driver for post 2013 AMD GPU hardware) thi= s > weekend. I find this exciting as it means that FreeBSD will be able to ru= n on > _all_ x86 graphics hardware. Unfortunately, most if not all ARM GPU drive= rs > are closed source. > > This isn't quite everything though. There is a new "userptr" ioctl in > graphics drivers that allows applications to safely register user memory = with > the driver. This works because if pages are evicted or the process goes = away, > the driver will be notified first. This notification takes place through = the > "mmu notifier" facility. > > MMU Notifiers are essentially just VM callbacks (or in FreeBSD parlance > eventhandlers). They were first added to Linux to enable KVM to better > interoperate with the native VM's resident set management: > > https://lwn.net/Articles/266320/ > > They're now also used by IB, IOMMUs, MPSS (Xeon Phi), and graphics driver= s. > The API is well enough documented in the header file: > > http://lxr.free-electrons.com/source/include/linux/mmu_notifier.h > > > I don't have strong feelings for what these are called or their precise > location. I do, however, need to be able to support roughly equivalent > semantics when I register these eventhandlers from the notifer calls in t= he > linuxkpi. > > I haven't worked on the VM in a year or two so don't know quite how to pr= oceed. > I'm happy to produce patches for review, but fear that they might be reje= cted > out of hand for one reason or another. I'd like feedback on the approach > itself and what I might do to make any proposed patches more palatable. > I have a laptop with both an nVidia GeForce 940M and a built-in graphics card on the Intel i7 6500U CPU: vgapci0@pci0:0:2:0: class=3D0x030000 card=3D0x09851025 chip=3D0x1916808= 6 rev=3D0x07 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'HD Graphics 520' class =3D display subclass =3D VGA So if I understand correctly this 520 chip might work with your patch? Stock FreeBSD 11.0 r298793 with xf86-video-intel 2.21.15_9 does not see the driver. For now it does run with scfb. Thanks for your work, Ren=C3=A9 --=20 https://rene-ladan.nl/ From owner-freebsd-arch@freebsd.org Thu May 12 10:45:11 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDD45B38B3F for ; Thu, 12 May 2016 10:45:11 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9C4941749 for ; Thu, 12 May 2016 10:45:11 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 97F44B38B3E; Thu, 12 May 2016 10:45:11 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95476B38B3D for ; Thu, 12 May 2016 10:45:11 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 486141746; Thu, 12 May 2016 10:45:11 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mail-ig0-x22d.google.com with SMTP id m9so50126551ige.1; Thu, 12 May 2016 03:45:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc; bh=6wGGdF6zkL1rsWW6CeCrdHKqhIuQ4sDIn2KXiKN8dEc=; b=kwjkhlvsmtAraZdMZt9CYPeHrZgofgtlvwiuP1ZQ6A87cBWxNMcmI0s74M9QIfPrrW sX8NEw69wLlLyFe5D21GTJbbpbZjKTXvKAYcQz6hV7oA/JNfYvbYTpW3l1FSFZ7tzkQS OWR2VMh2FP98Nk/Q61fk0zDhkbL7vO5rZn2r5ossu/Q9Fk7BFnLUDaWQZyf8H+2U3xu6 +tf7LUFwNvPLpVCdop0rY3UX/0bM8Nt0stBW6XVCI9OzZDrstE0iWZjV/v6oBOHq2qXH Dpy+NHkQd1mDgKrG5k2e0b8OYuLGmc6rUCUDiuw9DQZE9s4QDVPgsQMgu1s3fvdOwVgZ nOaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:date :message-id:subject:from:to:cc; bh=6wGGdF6zkL1rsWW6CeCrdHKqhIuQ4sDIn2KXiKN8dEc=; b=jsE4YFiyBWIaYEqPxup2DJzc8VsaJWmt7JXAUwn5Giy47fY6yQ5dgaaPszXx9mO3xH KYqpPlPV+Nr9oZJl8F0vjPwGWo5ojIgdEsIE3fKVchv0LKVEQanVemXf2rclVTbx8vJN DQsicDGqjjcwZ+17zLVP6uncfpU1X+NjwvEHmhg0cumqVzSFsl19LoTu+G56QAdoM8Yj abe4LL9cLMEBqTCJB7dczLiC3/fQWXDF732wI8xJd4dfi2uQ0r62ge2f7stomzYvMZFm TMrgY/94dFUR5zSgyP4a80idBZUPhMOAGnQScgXTCmjIw527OcA6DBbMcaIpOAcK5qIx qVdw== X-Gm-Message-State: AOPr4FVAkUnlJCOyakr7lxOuqeUIUeAuumS9w1rZ28FlgwEK9w2SY9GTLFxWP+hG7zFA9asa5oc6+FpaqDDpBg== MIME-Version: 1.0 X-Received: by 10.50.124.39 with SMTP id mf7mr5450871igb.95.1463049910589; Thu, 12 May 2016 03:45:10 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.107.140.8 with HTTP; Thu, 12 May 2016 03:45:10 -0700 (PDT) In-Reply-To: References: Date: Thu, 12 May 2016 03:45:10 -0700 X-Google-Sender-Auth: uwrdSodKrl-dsLjv_K_Oi6rHTmQ Message-ID: Subject: Re: MMU Notifiers and the quest for graphics feature parity From: "K. Macy" To: =?UTF-8?Q?Ren=C3=A9_Ladan?= Cc: "freebsd-arch@freebsd.org" , Mark Johnston , Ed Maste Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2016 10:45:11 -0000 On Thursday, May 12, 2016, Ren=C3=A9 Ladan wrote: > 2016-05-12 5:37 GMT+02:00 K. Macy >: > > My motivation for writing this email stems from my recent work extendin= g > the > > linuxkpi to facilitate updating graphics support for FreeBSD. I've made > > reasonable progress limiting my changes to the linuxkpi so far. I have > 3D HW > > acceleration working on an essentially unmodified 4.6-rc5 drm and i915 > driver. > > The 4.6 i915 driver supports all of Haswell (3.8 doesn't), Broadwall, > Skylake, > > and the, as of yet, unreleased Kaby Lake processors. Additionally, I am > > expecting a Thinkpad e565 (AMD based) to arrive on Friday which will > allow me > > to bring up amdgpu (the Radeon driver for post 2013 AMD GPU hardware) > this > > weekend. I find this exciting as it means that FreeBSD will be able to > run on > > _all_ x86 graphics hardware. Unfortunately, most if not all ARM GPU > drivers > > are closed source. > > > > This isn't quite everything though. There is a new "userptr" ioctl in > > graphics drivers that allows applications to safely register user memor= y > with > > the driver. This works because if pages are evicted or the process goe= s > away, > > the driver will be notified first. This notification takes place throug= h > the > > "mmu notifier" facility. > > > > MMU Notifiers are essentially just VM callbacks (or in FreeBSD parlance > > eventhandlers). They were first added to Linux to enable KVM to better > > interoperate with the native VM's resident set management: > > > > https://lwn.net/Articles/266320/ > > > > They're now also used by IB, IOMMUs, MPSS (Xeon Phi), and graphics > drivers. > > The API is well enough documented in the header file: > > > > http://lxr.free-electrons.com/source/include/linux/mmu_notifier.h > > > > > > I don't have strong feelings for what these are called or their precise > > location. I do, however, need to be able to support roughly equivalent > > semantics when I register these eventhandlers from the notifer calls in > the > > linuxkpi. > > > > I haven't worked on the VM in a year or two so don't know quite how to > proceed. > > I'm happy to produce patches for review, but fear that they might be > rejected > > out of hand for one reason or another. I'd like feedback on the approac= h > > itself and what I might do to make any proposed patches more palatable. > > > I have a laptop with both an nVidia GeForce 940M and a built-in > graphics card on the Intel i7 6500U CPU: > vgapci0@pci0:0:2:0: class=3D0x030000 card=3D0x09851025 chip=3D0x19168= 086 > rev=3D0x07 hdr=3D0x00 > vendor =3D 'Intel Corporation' > device =3D 'HD Graphics 520' > class =3D display > subclass =3D VGA > > So if I understand correctly this 520 chip might work with your patch? > Stock FreeBSD 11.0 r298793 with xf86-video-intel 2.21.15_9 does not > see the driver. > For now it does run with scfb. Yes. I don't have a Skylake based laptop (I will shortly), so I haven't actually tested it yet. That's part of what I'm waiting on for a CFT. I haven't tested prime support yet at all so switching between the discrete and integrated based on C state or other power management policy is a week or two off - at least. I appreciate your enthusiasm but please let's move all device support and other X11 discussion to the X11 list. Thanks. -M > > Thanks for your work, > Ren=C3=A9 > -- > https://rene-ladan.nl/ > From owner-freebsd-arch@freebsd.org Fri May 13 15:37:19 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B87C1B39DF2 for ; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id A6EEE1C78 for ; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id A2F3FB39DF0; Fri, 13 May 2016 15:37:19 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2796B39DEF; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3AD101C70; Fri, 13 May 2016 15:37:19 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id 0A28E358C68; Fri, 13 May 2016 17:37:16 +0200 (CEST) Received: by toad2.stack.nl (Postfix, from userid 1677) id 7228F892FF; Fri, 13 May 2016 17:37:16 +0200 (CEST) Date: Fri, 13 May 2016 17:37:16 +0200 From: Jilles Tjoelker To: Konstantin Belousov Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160513153716.GA30576@stack.nl> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160509025107.GN89104@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2016 15:37:19 -0000 On Mon, May 09, 2016 at 05:51:07AM +0300, Konstantin Belousov wrote: > On Sun, May 08, 2016 at 02:52:22PM +0200, Jilles Tjoelker wrote: > > OK. The patch still initializes umtx_shm_vnobj_persistent to 1 though. > > There is also a leak if umtx_shm_vnobj_persistent is toggled from 1 to 0 > > while an unmapped object with an off-page is active. > [snip] > > All this is POSIX-compliant since POSIX specifies that the state of > > synchronization objects becomes undefined on last unmap, and our > > implementation fundamentally depends on that possibility. Unfortunately, > Could you, please, point me to the exact place in the standard where > this is allowed ? The mmap() page in POSIX.1-2008tc1 XSH 3 has: ] The state of synchronization objects such as mutexes, semaphores, ] barriers, and conditional variables placed in shared memory mapped ] with MAP_SHARED becomes undefined when the last region in any process ] containing the synchronization object is unmapped. This is new in issue 7 (SUSv4): ] Austin Group Interpretations 1003.1-2001 #078 and #079 are applied, ] clarifying page alignment requirements and adding a note about the ] state of synchronization objects becoming undefined when a shared ] region is unmapped. > > Linux and Solaris do not need the possibility. The automatic > > re-initialization and umtx_vnode_persistent are just hacks that make > > certain applications almost always work (but not always, and in such > > cases it will be hard to debug). > > Another issue with umtx_vnode_persistent is that it can hide high memory > > usage. Filling up a page with pthread_mutex_t will create many pages > > full of actual mutexes. This memory usage is only visible as long as it > > is still mapped somewhere. > There is already a resource limit for the number of pshared locks per > uid, RLIMIT_UMTXP. When exceeded, user would get somewhat obscure > failure mode, but excessive memory consumption is not allowed. And I > think that vmstat -o would give enough info to diagnose, except that > users must know about it and be quialified enough to interpret the > output. Hmm, OK. > > Apart from that, umtx_vnode_persistent can (at least conceptually) work > > fully reliably for shared memory objects and tmpfs files, which do not > > have persistent storage. > I changed defaults for the umtx_vnode_persistent to 0 in the published > patch. OK. > > Hmm, libthr2 or non-standard synchronization primitive implementations > > seem a good reason to not check for umtx shm page. > > However, the existing checks can be made stricter. The umtx_handle_rb() > > from robust.3.patch will use m_rb_lnk with no validation at all except > > that it is a valid pointer. However, if the UMUTEX_ROBUST flag is not > > set, the mutex should not have been in this list at all and it is > > probably safer to ignore m_rb_lnk. > Ok, I changed the code to consider lack of UMUTEX_ROBUST as a stopper > for the list walk. Also, I stop the walk if mutex is not owned by > the current thread, except when the mutex was stored in inact slot. > The same piece of changes hopefully fixes list walk for COMPAT32 on > big-endian machines. OK. > > There is a difference between chunked allocations and the current > > m_rb_lnk in that the list would reside in local memory, not vulnerable > > to other processes scribbling over it. This is probably not a major > > issue since sharing a mutex already allows threads to block each other > > indefinitely. > I would easily deletegate the chunked array to some future reimplementation > if not the ABI issue. Still, I do not like it. An array only works well for this if you know beforehand how long it needs to be, and I don't think we can do this since Linux's limit is so high that an array would waste a lot of memory. The existence of some limit is, however, unavoidable and it could be considered a bug that pthread_mutex_lock() for a robust mutex returns success even if it will not fulfill its promise to do the EOWNERDEAD thing. > Current updates to the patch https://kib.kiev.ua/kib/pshared/robust.4.patch -- Jilles Tjoelker From owner-freebsd-arch@freebsd.org Fri May 13 20:19:22 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F35DDB3AF59 for ; Fri, 13 May 2016 20:19:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id DD6BD1B4E for ; Fri, 13 May 2016 20:19:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id D9281B3AF56; Fri, 13 May 2016 20:19:22 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8BC4B3AF55; Fri, 13 May 2016 20:19:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5A7131B4B; Fri, 13 May 2016 20:19:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u4DKJGNV052208 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 13 May 2016 23:19:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u4DKJGNV052208 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u4DKJGQj052207; Fri, 13 May 2016 23:19:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 13 May 2016 23:19:16 +0300 From: Konstantin Belousov To: Jilles Tjoelker Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160513201916.GF89104@kib.kiev.ua> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> <20160513153716.GA30576@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160513153716.GA30576@stack.nl> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2016 20:19:23 -0000 On Fri, May 13, 2016 at 05:37:16PM +0200, Jilles Tjoelker wrote: > On Mon, May 09, 2016 at 05:51:07AM +0300, Konstantin Belousov wrote: > The mmap() page in POSIX.1-2008tc1 XSH 3 has: > > ] The state of synchronization objects such as mutexes, semaphores, > ] barriers, and conditional variables placed in shared memory mapped > ] with MAP_SHARED becomes undefined when the last region in any process > ] containing the synchronization object is unmapped. > > This is new in issue 7 (SUSv4): > > ] Austin Group Interpretations 1003.1-2001 #078 and #079 are applied, > ] clarifying page alignment requirements and adding a note about the > ] state of synchronization objects becoming undefined when a shared > ] region is unmapped. Very interesting, thanks. BTW, is there a chance of Austin Group get notified of, and possibly adopting, MAP_EXCL flag ? > > > Current updates to the patch https://kib.kiev.ua/kib/pshared/robust.4.patch > Do you have any further notes, or do you want to give the patch more time ? If not, are you fine with 'Reviewed by' attribution ? Thanks. From owner-freebsd-arch@freebsd.org Fri May 13 21:21:04 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCACAB3A384 for ; Fri, 13 May 2016 21:21:04 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A92D41AF3 for ; Fri, 13 May 2016 21:21:04 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id A87D6B3A382; Fri, 13 May 2016 21:21:04 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A80CFB3A381; Fri, 13 May 2016 21:21:04 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 99FC81AF2; Fri, 13 May 2016 21:21:04 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [IPv6:::1]) by freefall.freebsd.org (Postfix) with ESMTP id 88C58130D; Fri, 13 May 2016 21:21:04 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [172.31.3.2]) by mail.xzibition.com (Postfix) with ESMTP id 44C241DE32; Fri, 13 May 2016 21:21:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at mail.xzibition.com Received: from mail.xzibition.com ([172.31.3.2]) by mail.xzibition.com (mail.xzibition.com [172.31.3.2]) (amavisd-new, port 10026) with LMTP id yXO8NbM6egXt; Fri, 13 May 2016 21:21:00 +0000 (UTC) To: toolchain@FreeBSD.org DKIM-Filter: OpenDKIM Filter v2.9.2 mail.xzibition.com B30C31DE2C Cc: arch@FreeBSD.org From: Bryan Drewery Subject: universe/buildworld: Skipping building cross clang when /usr/bin/cc matches the source tree. Organization: FreeBSD Message-ID: <9c5bdbae-9faf-1f16-8358-e35f5e079426@FreeBSD.org> Date: Fri, 13 May 2016 14:20:57 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2016 21:21:04 -0000 Review at: https://reviews.freebsd.org/D6357 Full patch: https://people.freebsd.org/~bdrewery/patches/skip-clang-bootstrap-1.diff For context, when using an external compiler with our build currently it will use --sysroot/-L/-B flags as well as -target with clang. GCC assumes it is building the same arch as the compiler is built for since it lacks -target support. [This is just my commit message but I think it explains it well enough] Opportunistically skip building a cross-compiler with SMART_CROSS_COMPILER set. This will still build the compiler for the target but will not build the bootstrap cross-compiler in the cross-tools phase. Other toolchain bootstrapping, such as elftoolchan an binutils, currently still occurs. This will utilize the default CC (cc, /usr/bin/cc) as an external compile= r. This is planned to be on-by-default eventually. This will utilize the __FreeBSD_cc_version compiler macro defined in the source tree and compare it to CC's version. If they match then the cross-compiler is skipped. If [X]CC is an external compiler (absolute path) or WITHOUT_CROSS_COMPILER is already set, then this logic is skippe= d. If the expected bootstrap compiler type no longer matches the found CC compiler type (clang vs gcc), then the logic is skipped. As an extra safety check the version number is also compared from the compiler to the tree version. Clang: The macro FREEBSD_CC_VERSION is defined in: lib/clang/include/clang/Basic/Version.inc For clang -target will be used if TARGET_ARCH !=3D MACHINE_ARCH. This is from the current external toolchain logic. There is currently an assumption that the host compiler can build the TARGET_ARCH. This will usually be the case since we don't conditionalize target arch support in clang, but it will break when introducing new architectures. This problem is mitigated by incrementing the version when adding new architectures. GCC: The macro FBSD_CC_VER is defined in: gnu/usr.bin/cc/cc_tools/freebsd-native.h For GCC there is no simple -target support when TARGET_ARCH !=3D MACHINE_ARCH. In this case the opportunistic skip is not done. If we add proper support for this case in external toolchain logic then it will be fine to enable. This relies on the macros being incremented whenever any change occurs to these compilers that warrant rebuilding files. It also should never repeat earlier values. This also depends on http://reviews.llvm.org/D20037 and a change such as this: > index 94dc282..373cdb4 100644 > --- lib/clang/include/clang/Basic/Version.inc > +++ lib/clang/include/clang/Basic/Version.inc > @@ -8,3 +8,5 @@ > #define CLANG_VENDOR "FreeBSD " >=20 > #define SVN_REVISION "262564" > + > +#define FREEBSD_CC_VERSION 1100001U Here is a ministat as well (no ccache and clean build on all 6 builds). It's kind of obvious that skipping one of the clangs helps but I wanted to show how much it did for me: > x before > + after > +----------------------------------------------------------------------= ----+ > | + = x| > |+ + = x| > ||AM| = A| > +----------------------------------------------------------------------= ----+ > N Min Max Median Avg St= ddev > x 3 2966 2968 2966 2966.6667 1.154= 7005 > + 3 2300 2319 2318 2312.3333 10.69= 2677 > Difference at 95.0% confidence > -654.333 +/- 17.2371 > -22.0562% +/- 0.581024% > (Student's t, pooled s =3D 7.60482) This also works for universe! > ~/git/freebsd # grep SMART _*.buildworld > _.amd64.amd64.buildworld:make[2]: "/root/git/freebsd/Makefile.inc1" lin= e 144: SMART_CROSS_COMPILER: Determined that CC=3D/usr/local/bin/ccache c= c matches the source tree. Not bootstrapping a cross-compiler. > _.arm.arm.buildworld:make[2]: "/root/git/freebsd/Makefile.inc1" line 14= 4: SMART_CROSS_COMPILER: Determined that CC=3D/usr/local/bin/ccache cc ma= tches the source tree. Not bootstrapping a cross-compiler. > _.arm.armeb.buildworld:make[2]: "/root/git/freebsd/Makefile.inc1" line = 144: SMART_CROSS_COMPILER: Determined that CC=3D/usr/local/bin/ccache cc = matches the source tree. Not bootstrapping a cross-compiler. > _.arm.armv6.buildworld:make[2]: "/root/git/freebsd/Makefile.inc1" line = 144: SMART_CROSS_COMPILER: Determined that CC=3D/usr/local/bin/ccache cc = matches the source tree. Not bootstrapping a cross-compiler. > _.arm.armv6hf.buildworld:make[2]: "/root/git/freebsd/Makefile.inc1" lin= e 144: SMART_CROSS_COMPILER: Determined that CC=3D/usr/local/bin/ccache c= c matches the source tree. Not bootstrapping a cross-compiler. > _.i386.i386.buildworld:make[2]: "/root/git/freebsd/Makefile.inc1" line = 144: SMART_CROSS_COMPILER: Determined that CC=3D/usr/local/bin/ccache cc = matches the source tree. Not bootstrapping a cross-compiler. > _.pc98.i386.buildworld:make[2]: "/root/git/freebsd/Makefile.inc1" line = 144: SMART_CROSS_COMPILER: Determined that CC=3D/usr/local/bin/ccache cc = matches the source tree. Not bootstrapping a cross-compiler. A future improvement will be to build 1 clang before universe if /usr/bin/cc cannot be used. It will not be a cross-compiler but just a staged (in OBJDIR) external compiler that will use -target. I won't give GCC the same treatment since it lacks the nice -target support. --=20 Regards, Bryan Drewery From owner-freebsd-arch@freebsd.org Fri May 13 22:34:38 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31CC5B3A197 for ; Fri, 13 May 2016 22:34:38 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1FAE61EEC for ; Fri, 13 May 2016 22:34:38 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mailman.ysv.freebsd.org (Postfix) id 1A9E0B3A195; Fri, 13 May 2016 22:34:38 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A289B3A194; Fri, 13 May 2016 22:34:38 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DA77E1EEB; Fri, 13 May 2016 22:34:37 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mx1.stack.nl (Postfix) with ESMTP id 82C52358C5C; Sat, 14 May 2016 00:34:34 +0200 (CEST) Received: by toad2.stack.nl (Postfix, from userid 1677) id E9300892FF; Sat, 14 May 2016 00:34:34 +0200 (CEST) Date: Sat, 14 May 2016 00:34:34 +0200 From: Jilles Tjoelker To: Konstantin Belousov Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160513223434.GA47987@stack.nl> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> <20160513153716.GA30576@stack.nl> <20160513201916.GF89104@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160513201916.GF89104@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2016 22:34:38 -0000 On Fri, May 13, 2016 at 11:19:16PM +0300, Konstantin Belousov wrote: > On Fri, May 13, 2016 at 05:37:16PM +0200, Jilles Tjoelker wrote: > > On Mon, May 09, 2016 at 05:51:07AM +0300, Konstantin Belousov wrote: > > The mmap() page in POSIX.1-2008tc1 XSH 3 has: > > ] The state of synchronization objects such as mutexes, semaphores, > > ] barriers, and conditional variables placed in shared memory mapped > > ] with MAP_SHARED becomes undefined when the last region in any process > > ] containing the synchronization object is unmapped. > > This is new in issue 7 (SUSv4): > > ] Austin Group Interpretations 1003.1-2001 #078 and #079 are applied, > > ] clarifying page alignment requirements and adding a note about the > > ] state of synchronization objects becoming undefined when a shared > > ] region is unmapped. > Very interesting, thanks. > BTW, is there a chance of Austin Group get notified of, and possibly > adopting, MAP_EXCL flag ? You could send a message to the mailing list. MAP_EXCL is not used in the FreeBSD base but Debian code search shows Chromium and Hurd ftpfs mentioning it in comments. > > > Current updates to the patch > > > https://kib.kiev.ua/kib/pshared/robust.4.patch > Do you have any further notes, or do you want to give the patch more time ? > If not, are you fine with 'Reviewed by' attribution ? I don't trust the if (umtx_shm_vnobj_persistent) in sys/vm/vnode_pager.c: + if (umtx_shm_vnobj_persistent) + umtx_shm_object_terminated(obj); If umtx_shm_vnobj_persistent is turned off via sysctl between the check in sys/vm/vm_object.c and this one, don't we have a leak? 'Reviewed by: jilles' is fine otherwise. -- Jilles Tjoelker From owner-freebsd-arch@freebsd.org Sat May 14 10:10:50 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 881CEB3A893 for ; Sat, 14 May 2016 10:10:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7152B1B7E for ; Sat, 14 May 2016 10:10:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 6C733B3A891; Sat, 14 May 2016 10:10:50 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6BDF0B3A88F; Sat, 14 May 2016 10:10:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E25271B7A; Sat, 14 May 2016 10:10:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u4EAAiVJ060181 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 14 May 2016 13:10:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u4EAAiVJ060181 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u4EAAiVg060178; Sat, 14 May 2016 13:10:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 14 May 2016 13:10:44 +0300 From: Konstantin Belousov To: Jilles Tjoelker Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: Robust mutexes implementation Message-ID: <20160514101044.GM89104@kib.kiev.ua> References: <20160505131029.GE2422@kib.kiev.ua> <20160506233011.GA99994@stack.nl> <20160507165956.GC89104@kib.kiev.ua> <20160508125222.GA48862@stack.nl> <20160509025107.GN89104@kib.kiev.ua> <20160513153716.GA30576@stack.nl> <20160513201916.GF89104@kib.kiev.ua> <20160513223434.GA47987@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160513223434.GA47987@stack.nl> User-Agent: Mutt/1.6.1 (2016-04-27) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 May 2016 10:10:50 -0000 On Sat, May 14, 2016 at 12:34:34AM +0200, Jilles Tjoelker wrote: > On Fri, May 13, 2016 at 11:19:16PM +0300, Konstantin Belousov wrote: > > BTW, is there a chance of Austin Group get notified of, and possibly > > adopting, MAP_EXCL flag ? > > You could send a message to the mailing list. MAP_EXCL is not used in > the FreeBSD base but Debian code search shows Chromium and Hurd ftpfs > mentioning it in comments. It is not clear if they reference FreeBSD flag, or just come to this semi-obvious idea independently. Anway, it is not used because feature is not present anywhere (except us). I do not want to present this on ag list as a stranger with bad language. > I don't trust the if (umtx_shm_vnobj_persistent) in > sys/vm/vnode_pager.c: > > + if (umtx_shm_vnobj_persistent) > + umtx_shm_object_terminated(obj); > > If umtx_shm_vnobj_persistent is turned off via sysctl between the check > in sys/vm/vm_object.c and this one, don't we have a leak? Removed the test in the vnode_destroy_vobject(), doing the umtx termination unconditionally on reclaim. > > 'Reviewed by: jilles' is fine otherwise. Updated patch is at https://www.kib.kiev.ua/kib/pshared/robust.5.patch P.S. I will document UMUTEX_ROBUST/UMUTEX_RB_* and UMTX_OP_ROBUST_LISTS after the feature is committed.