From owner-freebsd-threads@FreeBSD.ORG Mon Jul 16 11:09:34 2012 Return-Path: Delivered-To: freebsd-threads@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 10F9A106566C for ; Mon, 16 Jul 2012 11:09:34 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D53C88FC21 for ; Mon, 16 Jul 2012 11:09:33 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q6GB9XXZ094171 for ; Mon, 16 Jul 2012 11:09:33 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q6GB9WVq094167 for freebsd-threads@FreeBSD.org; Mon, 16 Jul 2012 11:09:32 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 Jul 2012 11:09:32 GMT Message-Id: <201207161109.q6GB9WVq094167@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jul 2012 11:09:34 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/168417 threads pthread_getcpuclockid() does not work to specification o threa/165173 threads [build] clang buildworld breaks libthr o threa/163512 threads libc defaults to single threaded o threa/160708 threads possible security problem with RLIMIT_VMEM o threa/150959 threads [libc] Stub pthread_once in libc should call _libc_onc o threa/148515 threads Memory / syslog strangeness in FreeBSD 8.x ( possible o threa/141721 threads rtprio(1): (id|rt)prio priority resets when new thread o threa/135673 threads databases/mysql50-server - MySQL query lock-ups on 7.2 o threa/128922 threads threads hang with xorg running o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/30464 threads [patch] pthread mutex attributes -- pshared 21 problems total. From owner-freebsd-threads@FreeBSD.ORG Thu Jul 19 09:34:57 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AD32C1065670; Thu, 19 Jul 2012 09:34:57 +0000 (UTC) (envelope-from listlog2011@gmail.com) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 971238FC0A; Thu, 19 Jul 2012 09:34:57 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q6J9Ytan017829; Thu, 19 Jul 2012 09:34:56 GMT (envelope-from listlog2011@gmail.com) Message-ID: <5007D4BD.6030402@gmail.com> Date: Thu, 19 Jul 2012 17:34:53 +0800 From: David Xu User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: David Xu References: <201207120800.q6C80ISC073892@freefall.freebsd.org> In-Reply-To: <201207120800.q6C80ISC073892@freefall.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: threads/168417: pthread_getcpuclockid() does not work to specification X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: davidxu@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 09:34:57 -0000 On 2012/7/12 16:00, David Xu wrote: > The following reply was made to PR threads/168417; it has been noted by GNATS. > > From: David Xu > To: bug-followup@freebsd.org, chris.hall@highwayman.com > Cc: > Subject: Re: threads/168417: pthread_getcpuclockid() does not work to specification > Date: Thu, 12 Jul 2012 15:51:02 +0800 > > I have worked out a patch trying to fix the problem: > http://people.freebsd.org/~davidxu/patch/cputime_clockid.diff > I have updated the patch, if no objection, I would commit it soon. http://people.freebsd.org/~davidxu/patch/cputime_clockid3.diff Regards, David Xu From owner-freebsd-threads@FreeBSD.ORG Thu Jul 19 09:39:32 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C409D1065680; Thu, 19 Jul 2012 09:39:32 +0000 (UTC) (envelope-from listlog2011@gmail.com) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 329B38FC25; Thu, 19 Jul 2012 09:39:32 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q6J9dTjH018402; Thu, 19 Jul 2012 09:39:30 GMT (envelope-from listlog2011@gmail.com) Message-ID: <5007D5D0.8060007@gmail.com> Date: Thu, 19 Jul 2012 17:39:28 +0800 From: David Xu User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: davidxu@freebsd.org References: <201207120800.q6C80ISC073892@freefall.freebsd.org> <5007D4BD.6030402@gmail.com> In-Reply-To: <5007D4BD.6030402@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: threads/168417: pthread_getcpuclockid() does not work to specification X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: davidxu@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 09:39:32 -0000 On 2012/7/19 17:34, David Xu wrote: > On 2012/7/12 16:00, David Xu wrote: >> The following reply was made to PR threads/168417; it has been noted >> by GNATS. >> >> From: David Xu >> To: bug-followup@freebsd.org, chris.hall@highwayman.com >> Cc: >> Subject: Re: threads/168417: pthread_getcpuclockid() does not work to >> specification >> Date: Thu, 12 Jul 2012 15:51:02 +0800 >> >> I have worked out a patch trying to fix the problem: >> http://people.freebsd.org/~davidxu/patch/cputime_clockid.diff > > I have updated the patch, if no objection, I would commit it soon. > > http://people.freebsd.org/~davidxu/patch/cputime_clockid3.diff I still only allow getting thread cpu time in same process, note that it does not support creating timer via timer_create(), it is just a counter. From owner-freebsd-threads@FreeBSD.ORG Thu Jul 19 18:41:58 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BE97F106566C for ; Thu, 19 Jul 2012 18:41:58 +0000 (UTC) (envelope-from freebsdml@marino.st) Received: from shepard.synsport.net (mail.synsport.com [208.69.230.148]) by mx1.freebsd.org (Postfix) with ESMTP id 96F2D8FC12 for ; Thu, 19 Jul 2012 18:41:58 +0000 (UTC) Received: from [192.168.0.225] (atoulouse-256-1-140-121.w90-45.abo.wanadoo.fr [90.45.187.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shepard.synsport.net (Postfix) with ESMTP id 5C28043B97 for ; Thu, 19 Jul 2012 13:41:57 -0500 (CDT) Message-ID: <500854EC.3040305@marino.st> Date: Thu, 19 Jul 2012 20:41:48 +0200 From: John Marino User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 18:41:58 -0000 Hi guys, it's been a while. Before FreeBSD 9.0 was released, we made sure lang/gnat-aux built on it, and it did. It wasn't until after it was released that we realized that the Ada compiler GNAT tasking was broken. lang/gnat-aux is based on gcc-4.6. I brought this to Adacore's attention as they maintain the Ada compiler in GCC. They blew it off and said, "try gcc 4.7 first". It was true that a lot of tasking code had changed, but I ported some of it to gnat-aux and it didn't fix a thing, so I wasn't too hopeful. I finally got around to building gcc-4.7 and sure enough, it's tasking is just as broken. The pending port lang/gcc-aux is here: http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/169951 The problem appears to stem from this commit: "Convert thread list lock from mutex to rwlock." by davidxu https://github.com/freebsd/freebsd/commit/c741b41fb2a88f630e77ada2888443e3fea358c9 When an Ada task exits (not sure if it's all tasks or just some of them, but it seems to be all), it aborts with the message "thread exits with resources held!" I brought this back to Adacore and said that message was caused by either a locklevel > 0 or a critical_count > 0. At that point, they said they needed a lot more convincing that it was a posix-wide problem that FreeBSD just happened to detect. (it turns out that critical_count is the culprit by the way.) The DragonFly BSD threading library has the same locklevel and critical_count thread properties, but putting the same THR_IN_CRITICAL panic in that library did NOT result in thread panics on DragonFly. All the Ada task tests passed completely. That strengthens Adacore's position that FreeBSD is "fishy" and coming up with false positives. NetBSD's thread library is too different to try to check there. If I remove > #if defined(_PTHREADS_INVARIANTS) > if (THR_IN_CRITICAL(curthread)) > PANIC("thread exits with resources held!"); > #endif then tasking tests pass on FreeBSD as well. Unfortunately there doesn't seem to be any way to disable the check without modifying and rebuilding the libpthread So what can I do? My first blame was aimed towards GCC and GNAT, but now I'm not so sure. And if, in the worst case, this is actually a FreeBSD threading bug, how can it be worked around for FreeBSD 9? I haven't been very successful using GDB to troubleshoot this thing. Regards, John From owner-freebsd-threads@FreeBSD.ORG Thu Jul 19 21:23:32 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58C04106566C for ; Thu, 19 Jul 2012 21:23:32 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 30EFA8FC14 for ; Thu, 19 Jul 2012 21:23:32 +0000 (UTC) Received: by elvis.mu.org (Postfix, from userid 1192) id 5F6D61A3C56; Thu, 19 Jul 2012 14:23:26 -0700 (PDT) Date: Thu, 19 Jul 2012 14:23:26 -0700 From: Alfred Perlstein To: John Marino Message-ID: <20120719212326.GN98608@elvis.mu.org> References: <500854EC.3040305@marino.st> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <500854EC.3040305@marino.st> User-Agent: Mutt/1.4.2.3i Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 21:23:32 -0000 Hey John, I find the best way to figure stuff like this out would be to instrument the code. I think what could happen here is simply adding a FILE,LINE to the struct thread and have THR_CRITICAL_ENTER record the last place it was called by stuffing the current __FILE__ and __LINE__ into those variables. Then when you hit that assertion you can dump the last place. The only problem you could face with such a system is a false positive if the code goes multiple levels deep, you'll probably want to clear the data there when you see a THR_CRITICAL_LEAVE. Then if in your assertion you see that it's clear/NULL then you want to probably implement a static stack and use (thrd)->critical_count and (thrd)->locklevel as indecies to respective traceback stacks. It really shouldn't take more than a few hours to write the instrumentation code and I could see it staying inside the code under a PTHREAD_HEAVY_DEBUG flag if needed. -Alfred * John Marino [120719 11:42] wrote: > Hi guys, it's been a while. > > Before FreeBSD 9.0 was released, we made sure lang/gnat-aux built on it, > and it did. It wasn't until after it was released that we realized that > the Ada compiler GNAT tasking was broken. lang/gnat-aux is based on > gcc-4.6. > > I brought this to Adacore's attention as they maintain the Ada compiler > in GCC. They blew it off and said, "try gcc 4.7 first". It was true > that a lot of tasking code had changed, but I ported some of it to > gnat-aux and it didn't fix a thing, so I wasn't too hopeful. > > I finally got around to building gcc-4.7 and sure enough, it's tasking > is just as broken. The pending port lang/gcc-aux is here: > http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/169951 > > The problem appears to stem from this commit: > "Convert thread list lock from mutex to rwlock." by davidxu > > https://github.com/freebsd/freebsd/commit/c741b41fb2a88f630e77ada2888443e3fea358c9 > > When an Ada task exits (not sure if it's all tasks or just some of them, > but it seems to be all), it aborts with the message "thread exits with > resources held!" > > I brought this back to Adacore and said that message was caused by > either a locklevel > 0 or a critical_count > 0. At that point, they > said they needed a lot more convincing that it was a posix-wide problem > that FreeBSD just happened to detect. (it turns out that critical_count > is the culprit by the way.) > > The DragonFly BSD threading library has the same locklevel and > critical_count thread properties, but putting the same THR_IN_CRITICAL > panic in that library did NOT result in thread panics on DragonFly. All > the Ada task tests passed completely. That strengthens Adacore's > position that FreeBSD is "fishy" and coming up with false positives. > NetBSD's thread library is too different to try to check there. > > If I remove > >#if defined(_PTHREADS_INVARIANTS) > >if (THR_IN_CRITICAL(curthread)) > > PANIC("thread exits with resources held!"); > >#endif > then tasking tests pass on FreeBSD as well. Unfortunately there doesn't > seem to be any way to disable the check without modifying and rebuilding > the libpthread > > So what can I do? > My first blame was aimed towards GCC and GNAT, but now I'm not so sure. > And if, in the worst case, this is actually a FreeBSD threading bug, > how can it be worked around for FreeBSD 9? > > I haven't been very successful using GDB to troubleshoot this thing. > > Regards, > John > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" -- - Alfred Perlstein .- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10 .- FreeBSD committer From owner-freebsd-threads@FreeBSD.ORG Fri Jul 20 07:59:00 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7F028106564A; Fri, 20 Jul 2012 07:59:00 +0000 (UTC) (envelope-from freebsdml@marino.st) Received: from shepard.synsport.net (mail.synsport.com [208.69.230.148]) by mx1.freebsd.org (Postfix) with ESMTP id 501D58FC08; Fri, 20 Jul 2012 07:59:00 +0000 (UTC) Received: from [192.168.0.225] (atoulouse-256-1-140-121.w90-45.abo.wanadoo.fr [90.45.187.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shepard.synsport.net (Postfix) with ESMTP id 9C69043B91; Fri, 20 Jul 2012 02:58:58 -0500 (CDT) Message-ID: <50090FB9.6050606@marino.st> Date: Fri, 20 Jul 2012 09:58:49 +0200 From: John Marino User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: Alfred Perlstein References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> In-Reply-To: <20120719212326.GN98608@elvis.mu.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 07:59:00 -0000 On 7/19/2012 23:23, Alfred Perlstein wrote: > Hey John, > > I find the best way to figure stuff like this out would be to > instrument the code. > > I think what could happen here is simply adding a FILE,LINE to the struct > thread and have THR_CRITICAL_ENTER record the last place it was called > by stuffing the current __FILE__ and __LINE__ into those variables. > > Then when you hit that assertion you can dump the last place. > > The only problem you could face with such a system is a false positive > if the code goes multiple levels deep, you'll probably want to clear > the data there when you see a THR_CRITICAL_LEAVE. > > Then if in your assertion you see that it's clear/NULL then you want to > probably implement a static stack and use (thrd)->critical_count and > (thrd)->locklevel as indecies to respective traceback stacks. > > It really shouldn't take more than a few hours to write the instrumentation > code and I could see it staying inside the code under a PTHREAD_HEAVY_DEBUG > flag if needed. Hi Alfred, Thanks for providing some techniques that can perhaps help track down what's going on. I'm still interested in the big picture, though. We've got a package that runs on FreeBSD 6, 7, 8 and broke on 9. Similarly the "critical_count" property is at the expected 0 value on thread exit on DragonFly. The new thread panic caused a regression - Was it necessary to put this panic there? What are the consequences of continuing? Were these resources being "held" before when a mutex was used and just not detected? (which implies consequences are not high so why panic?) Has there been other fallout from this change? I'm guessing your first inclination would be to blame GNAT, and say that if the crit count is wrong, something must not be getting cleaned up and you may be right, but the fact remains that software that builds and runs on FreeBSD 6, 7, and 8 doesn't run on 9. I assume that was unintended. John From owner-freebsd-threads@FreeBSD.ORG Fri Jul 20 15:08:03 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A7EE8106566B for ; Fri, 20 Jul 2012 15:08:03 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 9145C8FC0C for ; Fri, 20 Jul 2012 15:08:03 +0000 (UTC) Received: by elvis.mu.org (Postfix, from userid 1192) id 369131A3C7F; Fri, 20 Jul 2012 08:08:03 -0700 (PDT) Date: Fri, 20 Jul 2012 08:08:03 -0700 From: Alfred Perlstein To: John Marino Message-ID: <20120720150803.GS98608@elvis.mu.org> References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> <50090FB9.6050606@marino.st> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50090FB9.6050606@marino.st> User-Agent: Mutt/1.4.2.3i Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 15:08:03 -0000 * John Marino [120720 00:59] wrote: > On 7/19/2012 23:23, Alfred Perlstein wrote: > >Hey John, > > > >I find the best way to figure stuff like this out would be to > >instrument the code. > > > >I think what could happen here is simply adding a FILE,LINE to the struct > >thread and have THR_CRITICAL_ENTER record the last place it was called > >by stuffing the current __FILE__ and __LINE__ into those variables. > > > >Then when you hit that assertion you can dump the last place. > > > >The only problem you could face with such a system is a false positive > >if the code goes multiple levels deep, you'll probably want to clear > >the data there when you see a THR_CRITICAL_LEAVE. > > > >Then if in your assertion you see that it's clear/NULL then you want to > >probably implement a static stack and use (thrd)->critical_count and > >(thrd)->locklevel as indecies to respective traceback stacks. > > > >It really shouldn't take more than a few hours to write the instrumentation > >code and I could see it staying inside the code under a PTHREAD_HEAVY_DEBUG > >flag if needed. > > Hi Alfred, > Thanks for providing some techniques that can perhaps help track down > what's going on. > > I'm still interested in the big picture, though. There is no big picture unless you take the time to diagnose what is happening. There is a bug somewhere. Talking about "big picture" doesn't mean anything. The bug could be due to any of the reasons you described, or due to other reasons. What needs to be done is some investigation into what is triggering the bug and then determine if it's a bug, false positive, corruption or something else. > We've got a package that runs on FreeBSD 6, 7, 8 and broke on 9. > Similarly the "critical_count" property is at the expected 0 value on > thread exit on DragonFly. > > The new thread panic caused a regression - > Was it necessary to put this panic there? > What are the consequences of continuing? > Were these resources being "held" before when a mutex was used and just > not detected? (which implies consequences are not high so why panic?) > Has there been other fallout from this change? > > I'm guessing your first inclination would be to blame GNAT, and say that > if the crit count is wrong, something must not be getting cleaned up and > you may be right, but the fact remains that software that builds and > runs on FreeBSD 6, 7, and 8 doesn't run on 9. I assume that was unintended. > > John > It sounds like you're advocating for just removing an assertion without proving it's a false positive. I don't think that will work out unfortunately. -- - Alfred Perlstein .- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10 .- FreeBSD committer From owner-freebsd-threads@FreeBSD.ORG Fri Jul 20 16:29:01 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BBE37106566C; Fri, 20 Jul 2012 16:29:01 +0000 (UTC) (envelope-from freebsdml@marino.st) Received: from shepard.synsport.net (mail.synsport.com [208.69.230.148]) by mx1.freebsd.org (Postfix) with ESMTP id 873B18FC1A; Fri, 20 Jul 2012 16:29:01 +0000 (UTC) Received: from [192.168.0.225] (atoulouse-256-1-140-121.w90-45.abo.wanadoo.fr [90.45.187.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shepard.synsport.net (Postfix) with ESMTP id B323D43B91; Fri, 20 Jul 2012 11:28:53 -0500 (CDT) Message-ID: <5009873C.4080709@marino.st> Date: Fri, 20 Jul 2012 18:28:44 +0200 From: John Marino User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: Alfred Perlstein References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> <50090FB9.6050606@marino.st> <20120720150803.GS98608@elvis.mu.org> In-Reply-To: <20120720150803.GS98608@elvis.mu.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 16:29:01 -0000 On 7/20/2012 17:08, Alfred Perlstein wrote: > > There is no big picture unless you take the time to diagnose what > is happening. There is a bug somewhere. Talking about "big > picture" doesn't mean anything. > > The bug could be due to any of the reasons you described, or due > to other reasons. What needs to be done is some investigation into > what is triggering the bug and then determine if it's a bug, false > positive, corruption or something else. I agree, and I'll try. What I was trying to get at is *if* the problem really is with FreeBSD, there should be more than just me looking into it given the implications of what that would mean. The worst case scenario is that this is 100% a FreeBSD problem, and that would have some fallout. In my previous work, we had to map out every possibility and plan for it -- we didn't have the luxury of determining the fault and then coming up with a plan. That mentality is carrying over probably. > > It sounds like you're advocating for just removing an assertion without > proving it's a false positive. I don't think that will work out unfortunately. > Not at all, but there was a serious consequence of this assertion and it's not a pre-production assertion either. I don't have any preconceived notion of the cause of the fault nor the solution. My mind is wide open at this point. John From owner-freebsd-threads@FreeBSD.ORG Fri Jul 20 16:32:13 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 997C51065670; Fri, 20 Jul 2012 16:32:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 0C86E8FC1A; Fri, 20 Jul 2012 16:32:12 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q6KGWIow026379; Fri, 20 Jul 2012 19:32:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q6KGW5W7019710; Fri, 20 Jul 2012 19:32:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q6KGW5mW019709; Fri, 20 Jul 2012 19:32:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 20 Jul 2012 19:32:05 +0300 From: Konstantin Belousov To: John Marino Message-ID: <20120720163205.GR2676@deviant.kiev.zoral.com.ua> References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> <50090FB9.6050606@marino.st> <20120720150803.GS98608@elvis.mu.org> <5009873C.4080709@marino.st> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SUcp5/vPb8ZOOFiR" Content-Disposition: inline In-Reply-To: <5009873C.4080709@marino.st> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 16:32:13 -0000 --SUcp5/vPb8ZOOFiR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 20, 2012 at 06:28:44PM +0200, John Marino wrote: > On 7/20/2012 17:08, Alfred Perlstein wrote: > > > >There is no big picture unless you take the time to diagnose what > >is happening. There is a bug somewhere. Talking about "big > >picture" doesn't mean anything. > > > >The bug could be due to any of the reasons you described, or due > >to other reasons. What needs to be done is some investigation into > >what is triggering the bug and then determine if it's a bug, false > >positive, corruption or something else. >=20 > I agree, and I'll try. What I was trying to get at is *if* the problem= =20 > really is with FreeBSD, there should be more than just me looking into=20 > it given the implications of what that would mean. The worst case=20 > scenario is that this is 100% a FreeBSD problem, and that would have=20 > some fallout. In my previous work, we had to map out every possibility= =20 > and plan for it -- we didn't have the luxury of determining the fault=20 > and then coming up with a plan. That mentality is carrying over probably. >=20 > > > >It sounds like you're advocating for just removing an assertion without > >proving it's a false positive. I don't think that will work out=20 > >unfortunately. > > >=20 > Not at all, but there was a serious consequence of this assertion and=20 > it's not a pre-production assertion either. I don't have any=20 > preconceived notion of the cause of the fault nor the solution. My mind= =20 > is wide open at this point. FYI, this problem is supposedly fixed by r238637, as much as I can judge based on your description and commit itself. --SUcp5/vPb8ZOOFiR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlAJiAUACgkQC3+MBN1Mb4j97QCg2nTw6N4wxx2M586I/+uvN31Y QGQAn3pHIIkcDTQEiTVx3HeeLHcgVxwo =3aun -----END PGP SIGNATURE----- --SUcp5/vPb8ZOOFiR-- From owner-freebsd-threads@FreeBSD.ORG Fri Jul 20 16:39:19 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84F97106566C; Fri, 20 Jul 2012 16:39:19 +0000 (UTC) (envelope-from freebsdml@marino.st) Received: from shepard.synsport.net (mail.synsport.com [208.69.230.148]) by mx1.freebsd.org (Postfix) with ESMTP id 58D2F8FC0A; Fri, 20 Jul 2012 16:39:19 +0000 (UTC) Received: from [192.168.0.225] (atoulouse-256-1-140-121.w90-45.abo.wanadoo.fr [90.45.187.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shepard.synsport.net (Postfix) with ESMTP id 1591843B91; Fri, 20 Jul 2012 11:39:17 -0500 (CDT) Message-ID: <500989AC.7030200@marino.st> Date: Fri, 20 Jul 2012 18:39:08 +0200 From: John Marino User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: Konstantin Belousov References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> <50090FB9.6050606@marino.st> <20120720150803.GS98608@elvis.mu.org> <5009873C.4080709@marino.st> <20120720163205.GR2676@deviant.kiev.zoral.com.ua> In-Reply-To: <20120720163205.GR2676@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2012 16:39:19 -0000 On 7/20/2012 18:32, Konstantin Belousov wrote: > > FYI, this problem is supposedly fixed by r238637, as much as I can judge > based on your description and commit itself. That commit is only 15 hours old! Coincidence? Okay, should I try patching my FreeBSD 9.0 thread library with that commit and see if that clears up the issue? John From owner-freebsd-threads@FreeBSD.ORG Sat Jul 21 00:22:01 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2135C106564A; Sat, 21 Jul 2012 00:22:01 +0000 (UTC) (envelope-from freebsdml@marino.st) Received: from shepard.synsport.net (mail.synsport.com [208.69.230.148]) by mx1.freebsd.org (Postfix) with ESMTP id 215F68FC12; Sat, 21 Jul 2012 00:22:00 +0000 (UTC) Received: from [192.168.0.225] (atoulouse-256-1-140-121.w90-45.abo.wanadoo.fr [90.45.187.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shepard.synsport.net (Postfix) with ESMTP id A690A43B4A; Fri, 20 Jul 2012 19:21:58 -0500 (CDT) Message-ID: <5009F61D.4010203@marino.st> Date: Sat, 21 Jul 2012 02:21:49 +0200 From: John Marino User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: Konstantin Belousov References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> <50090FB9.6050606@marino.st> <20120720150803.GS98608@elvis.mu.org> <5009873C.4080709@marino.st> <20120720163205.GR2676@deviant.kiev.zoral.com.ua> In-Reply-To: <20120720163205.GR2676@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jul 2012 00:22:01 -0000 On 7/20/2012 18:32, Konstantin Belousov wrote: > > FYI, this problem is supposedly fixed by r238637, as much as I can judge > based on your description and commit itself. Hi Konstantin, I went ahead and patched the FreeBSD 9.0 thread library with r238637 and reran the Ada testsuite. Add the tasking tests that failed before now pass. It appears that is indeed the fix that is needed! In practical terms, it means people will either have to patch and rebuild the library, or upgrade to FreeBSD 9.1. Will this revision make it into 9.1? it's seems to be in beta already. John From owner-freebsd-threads@FreeBSD.ORG Sat Jul 21 00:32:18 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A73B6106566C; Sat, 21 Jul 2012 00:32:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 215AD8FC0C; Sat, 21 Jul 2012 00:32:17 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q6L0WQVD059057; Sat, 21 Jul 2012 03:32:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q6L0WDPA021739; Sat, 21 Jul 2012 03:32:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q6L0WDrM021738; Sat, 21 Jul 2012 03:32:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 21 Jul 2012 03:32:13 +0300 From: Konstantin Belousov To: John Marino Message-ID: <20120721003213.GU2676@deviant.kiev.zoral.com.ua> References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> <50090FB9.6050606@marino.st> <20120720150803.GS98608@elvis.mu.org> <5009873C.4080709@marino.st> <20120720163205.GR2676@deviant.kiev.zoral.com.ua> <5009F61D.4010203@marino.st> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="V6UBx4j9pGkVmsf8" Content-Disposition: inline In-Reply-To: <5009F61D.4010203@marino.st> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jul 2012 00:32:18 -0000 --V6UBx4j9pGkVmsf8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jul 21, 2012 at 02:21:49AM +0200, John Marino wrote: > On 7/20/2012 18:32, Konstantin Belousov wrote: > > > >FYI, this problem is supposedly fixed by r238637, as much as I can judge > >based on your description and commit itself. >=20 > Hi Konstantin, >=20 > I went ahead and patched the FreeBSD 9.0 thread library with r238637 and= =20 > reran the Ada testsuite. Add the tasking tests that failed before now=20 > pass. It appears that is indeed the fix that is needed! >=20 > In practical terms, it means people will either have to patch and=20 > rebuild the library, or upgrade to FreeBSD 9.1. Will this revision make= =20 > it into 9.1? it's seems to be in beta already. David specified MFC timeout for 3 days, as you can see on your own in the commit message. --V6UBx4j9pGkVmsf8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlAJ+I0ACgkQC3+MBN1Mb4jM4QCgxn4LdW9RhZykY99D+Q49zkff p70AoJ9pXpCzljsVtLLvTlAIxVaHkz8I =Uz+d -----END PGP SIGNATURE----- --V6UBx4j9pGkVmsf8-- From owner-freebsd-threads@FreeBSD.ORG Sat Jul 21 00:36:10 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A629A1065672; Sat, 21 Jul 2012 00:36:10 +0000 (UTC) (envelope-from freebsdml@marino.st) Received: from shepard.synsport.net (mail.synsport.com [208.69.230.148]) by mx1.freebsd.org (Postfix) with ESMTP id 782108FC14; Sat, 21 Jul 2012 00:36:10 +0000 (UTC) Received: from [192.168.0.225] (atoulouse-256-1-140-121.w90-45.abo.wanadoo.fr [90.45.187.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by shepard.synsport.net (Postfix) with ESMTP id DE1C743B4A; Fri, 20 Jul 2012 19:36:08 -0500 (CDT) Message-ID: <5009F970.9060504@marino.st> Date: Sat, 21 Jul 2012 02:36:00 +0200 From: John Marino User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0) Gecko/20120129 Thunderbird/10.0 MIME-Version: 1.0 To: Konstantin Belousov References: <500854EC.3040305@marino.st> <20120719212326.GN98608@elvis.mu.org> <50090FB9.6050606@marino.st> <20120720150803.GS98608@elvis.mu.org> <5009873C.4080709@marino.st> <20120720163205.GR2676@deviant.kiev.zoral.com.ua> <5009F61D.4010203@marino.st> <20120721003213.GU2676@deviant.kiev.zoral.com.ua> In-Reply-To: <20120721003213.GU2676@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jul 2012 00:36:10 -0000 On 7/21/2012 02:32, Konstantin Belousov wrote: > On Sat, Jul 21, 2012 at 02:21:49AM +0200, John Marino wrote: >> On 7/20/2012 18:32, Konstantin Belousov wrote: >>> >>> FYI, this problem is supposedly fixed by r238637, as much as I can judge >>> based on your description and commit itself. >> >> Hi Konstantin, >> >> I went ahead and patched the FreeBSD 9.0 thread library with r238637 and >> reran the Ada testsuite. Add the tasking tests that failed before now >> pass. It appears that is indeed the fix that is needed! >> >> In practical terms, it means people will either have to patch and >> rebuild the library, or upgrade to FreeBSD 9.1. Will this revision make >> it into 9.1? it's seems to be in beta already. > > David specified MFC timeout for 3 days, as you can see on your own in the > commit message. Ok thanks. I wasn't 100% sure which branch the MFC referred to.