From owner-freebsd-threads@FreeBSD.ORG Thu Jul 19 21:23:32 2012 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58C04106566C for ; Thu, 19 Jul 2012 21:23:32 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 30EFA8FC14 for ; Thu, 19 Jul 2012 21:23:32 +0000 (UTC) Received: by elvis.mu.org (Postfix, from userid 1192) id 5F6D61A3C56; Thu, 19 Jul 2012 14:23:26 -0700 (PDT) Date: Thu, 19 Jul 2012 14:23:26 -0700 From: Alfred Perlstein To: John Marino Message-ID: <20120719212326.GN98608@elvis.mu.org> References: <500854EC.3040305@marino.st> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <500854EC.3040305@marino.st> User-Agent: Mutt/1.4.2.3i Cc: freebsd-threads@freebsd.org Subject: Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2012 21:23:32 -0000 Hey John, I find the best way to figure stuff like this out would be to instrument the code. I think what could happen here is simply adding a FILE,LINE to the struct thread and have THR_CRITICAL_ENTER record the last place it was called by stuffing the current __FILE__ and __LINE__ into those variables. Then when you hit that assertion you can dump the last place. The only problem you could face with such a system is a false positive if the code goes multiple levels deep, you'll probably want to clear the data there when you see a THR_CRITICAL_LEAVE. Then if in your assertion you see that it's clear/NULL then you want to probably implement a static stack and use (thrd)->critical_count and (thrd)->locklevel as indecies to respective traceback stacks. It really shouldn't take more than a few hours to write the instrumentation code and I could see it staying inside the code under a PTHREAD_HEAVY_DEBUG flag if needed. -Alfred * John Marino [120719 11:42] wrote: > Hi guys, it's been a while. > > Before FreeBSD 9.0 was released, we made sure lang/gnat-aux built on it, > and it did. It wasn't until after it was released that we realized that > the Ada compiler GNAT tasking was broken. lang/gnat-aux is based on > gcc-4.6. > > I brought this to Adacore's attention as they maintain the Ada compiler > in GCC. They blew it off and said, "try gcc 4.7 first". It was true > that a lot of tasking code had changed, but I ported some of it to > gnat-aux and it didn't fix a thing, so I wasn't too hopeful. > > I finally got around to building gcc-4.7 and sure enough, it's tasking > is just as broken. The pending port lang/gcc-aux is here: > http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/169951 > > The problem appears to stem from this commit: > "Convert thread list lock from mutex to rwlock." by davidxu > > https://github.com/freebsd/freebsd/commit/c741b41fb2a88f630e77ada2888443e3fea358c9 > > When an Ada task exits (not sure if it's all tasks or just some of them, > but it seems to be all), it aborts with the message "thread exits with > resources held!" > > I brought this back to Adacore and said that message was caused by > either a locklevel > 0 or a critical_count > 0. At that point, they > said they needed a lot more convincing that it was a posix-wide problem > that FreeBSD just happened to detect. (it turns out that critical_count > is the culprit by the way.) > > The DragonFly BSD threading library has the same locklevel and > critical_count thread properties, but putting the same THR_IN_CRITICAL > panic in that library did NOT result in thread panics on DragonFly. All > the Ada task tests passed completely. That strengthens Adacore's > position that FreeBSD is "fishy" and coming up with false positives. > NetBSD's thread library is too different to try to check there. > > If I remove > >#if defined(_PTHREADS_INVARIANTS) > >if (THR_IN_CRITICAL(curthread)) > > PANIC("thread exits with resources held!"); > >#endif > then tasking tests pass on FreeBSD as well. Unfortunately there doesn't > seem to be any way to disable the check without modifying and rebuilding > the libpthread > > So what can I do? > My first blame was aimed towards GCC and GNAT, but now I'm not so sure. > And if, in the worst case, this is actually a FreeBSD threading bug, > how can it be worked around for FreeBSD 9? > > I haven't been very successful using GDB to troubleshoot this thing. > > Regards, > John > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" -- - Alfred Perlstein .- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10 .- FreeBSD committer