Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 19 Jul 2012 14:23:26 -0700
From:      Alfred Perlstein <alfred@freebsd.org>
To:        John Marino <freebsdml@marino.st>
Cc:        freebsd-threads@freebsd.org
Subject:   Re: Fingerpointing about broken Ada tasking starting with FreeBSD 9.0 threading
Message-ID:  <20120719212326.GN98608@elvis.mu.org>
In-Reply-To: <500854EC.3040305@marino.st>
References:  <500854EC.3040305@marino.st>

next in thread | previous in thread | raw e-mail | index | archive | help
Hey John,

I find the best way to figure stuff like this out would be to
instrument the code.

I think what could happen here is simply adding a FILE,LINE to the struct
thread and have THR_CRITICAL_ENTER record the last place it was called
by stuffing the current __FILE__ and __LINE__ into those variables.

Then when you hit that assertion you can dump the last place.  

The only problem you could face with such a system is a false positive
if the code goes multiple levels deep, you'll probably want to clear
the data there when you see a THR_CRITICAL_LEAVE.

Then if in your assertion you see that it's clear/NULL then you want to
probably implement a static stack and use (thrd)->critical_count and
(thrd)->locklevel as indecies to respective traceback stacks.

It really shouldn't take more than a few hours to write the instrumentation
code and I could see it staying inside the code under a PTHREAD_HEAVY_DEBUG
flag if needed.

-Alfred

* John Marino <freebsdml@marino.st> [120719 11:42] wrote:
> Hi guys, it's been a while.
> 
> Before FreeBSD 9.0 was released, we made sure lang/gnat-aux built on it, 
> and it did.  It wasn't until after it was released that we realized that 
> the Ada compiler GNAT tasking was broken.  lang/gnat-aux is based on 
> gcc-4.6.
> 
> I brought this to Adacore's attention as they maintain the Ada compiler 
> in GCC.  They blew it off and said, "try gcc 4.7 first".  It was true 
> that a lot of tasking code had changed, but I ported some of it to 
> gnat-aux and it didn't fix a thing, so I wasn't too hopeful.
> 
> I finally got around to building gcc-4.7 and sure enough, it's tasking 
> is just as broken.  The pending port lang/gcc-aux is here:
> http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/169951
> 
> The problem appears to stem from this commit:
> "Convert thread list lock from mutex to rwlock." by davidxu
> 
> https://github.com/freebsd/freebsd/commit/c741b41fb2a88f630e77ada2888443e3fea358c9
> 
> When an Ada task exits (not sure if it's all tasks or just some of them, 
> but it seems to be all), it aborts with the message "thread exits with 
> resources held!"
> 
> I brought this back to Adacore and said that message was caused by 
> either a locklevel > 0 or a critical_count > 0.  At that point, they 
> said they needed a lot more convincing that it was a posix-wide problem 
> that FreeBSD just happened to detect. (it turns out that critical_count 
> is the culprit by the way.)
> 
> The DragonFly BSD threading library has the same locklevel and 
> critical_count thread properties, but putting the same THR_IN_CRITICAL 
> panic in that library did NOT result in thread panics on DragonFly.  All 
> the Ada task tests passed completely.  That strengthens Adacore's 
> position that FreeBSD is "fishy" and coming up with false positives. 
> NetBSD's thread library is too different to try to check there.
> 
> If I remove
> >#if defined(_PTHREADS_INVARIANTS)
> >if (THR_IN_CRITICAL(curthread))
> >   PANIC("thread exits with resources held!");	
> >#endif
> then tasking tests pass on FreeBSD as well.  Unfortunately there doesn't 
> seem to be any way to disable the check without modifying and rebuilding 
> the libpthread
> 
> So what can I do?
> My first blame was aimed towards GCC and GNAT, but now I'm not so sure. 
>  And if, in the worst case, this is actually a FreeBSD threading bug, 
> how can it be worked around for FreeBSD 9?
> 
> I haven't been very successful using GDB to troubleshoot this thing.
> 
> Regards,
> John
> 
> _______________________________________________
> freebsd-threads@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-threads
> To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org"

-- 
- Alfred Perlstein
.- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10
.- FreeBSD committer



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120719212326.GN98608>