From owner-freebsd-threads@freebsd.org  Tue Dec  6 14:48:17 2016
Return-Path: <owner-freebsd-threads@freebsd.org>
Delivered-To: freebsd-threads@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id D189BC6A9A4
 for <freebsd-threads@mailman.ysv.freebsd.org>;
 Tue,  6 Dec 2016 14:48:17 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
 (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 672DA3DB
 for <freebsd-threads@freebsd.org>; Tue,  6 Dec 2016 14:48:17 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from tom.home (kib@localhost [127.0.0.1])
 by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uB6EmCGa019284
 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO);
 Tue, 6 Dec 2016 16:48:12 +0200 (EET)
 (envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uB6EmCGa019284
Received: (from kostik@localhost)
 by tom.home (8.15.2/8.15.2/Submit) id uB6EmCrL019283;
 Tue, 6 Dec 2016 16:48:12 +0200 (EET)
 (envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
 using -f
Date: Tue, 6 Dec 2016 16:48:12 +0200
From: Konstantin Belousov <kostikbel@gmail.com>
To: Dimitri Staessens <dimitri.staessens@intec.ugent.be>
Cc: freebsd-threads@freebsd.org
Subject: Re: Unlocking a robust mutex in a cleanup handler
Message-ID: <20161206144812.GS54029@kib.kiev.ua>
References: <119e59d4-6125-f313-e6e6-67055a15d224@intec.ugent.be>
 <20161206112558.GN54029@kib.kiev.ua>
 <6a7139cd-b6db-d078-ee5e-b7c590eb13d1@intec.ugent.be>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <6a7139cd-b6db-d078-ee5e-b7c590eb13d1@intec.ugent.be>
User-Agent: Mutt/1.7.1 (2016-10-04)
X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no
 autolearn_force=no version=3.4.1
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home
X-BeenThere: freebsd-threads@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Threading on FreeBSD <freebsd-threads.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-threads>, 
 <mailto:freebsd-threads-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-threads/>
List-Post: <mailto:freebsd-threads@freebsd.org>
List-Help: <mailto:freebsd-threads-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-threads>, 
 <mailto:freebsd-threads-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Dec 2016 14:48:17 -0000

On Tue, Dec 06, 2016 at 03:17:41PM +0100, Dimitri Staessens wrote:
> Dear Konstantin,
> 
> thanks for your immediate response. Please find attached a minimal code 
> example. I do hope I'm sending in the correct format, I'm new to the 
> community.
> 
> The test creates an integer, robust mutex and condition variable in 
> shared memory. A thread blocks on that condition variable with the 
> associated mutex. After one second, the main thread cancels the blocking 
> thread.
> 
> I compile as follows:
> 
> gcc robust_test.c -lpthread -lrt -o robust_test
> 
> and run
> 
> ./robust_test
> 
> On linux it gives the following output:
> [dstaesse@phoneutria]$ ./robust_test
> Initializing...
> Starting thread...
> Sleeping for one second...
> Thread started...
> Cancelling thread...
> Thread finished.
> Bye.
> 
> On FreeBSD I get the following:
> $ ./robust_test
> Initializing...
> Starting thread...
> Sleeping for one second...
> Thread started...
> Cancelling thread...
> Fatal error 'inact_mtx enter' at line 188 in file 
> /usr/src/lib/libthr/thread/thr_mutex.c (errno = 0)
> Abort trap (core dumped)

Try this patch.  It worked for me.
It is enough to patch and then rebuild only libthr:
cd /usr/src
patch -p1 <patch
(cd lib/libthr && make WITHOUT_TESTS=yes all install)

diff --git a/lib/libthr/thread/thr_cond.c b/lib/libthr/thread/thr_cond.c
index 506b8eca9e7..64d075ca06f 100644
--- a/lib/libthr/thread/thr_cond.c
+++ b/lib/libthr/thread/thr_cond.c
@@ -224,16 +224,26 @@ cond_wait_kernel(struct pthread_cond *cvp, struct pthread_mutex *mp,
 		 * state and unlock the mutex without making the state
 		 * consistent and the state will be unrecoverable.
 		 */
-		if (error2 == 0 && cancel)
+		if (error2 == 0 && cancel) {
+			if (robust) {
+				_mutex_leave_robust(curthread, mp);
+				robust = false;
+			}
 			_thr_testcancel(curthread);
+		}
 
 		if (error == EINTR)
 			error = 0;
 	} else {
 		/* We know that it didn't unlock the mutex. */
 		_mutex_cv_attach(mp, recurse);
-		if (cancel)
+		if (cancel) {
+			if (robust) {
+				_mutex_leave_robust(curthread, mp);
+				robust = false;
+			}
 			_thr_testcancel(curthread);
+		}
 		error2 = 0;
 	}
 	if (robust)