From owner-freebsd-threads@freebsd.org Tue Dec 6 16:38:18 2016 Return-Path: Delivered-To: freebsd-threads@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36A25C6A3A3 for ; Tue, 6 Dec 2016 16:38:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE7FC17CA for ; Tue, 6 Dec 2016 16:38:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uB6Gc7Rp046147 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 6 Dec 2016 18:38:07 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uB6Gc7Rp046147 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uB6Gc7id046146; Tue, 6 Dec 2016 18:38:07 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 6 Dec 2016 18:38:07 +0200 From: Konstantin Belousov To: Dimitri Staessens Cc: freebsd-threads@freebsd.org Subject: Re: Unlocking a robust mutex in a cleanup handler Message-ID: <20161206163807.GT54029@kib.kiev.ua> References: <119e59d4-6125-f313-e6e6-67055a15d224@intec.ugent.be> <20161206112558.GN54029@kib.kiev.ua> <6a7139cd-b6db-d078-ee5e-b7c590eb13d1@intec.ugent.be> <20161206144812.GS54029@kib.kiev.ua> <35726dbb-75f7-682d-ad41-c78b96675485@intec.ugent.be> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <35726dbb-75f7-682d-ad41-c78b96675485@intec.ugent.be> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2016 16:38:18 -0000 On Tue, Dec 06, 2016 at 04:30:14PM +0100, Dimitri Staessens wrote: > Dear Konstantin, > > I didn't get the error, but on my machine the thread never exists the > condwait when the pthread_cancel is called. > > gdb output: > > $ sudo gdb ./robust_test 3246 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"...(no debugging > symbols found)... > Attaching to program: /usr/home/dstaesse/robust_test, process 3246 > Reading symbols from /lib/libthr.so.3...Reading symbols from > /usr/lib/debug//lib/libthr.so.3.debug...done. > [New Thread 801416500 (LWP 100404/robust_test)] > [New Thread 801416000 (LWP 100313/robust_test)] > done. > Loaded symbols for /lib/libthr.so.3 > Reading symbols from /usr/lib/librt.so.1...done. > Loaded symbols for /usr/lib/librt.so.1 > Reading symbols from /lib/libc.so.7...done. > Loaded symbols for /lib/libc.so.7 > Reading symbols from /libexec/ld-elf.so.1...done. > Loaded symbols for /libexec/ld-elf.so.1 > [Switching to Thread 801416000 (LWP 100313/robust_test)] > 0x00000008008386ac in _umtx_op_err () from /lib/libthr.so.3 > (gdb) info threads > * 2 Thread 801416000 (LWP 100313/robust_test) 0x00000008008386ac in > _umtx_op_err () from /lib/libthr.so.3 > 1 Thread 801416500 (LWP 100404/robust_test) _thr_ast > (curthread=0x801416500) at /usr/src/lib/libthr/thread/thr_sig.c:271 > Current language: auto; currently minimal > (gdb) bt > #0 0x00000008008386ac in _umtx_op_err () from /lib/libthr.so.3 > #1 0x0000000800834df6 in join_common (pthread=, > thread_return=, abstime=) > at /usr/src/lib/libthr/thread/thr_join.c:125 > #2 0x0000000000401186 in main () > (gdb) thread 1 > [Switching to thread 1 (Thread 801416500 (LWP 100404/robust_test))]#0 > _thr_ast (curthread=0x801416500) > at /usr/src/lib/libthr/thread/thr_sig.c:271 > 271 check_suspend(curthread); > (gdb) bt > #0 _thr_ast (curthread=0x801416500) at > /usr/src/lib/libthr/thread/thr_sig.c:271 > #1 0x0000000800837a5b in __thr_pshared_offpage (key= out>, doalloc=) > at /usr/src/lib/libthr/thread/thr_pshared.c:86 > #2 0x00000008008363cb in cond_wait_common (cond=, > mutex=0x800643004, abstime=0x0, cancel=1) > at /usr/src/lib/libthr/thread/thr_cond.c:349 > #3 0x0000000000400ff2 in blockfunc () > #4 0x000000080082ab55 in thread_start (curthread=) > at /usr/src/lib/libthr/thread/thr_create.c:289 > #5 0x0000000000000000 in ?? () > (gdb) > I suspect that there is an issue with the test program itself. If you terminate your program, e.g. with SIGING/Ctrl-C, then shm_unlink() call is not performed at the end, and orphaned locked robust mutex is kept associated with that memory segment. Then, since you have the loop around pthread_cond_wait() call, it seems feasible to assume that the next instance of the program gets ignored errors from pthread_mutex_lock() and pthread_cond_wait(). This is explicitely allowed by POSIX, which states that "Attempting to initialize an already initialized mutex results in undefined behavior." Can you try the following modification of your test program, without rebooting the machine, so that the shared segment and mutex were kept around ? #define _POSIX_C_SOURCE 200809L #define __XSI_VISIBLE 500 #include #include #include #include #include #include #define FN "/robust" #define FS (sizeof(int) + sizeof(pthread_mutex_t) + sizeof(pthread_cond_t)) /* contents of the shm segment */ int * shm_int; pthread_mutex_t * shm_mtx; pthread_cond_t * shm_cnd; /* function for thread */ void * blockfunc(void * o) { int error; printf("Thread started...\n"); error = pthread_mutex_lock(shm_mtx); if (error != 0) printf("mutex_lock err %d %s\n", error, strerror(error)); pthread_cleanup_push((void (*)(void *)) pthread_mutex_unlock, (void *) shm_mtx); error = 0; while (*shm_int == 0 && error == 0) error = pthread_cond_wait(shm_cnd, shm_mtx); if (error != 0) printf("cond_wait err %d %s\n", error, strerror(error)); pthread_cleanup_pop(1); return (void *) 0; } int main(void) { /* file descriptor for shm_open */ int fd; /* mutex and condvar attributes */ pthread_mutexattr_t mattr; pthread_condattr_t cattr; /* thread that will block on the convar in shm */ pthread_t thr; printf("Initializing...\n"); /* create shm segment containing an int, a mutex, and a condvar */ fd = shm_open(FN, O_CREAT | O_RDWR, 0666); ftruncate(fd, FS - 1); shm_int = mmap(NULL, FS, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); shm_mtx = (pthread_mutex_t *) (shm_int + 1); shm_cnd = (pthread_cond_t *) (shm_mtx + 1); close(fd); /* initialize the contents */ pthread_mutexattr_init(&mattr); pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED); pthread_mutexattr_setrobust(&mattr, PTHREAD_MUTEX_ROBUST); pthread_condattr_init(&cattr); pthread_condattr_setpshared(&cattr, PTHREAD_PROCESS_SHARED); pthread_mutex_init(shm_mtx, &mattr); pthread_cond_init(shm_cnd, &cattr); *shm_int = 0; /* start the thread */ printf("Starting thread...\n"); pthread_create(&thr, NULL, blockfunc, NULL); /* sleep for a second */ printf("Sleeping for one second...\n"); sleep(1); /* cancel the thread */ printf("Cancelling thread...\n"); pthread_cancel(thr); /* wait for the thread to join */ pthread_join(thr, NULL); printf("Thread finished.\n"); pthread_mutex_destroy(shm_mtx); pthread_cond_destroy(shm_cnd); /* cleanup shared memory */ munmap(shm_int, FS); shm_unlink(FN); printf("Bye.\n"); return (0); }