From owner-freebsd-threads@FreeBSD.ORG Mon Jun 27 11:02:02 2005 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9BCD216A420 for ; Mon, 27 Jun 2005 11:02:02 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7EE9643D4C for ; Mon, 27 Jun 2005 11:02:02 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id j5RB22uJ043214 for ; Mon, 27 Jun 2005 11:02:02 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id j5RB214N043205 for freebsd-threads@freebsd.org; Mon, 27 Jun 2005 11:02:01 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 27 Jun 2005 11:02:01 GMT Message-Id: <200506271102.j5RB214N043205@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jun 2005 11:02:02 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2005/01/26] threads/76690threads fork hang in child for (-lc_r & -lthr) o [2005/05/11] threads/80887threads ULE with SMP broke libpthread/libthr on 5 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2000/08/26] kern/20861 threads libc_r does not honor socket timeouts o [2001/01/20] threads/24472threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] threads/24632threads libc_r delicate deviation from libc in ha o [2001/01/25] kern/24641 threads pthread_rwlock_rdlock can deadlock o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/02/01] threads/34536threads accept() blocks other threads o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/06/27] threads/39922threads [PATCH?] Threaded applications executed w o [2002/08/04] kern/41331 threads Pthread library open sets O_NONBLOCK flag o [2003/03/02] threads/48856threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] threads/49087threads Signals lost in programs linked with libc o [2003/05/08] threads/51949threads thread in accept cannot be cancelled s [2004/03/15] kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/un o [2004/08/26] threads/70975threads unexpected and unreliable behaviour when o [2004/09/14] threads/71725threads Mysql Crashes frequently giving Sock Erro o [2004/10/05] threads/72353threads Assertion fails in /usr/src/lib/libpthrea o [2004/10/07] threads/72429threads threads blocked in stdio (fgets, etc) are o [2004/10/21] threads/72953threads fork() unblocks blocked signals w/o PTHRE o [2004/12/19] threads/75273threads FBSD 5.3 libpthread (KSE) bug o [2004/12/21] threads/75374threads pthread_kill() ignores SA_SIGINFO flag o [2005/01/26] threads/76694threads fork cause hang in dup()/close() function o [2005/03/10] threads/78660threads Java hangs unkillably in STOP state after o [2005/04/08] threads/79683threads svctcp_create() fails if multiple threads o [2005/04/28] threads/80435threads panic on high loads o [2005/05/19] threads/81258threads Thread specific data is sometimes assigne 26 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/05/26] kern/18824 threads gethostbyname is not thread safe o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2000/10/21] kern/22190 threads A threaded read(2) from a socketpair(2) f o [2001/09/09] threads/30464threads pthread mutex attributes -- pshared o [2002/05/02] threads/37676threads libc_r: msgsnd(), msgrcv(), pread(), pwri s [2002/07/16] threads/40671threads pthread_cancel doesn't remove thread from o [2004/07/13] threads/69020threads pthreads library leaks _gc_mutex o [2004/09/21] threads/71966threads Mlnet Core Dumped : Fatal error '_pq_inse o [2004/11/21] threads/74180threads KSE problem. Applications those riched ma o [2005/01/20] threads/76513threads libpthread is not working o [2005/04/13] threads/79887threads [patch] freopen() isn't thread-safe o [2005/05/13] threads/80992threads abort() sometimes not caught by gdb depen o [2005/05/26] threads/81534threads [PATCH] libc_r close() will fail on any f 13 problems total. From owner-freebsd-threads@FreeBSD.ORG Wed Jun 29 10:56:41 2005 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B14516A41C for ; Wed, 29 Jun 2005 10:56:41 +0000 (GMT) (envelope-from NKoch@demig.de) Received: from server.absolute-media.de (server.absolute-media.de [213.239.231.9]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB96E43D1D for ; Wed, 29 Jun 2005 10:56:40 +0000 (GMT) (envelope-from NKoch@demig.de) Received: from localhost (unknown [127.0.0.1]) by server.absolute-media.de (Postfix) with ESMTP id B5B8380832 for ; Wed, 29 Jun 2005 12:56:38 +0200 (CEST) Received: from server.absolute-media.de ([127.0.0.1]) by localhost (server [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 11605-04 for ; Wed, 29 Jun 2005 12:56:33 +0200 (CEST) Received: from firewall.demig (p5083937F.dip0.t-ipconnect.de [80.131.147.127]) by server.absolute-media.de (Postfix) with ESMTP id D78BF8705C for ; Wed, 29 Jun 2005 12:56:32 +0200 (CEST) Received: from ws-ew-3 (ws-ew-3.w2kdemig [192.168.1.72]) by firewall.demig (8.13.4/8.13.1) with SMTP id j5TArBf7035030 for ; Wed, 29 Jun 2005 12:53:11 +0200 (CEST) (envelope-from NKoch@demig.de) From: "Norbert Koch" To: Date: Wed, 29 Jun 2005 12:53:11 +0200 Message-ID: <000701c57c98$be207ac0$4801a8c0@ws-ew-3.W2KDEMIG> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2120.0 Importance: Normal X-Virus-Scanned: by amavisd-new X-Virus-Scanned: by amavisd-new at absolute-media.de Subject: CLOCK_REALTIME/CLOCK_MONOTONIC X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Jun 2005 10:56:41 -0000 Hello. I am working on a multi-threaded application which may call settimeofday() and therefore may have serious problems with timing calculations. In my applications I calclulate time differences using clock_gettime(CLOCK_MONOTONIC) under FreeBSD-5. Under FreeBSD-4 it is a trivial kernel patch in kern_time.c to have CLOCK_MONOTONIC, as there already is a kernel function nanouptime(). int clock_gettime(p, uap) struct proc *p; struct clock_gettime_args *uap; { struct timespec ats; switch (SCARG(uap, clock_id)) { case CLOCK_REALTIME: nanotime(&ats); break; case CLOCK_MONOTONIC: nanouptime(&ats); break; default: return (EINVAL); }; return (copyout(&ats, SCARG(uap, tp), sizeof(ats))); } Looking through the sources of the various threading libraries I found that either gettimeofday() or clock_gettime(CLOCK_REALTIME) is used for all calculations. I am not sure, what posix currently says about this but found a chapter 'Condition variable wait clock' in [Butenhof] (p.359). As I understand it, Posix.1j expects an implementation to - at least for pthread_cond_timedwait() - use CLOCK_MONOTONIC by default. They introduce a new function pair pthread_condattr_(get|set)clock() to change pthread_cond_timedwait() to use either CLOCK_MONOTONIC or CLOCK_REALTIME. >From my understanding of the threading libraries' internals, it should be trivial to modify them to using CLOCK_MONOTONIC only, but not quite as trivial to implement pthread_condattr_(get|set)clock(). For FreeBSD-4 I already have a modified libc_r, where I call clock_gettime(CLOCK_MONOTONIC) once in _thread_init() and set a global variable _sched_clkid to either CLOCK_MONOTONIC or CLOCK_REALTIME for further calls to clock_gettime(). Any comments/ideas/opinions? Norbert From owner-freebsd-threads@FreeBSD.ORG Thu Jun 30 17:25:54 2005 Return-Path: X-Original-To: freebsd-threads@FreeBSD.org Delivered-To: freebsd-threads@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3C0E16A41C; Thu, 30 Jun 2005 17:25:54 +0000 (GMT) (envelope-from tataz@tataz.chchile.org) Received: from postfix3-2.free.fr (postfix3-2.free.fr [213.228.0.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5300C43D1D; Thu, 30 Jun 2005 17:25:54 +0000 (GMT) (envelope-from tataz@tataz.chchile.org) Received: from tatooine.tataz.chchile.org (vol75-8-82-233-239-98.fbx.proxad.net [82.233.239.98]) by postfix3-2.free.fr (Postfix) with ESMTP id DD5AAC085; Thu, 30 Jun 2005 19:25:52 +0200 (CEST) Received: by tatooine.tataz.chchile.org (Postfix, from userid 1000) id B9819405B; Thu, 30 Jun 2005 19:26:03 +0200 (CEST) Date: Thu, 30 Jun 2005 19:26:03 +0200 From: Jeremie Le Hen To: freebsd-hackers@FreeBSD.org Message-ID: <20050630172602.GI49933@obiwan.tataz.chchile.org> References: <20050630165017.GH49933@obiwan.tataz.chchile.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20050630165017.GH49933@obiwan.tataz.chchile.org> User-Agent: Mutt/1.5.9i Cc: freebsd-threads@FreeBSD.org Subject: Re: ProPolice and pthreads (was: ProPolice and FreeBSD) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Jun 2005 17:25:54 -0000 [ I'm not snipping anything since I'm Cc'ing to -threads@. ] On Thu, Jun 30, 2005 at 06:50:17PM +0200, Jeremie Le Hen wrote: > Hello, > > maybe this question should be asked on -threads@, I'm not sure. If it > is, please tell me and I will redirect my mail. > > I'm working on upgrading the ProPolice/SSP patch [1] to -CURRENT. I > initially used the patch against FreeBSD 5.1 to know which file I > should modify and in which way in the source tree, but I used the > newest patch against gcc 3.4.1 for gcc specific stuffs. > > After a little work, I got a full FreeBSD built with SSP functions > compiled in libc (it is also possible to compile it in libgcc but, > AFAIU, this would require the SSP stuff to be statically built in > all binaries since FreeBSD doesn't provide a shared libgcc). I also > read somewhere that some guys of the hardened Debian project have made > a libssp, but I find this a little bit overkill (comments ?). > > I recompiled host(1), libc and libpthread with debugging symbol. > > Now the questions. All binaries linked against libpthread immediately > get a SIGSEGV : > %%% > coyote:libc# gdb /usr/bin/host > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd"... > (gdb) r > Starting program: /usr/bin/host > warning: Unable to get location for thread creation breakpoint: generic error > [New LWP 100135] > > Program received signal SIGSEGV, Segmentation fault. > [Switching to LWP 100135] > _thr_cancel_enter (thread=0x0) > at /usr/src/lib/libpthread/thread/thr_cancel.c:269 > 269 THR_THREAD_LOCK(thread, thread); > (gdb) bt full > #0 _thr_cancel_enter (thread=0x0) > at /usr/src/lib/libpthread/thread/thr_cancel.c:269 > No locals. > #1 0x282dd51b in __open (path=0x281bc0c0 "ÀÀ\033( \220\033(", flags=0) > at /usr/src/lib/libpthread/thread/thr_open.c:53 > curthread = (struct pthread *) 0x0 > ret = 0 > mode = 0 > #2 0x2838f40b in __guard_setup () > at /usr/src/lib/libc/sys/stack_protector.c:51 > fd = 0 > #3 0x283cbe22 in lseek () from /lib/libc.so.6 > No symbol table info available. > #4 0x28316dd1 in _init () from /lib/libc.so.6 > No symbol table info available. > #5 0x281b8000 in ?? () > No symbol table info available. > #6 0x281ad6fc in ?? () from /libexec/ld-elf.so.1 > No symbol table info available. > #7 0xbfbfeca8 in ?? () > No symbol table info available. > #8 0x2818cc79 in find_symdef () from /libexec/ld-elf.so.1 > No symbol table info available. > #9 0x2818b759 in _rtld () from /libexec/ld-elf.so.1 > No symbol table info available. > #10 0x2818a98e in .rtld_start () from /libexec/ld-elf.so.1 > No symbol table info available. > %%% > > __guard_setup() is the constructor of the SSP patch, it generates a > random cookie for the application runtime : > %%% > static void __guard_setup(void) __attribute__ ((constructor)); > static void > __guard_setup(void) > { > int fd; > if (__guard[0]!=0) return; > fd = open ("/dev/urandom", 0); > if (fd != -1) { > ssize_t size = read (fd, (char*)&__guard, sizeof(__guard)); > close (fd) ; > if (size == sizeof(__guard)) return; > } > /* If a random generator can't be used, the protector switches the guard > to the "terminator canary" */ > ((char*)__guard)[0] = 0; ((char*)__guard)[1] = 0; > ((char*)__guard)[2] = '\n'; ((char*)__guard)[3] = 255; > } > %%% > > I am neither a gcc hacker nor a thread guru, so I have no clue on how to > resolve this issue. Advices are welcome. > > Thanks. > > Regards, > [1] http://www.trl.ibm.com/projects/security/ssp/ cognet@ sent me the following patch and it makes pthreaded programs work like a charm. He also said me that this change will be surely needed for libthr. %%% cvs diff: Diffing . Index: thr_open.c =================================================================== RCS file: /nfs/donald/repo/FreeBSD/src/lib/libpthread/thread/thr_open.c,v retrieving revision 1.16 diff -u -p -r1.16 thr_open.c --- thr_open.c 9 Dec 2003 02:20:56 -0000 1.16 +++ thr_open.c 30 Jun 2005 17:19:03 -0000 @@ -45,11 +45,14 @@ __weak_reference(__open, open); int __open(const char *path, int flags,...) { - struct pthread *curthread = _get_curthread(); + struct pthread *curthread; int ret; int mode = 0; va_list ap; + if (_thr_initial == NULL) + _libpthread_init(NULL); + curthread = _get_curthread(); _thr_cancel_enter(curthread); /* Check if the file is being created: */ %%% For now, I'm including this in my ProPolice patch. Regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org >