From owner-freebsd-threads@FreeBSD.ORG Mon Jul 12 08:10:06 2010 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8733E1065674 for ; Mon, 12 Jul 2010 08:10:06 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 46E218FC20 for ; Mon, 12 Jul 2010 08:10:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o6C8A6rx035555 for ; Mon, 12 Jul 2010 08:10:06 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o6C8A6fs035554; Mon, 12 Jul 2010 08:10:06 GMT (envelope-from gnats) Resent-Date: Mon, 12 Jul 2010 08:10:06 GMT Resent-Message-Id: <201007120810.o6C8A6fs035554@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Vikash Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 699861065674 for ; Mon, 12 Jul 2010 08:06:38 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id 58BCC8FC15 for ; Mon, 12 Jul 2010 08:06:38 +0000 (UTC) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.3/8.14.3) with ESMTP id o6C86bqQ051300 for ; Mon, 12 Jul 2010 08:06:37 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.3/8.14.3/Submit) id o6C86bjY051299; Mon, 12 Jul 2010 08:06:37 GMT (envelope-from nobody) Message-Id: <201007120806.o6C86bjY051299@www.freebsd.org> Date: Mon, 12 Jul 2010 08:06:37 GMT From: Vikash To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: threads/148515: Memory / syslog strangeness in FreeBSD 8.x ( possible leak/ threading issue ) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2010 08:10:06 -0000 >Number: 148515 >Category: threads >Synopsis: Memory / syslog strangeness in FreeBSD 8.x ( possible leak/ threading issue ) >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Jul 12 08:10:05 UTC 2010 >Closed-Date: >Last-Modified: >Originator: Vikash >Release: 8.0 RELEASE >Organization: >Environment: FreeBSD vix.isnet.net 8.0-RELEASE-p3 FreeBSD 8.0-RELEASE-p3 #11: Sun Jun 27 20:10:59 SAST 2010 root@vix.isnet.net:/usr/obj/usr/src/sys/GENERIC i386 >Description: Can someone look into this strangeness on FreeBSD 8.0 and 8.1-RC2 using a threaded test code, I see the that freebsd 8.x seems to be using more memory than freebsd 7.x: Results: 7.2 without syslog PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 26872 vikashb 1001 8 0 128M 14236K RUN 0:00 0.00% a.out with syslog PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 26881 vikashb 1001 44 0 128M 26236K RUN 0:00 0.00% a.out 8.0-RELEASE-p3 without syslog PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 61529 vikashb 1001 44 0 129M 14840K RUN 0:01 0.00% a.out with syslog PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 61507 vikashb 1001 44 0 257M 42708K RUN 0:30 0.00% a.out 8.1-RC2 without syslog PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 33062 vikashb 1001 44 0 129M 14804K RUN 0:00 0.00% a.out with syslog PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND 33056 vikashb 1001 44 0 257M 42708K RUN 0:03 0.00% a.out I have not been able to find any reasonable information via Google/ PR search either there is a leak or there is an undocumented behaviour in the 8.x threads. neither freebsd-questions@freebsd.org nor freebsd-threads@freebsd.org yielded ant responses. >How-To-Repeat: compile the code below and observe the memory usage 8.x uses more memory than 7.x #include #include #include #include #include #include #include #include #include char *ProgramName = "WTF"; int loop = 0; int LogToSTDOUT = 1; void LogMessage(int debug, const char *fmt,...) { extern int LogToSTDOUT; char message[8192]; memset(message, 0, sizeof(message)); va_list args; va_start(args, fmt); vsnprintf(message, sizeof(message), fmt, args); va_end(args); if ( LogToSTDOUT ) { printf("%s\n", message); } syslog(LOG_NOTICE, "%s", message); } unsigned long int getTimeNow() { struct timeval tv; if ( gettimeofday(&tv, NULL) == -1 ) { LogMessage(0, "ERROR(%d) %s\n", errno, strerror(errno)); tv.tv_sec = 0; } return tv.tv_sec; } void HandleSignal(int sig) { loop = 0; LogMessage(0, "loop = %d\n", loop); signal(sig, SIG_IGN); usleep(1000); } void *worker(int n) { while ( loop ) { LogMessage(0, "worker #%d logging", n); usleep(1000); } pthread_exit(0); } int main(int argc, char* argv[]) { pthread_t* tpool; int workers = 1000, i, rc; openlog(ProgramName, LOG_PID, LOG_MAIL); unsigned long int duration = 120, StartTime, TimeNow; signal(SIGINT, HandleSignal); signal(SIGTERM, HandleSignal); signal(SIGHUP, HandleSignal); signal(SIGQUIT, HandleSignal); StartTime = getTimeNow(); tpool = (pthread_t*)malloc(workers * sizeof(pthread_t)); if ( tpool == NULL ) { LogMessage(0, "malloc failed \n"); closelog(); exit(-1); } memset(tpool, 0, sizeof(pthread_t) * workers); loop = 1; for ( i = 0; i < workers; i++ ) { rc = pthread_create(&tpool[i], NULL, (void *(*)(void*))&worker, (void*)i); if ( rc != 0 ) { LogMessage(0, "pthread_create #%d failed\n", i ); pthread_cancel(tpool[i]); } else { pthread_detach(tpool[i]); } } LogMessage(0, "loop = %d\n", loop); while ( loop ) { TimeNow = getTimeNow(); if ( ( TimeNow - StartTime ) > duration ) { loop = 0; } usleep(1000); } for ( i = 0; i < workers; i++ ) { pthread_cancel(tpool[i]); } for ( i = 0; i < workers; i++ ) { pthread_join(tpool[i], NULL); } free(tpool); closelog(); exit(0); } >Fix: >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Mon Jul 12 11:07:10 2010 Return-Path: Delivered-To: freebsd-threads@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85B2A1065680 for ; Mon, 12 Jul 2010 11:07:10 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6B3888FC1B for ; Mon, 12 Jul 2010 11:07:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o6CB7ApZ094156 for ; Mon, 12 Jul 2010 11:07:10 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o6CB79uV094154 for freebsd-threads@FreeBSD.org; Mon, 12 Jul 2010 11:07:09 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 12 Jul 2010 11:07:09 GMT Message-Id: <201007121107.o6CB79uV094154@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-threads@FreeBSD.org X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2010 11:07:10 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o threa/148515 threads Memory / syslog strangeness in FreeBSD 8.x ( possible o threa/141721 threads rtprio(1): (id|rt)prio priority resets when new thread o threa/141198 threads [libc] src/lib/libc/stdio does not properly initialize o threa/136345 threads Recursive read rwlocks in thread A cause deadlock with o threa/135673 threads databases/mysql50-server - MySQL query lock-ups on 7.2 o threa/133734 threads 32 bit libthr failing pthread_create() o threa/128922 threads threads hang with xorg running o threa/127225 threads bug in lib/libthr/thread/thr_init.c o threa/122923 threads 'nice' does not prevent background process from steali o threa/121336 threads lang/neko threading ok on UP, broken on SMP (FreeBSD 7 f threa/118715 threads kse problem o threa/116668 threads can no longer use jdk15 with libthr on -stable SMP o threa/116181 threads /dev/io-related io access permissions are not propagat o threa/115211 threads pthread_atfork misbehaves in initial thread o threa/110636 threads [request] gdb(1): using gdb with multi thread applicat o threa/110306 threads apache 2.0 segmentation violation when calling gethost o threa/103975 threads Implicit loading/unloading of libpthread.so may crash o threa/101323 threads [patch] fork(2) in threaded programs broken. s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/79887 threads [patch] freopen() isn't thread-safe o threa/79683 threads svctcp_create() fails if multiple threads call at the s threa/76694 threads fork cause hang in dup()/close() function in child (-l s threa/76690 threads fork hang in child for -lc_r f threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST s threa/69020 threads pthreads library leaks _gc_mutex s threa/49087 threads Signals lost in programs linked with libc_r s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/34536 threads accept() blocks other threads s threa/32295 threads [libc_r] [patch] pthread(3) dont dequeue signals s threa/30464 threads pthread mutex attributes -- pshared s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o 38 problems total. From owner-freebsd-threads@FreeBSD.ORG Sat Jul 17 20:15:13 2010 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DD471065677; Sat, 17 Jul 2010 20:15:13 +0000 (UTC) (envelope-from jilles@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D93F58FC0C; Sat, 17 Jul 2010 20:15:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o6HKFCJf017372; Sat, 17 Jul 2010 20:15:12 GMT (envelope-from jilles@freefall.freebsd.org) Received: (from jilles@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o6HKFCmw017368; Sat, 17 Jul 2010 20:15:12 GMT (envelope-from jilles) Date: Sat, 17 Jul 2010 20:15:12 GMT Message-Id: <201007172015.o6HKFCmw017368@freefall.freebsd.org> To: marka@daemon.lab.isc.org, jilles@FreeBSD.org, freebsd-threads@FreeBSD.org From: jilles@FreeBSD.org Cc: Subject: Re: threads/72953: fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYSTEM X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jul 2010 20:15:13 -0000 Synopsis: fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYSTEM State-Changed-From-To: feedback->closed State-Changed-By: jilles State-Changed-When: Sat Jul 17 20:15:12 UTC 2010 State-Changed-Why: Feedback timeout. http://www.freebsd.org/cgi/query-pr.cgi?pr=72953 From owner-freebsd-threads@FreeBSD.ORG Sat Jul 17 20:20:21 2010 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA46A1065678; Sat, 17 Jul 2010 20:20:21 +0000 (UTC) (envelope-from jilles@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 5BEF28FC15; Sat, 17 Jul 2010 20:20:21 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o6HKKLYj019002; Sat, 17 Jul 2010 20:20:21 GMT (envelope-from jilles@freefall.freebsd.org) Received: (from jilles@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o6HKKLOB018981; Sat, 17 Jul 2010 20:20:21 GMT (envelope-from jilles) Date: Sat, 17 Jul 2010 20:20:21 GMT Message-Id: <201007172020.o6HKKLOB018981@freefall.freebsd.org> To: gary@velocity-servers.net, jilles@FreeBSD.org, freebsd-threads@FreeBSD.org From: jilles@FreeBSD.org Cc: Subject: Re: threads/118715: kse problem X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jul 2010 20:20:21 -0000 Synopsis: kse problem State-Changed-From-To: feedback->closed State-Changed-By: jilles State-Changed-When: Sat Jul 17 20:20:20 UTC 2010 State-Changed-Why: Feedback timeout. http://www.freebsd.org/cgi/query-pr.cgi?pr=118715