From owner-freebsd-threads@FreeBSD.ORG Mon Feb 19 11:08:39 2007 Return-Path: X-Original-To: freebsd-threads@FreeBSD.org Delivered-To: freebsd-threads@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 759AC16A559 for ; Mon, 19 Feb 2007 11:08:39 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [69.147.83.40]) by mx1.freebsd.org (Postfix) with ESMTP id 51FEF13C461 for ; Mon, 19 Feb 2007 11:08:39 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id l1JB8dr6021515 for ; Mon, 19 Feb 2007 11:08:39 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id l1JB8blG021511 for freebsd-threads@FreeBSD.org; Mon, 19 Feb 2007 11:08:37 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 19 Feb 2007 11:08:37 GMT Message-Id: <200702191108.l1JB8blG021511@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: linimon set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Feb 2007 11:08:39 -0000 Current FreeBSD problem reports Critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- s threa/76690 threads fork hang in child for -lc_r 1 problem total. Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/20016 threads pthreads: Cannot set scheduling timer/Cannot set virtu s threa/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVTIMEO socket o s threa/24632 threads libc_r delicate deviation from libc in handling SIGCHL s bin/32295 threads pthread dont dequeue signals s threa/34536 threads accept() blocks other threads o kern/38549 threads the procces compiled whith pthread stopped in pthread_ s threa/39922 threads [threads] [patch] Threaded applications executed with s threa/48856 threads Setting SIGCHLD to SIG_IGN still leaves zombies under s threa/49087 threads Signals lost in programs linked with libc_r s kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/unset O_NONBLOC o threa/70975 threads unexpected and unreliable behaviour when using SYSV se o threa/72429 threads threads blocked in stdio (fgets, etc) are not cancella o threa/72953 threads fork() unblocks blocked signals w/o PTHREAD_SCOPE_SYST o threa/75273 threads FBSD 5.3 libpthread (KSE) bug o threa/75374 threads pthread_kill() ignores SA_SIGINFO flag s threa/76694 threads fork cause hang in dup()/close() function in child (-l o threa/79683 threads svctcp_create() fails if multiple threads call at the o threa/80435 threads panic on high loads o threa/83914 threads [libc] popen() doesn't work in static threaded program s threa/84483 threads problems with devel/nspr and -lc_r on 4.x o threa/85160 threads [libthr] [patch] libobjc + libpthread/libthr crash pro o threa/90278 threads libthr, ULE and -current produces >100% WCPU with apac o kern/91266 threads [threads] Trying sleep, but thread marked as sleeping s threa/94467 threads send(), sendto() and sendmsg() are not correct in libc f threa/98256 threads gnome-system-monitor core dumps from pthread_testcance s threa/100815 threads FBSD 5.5 broke nanosleep in libc_r o threa/101323 threads fork(2) in threaded programs broken. o threa/103975 threads Implicit loading/unloading of libpthread.so may crash 28 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- s kern/19247 threads uthread_sigaction.c does not do anything wrt SA_NOCLDW s kern/22190 threads A threaded read(2) from a socketpair(2) fd can sometim s threa/30464 threads pthread mutex attributes -- pshared s threa/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwrite() need wra s threa/40671 threads pthread_cancel doesn't remove thread from condition qu s threa/69020 threads pthreads library leaks _gc_mutex o threa/79887 threads [patch] freopen() isn't thread-safe o threa/80992 threads abort() sometimes not caught by gdb depending on threa o threa/81534 threads [libc_r] [patch] libc_r close() will fail on any fd ty 9 problems total. From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 16:01:37 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ACDFF16A5D6 for ; Tue, 20 Feb 2007 16:01:37 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id 3646013C48D for ; Tue, 20 Feb 2007 16:01:36 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1KFnk5Q081102; Tue, 20 Feb 2007 16:49:47 +0100 (CET) (envelope-from mb@imp.ch) Date: Tue, 20 Feb 2007 16:49:46 +0100 (CET) From: Martin Blapp To: freebsd-threads@freebsd.org Message-ID: <20070220153632.E4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv Subject: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 16:01:37 -0000 Hi, After upgrading to clamd 0.90 I found that clamd doesn't work well with libpthread.so.2. Using libmap.conf entries for libthr.so.2 or libc_r.so.6 worked fine. With both thread libraries clamd works as it should. The effect with libpthread is that clamd works very very slow. It's only able to scan 1-2 files in 5 seconds instead of 400-500 in the same time with libthr/libc_r. A ktrace shows the following behaviour: clamd 0.90 and libpthread.so.2 1735 clamd RET fork 0 1735 clamd CALL kse_release(0xbe7eac20) 1735 clamd CALL kse_wakeup(0x9e4678c) 1735 clamd RET kse_wakeup 0 1735 clamd CALL kse_release(0x9ea3f10) 1735 clamd RET kse_release 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x805df4c) 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x9eabedc) 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x805dedc) 1735 clamd CALL kse_wakeup(0x9e4670c) 1735 clamd CALL kse_release(0x9eabedc) 1735 clamd RET kse_release 0 1735 clamd RET kse_wakeup 0 1735 clamd CALL kse_wakeup(0x9e4680c) 1735 clamd RET fork 0 1735 clamd CALL kse_wakeup(0x805740c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x9ea7f10) 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbef61c10) 1735 clamd CALL kse_release(0x805dedc) 1735 clamd CALL kse_wakeup(0x805740c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbec30f50) 1735 clamd CALL kse_wakeup(0x9e4678c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbe8fbb90) 1735 clamd CALL kse_wakeup(0x9e4670c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0x9ea7f40) 1735 clamd CALL kse_wakeup(0x9e4680c) 1735 clamd RET kse_release 0 1735 clamd RET kse_release 0 1735 clamd RET kse_wakeup 0 1735 clamd CALL kse_release(0x9ea7ed0) 1735 clamd CALL kse_wakeup(0x805740c) 1735 clamd CALL kse_release(0xbef61bd0) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 1735 clamd CALL kse_release(0xbee50bd0) 1735 clamd CALL kse_wakeup(0x9e4678c) 1735 clamd RET kse_wakeup 0 1735 clamd RET kse_release 0 After ~10000 lines which show all like this, clamd actually opens and scans something, then another ~10000 lines follow ... I guess it's a bug in libpthread, but it really doesn't work fine with this lib. Or is it bad programming in clamav ? Any comments ? Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 16:44:24 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5BAEB16B6B4 for ; Tue, 20 Feb 2007 16:44:24 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id DB89813C48E for ; Tue, 20 Feb 2007 16:44:23 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1KGiKaC076722; Tue, 20 Feb 2007 17:44:21 +0100 (CET) (envelope-from mb@imp.ch) Date: Tue, 20 Feb 2007 17:44:20 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: Message-ID: <20070220174221.B4139@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 16:44:24 -0000 Hi, > Probably because it's fork()ing and trying to call > non-async-signal-safe functions without doing an exec()? The strange thing is that I can't see any any fork() calls with libc_r or libthr. How's that possible ? Or do you think that its stuck at the beginning ? But why does it work then for some degree ? Martin From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 17:02:30 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 77FB416AF7B for ; Tue, 20 Feb 2007 17:02:30 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 31A0B13C491 for ; Tue, 20 Feb 2007 17:02:30 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.14.0/8.14.0/NETPLEX) with ESMTP id l1KGkX0a003698; Tue, 20 Feb 2007 11:46:33 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-3.0 (mail.ntplx.net [204.213.176.10]); Tue, 20 Feb 2007 11:46:33 -0500 (EST) Date: Tue, 20 Feb 2007 11:46:33 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Martin Blapp In-Reply-To: <20070220174221.B4139@godot.imp.ch> Message-ID: References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 17:02:30 -0000 On Tue, 20 Feb 2007, Martin Blapp wrote: > > Hi, > >> Probably because it's fork()ing and trying to call >> non-async-signal-safe functions without doing an exec()? > > The strange thing is that I can't see any any fork() calls > with libc_r or libthr. How's that possible ? Or do you think > that its stuck at the beginning ? But why does it work then > for some degree ? I dunno, maybe ktrace was confused and it wasn't a fork()? -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 17:02:31 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A86916AFD5 for ; Tue, 20 Feb 2007 17:02:31 +0000 (UTC) (envelope-from eischen@vigrid.com) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id C33CA13C4A3 for ; Tue, 20 Feb 2007 17:02:30 +0000 (UTC) (envelope-from eischen@vigrid.com) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.14.0/8.14.0/NETPLEX) with ESMTP id l1KGdjOp026765; Tue, 20 Feb 2007 11:39:46 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-3.0 (mail.ntplx.net [204.213.176.10]); Tue, 20 Feb 2007 11:39:46 -0500 (EST) Date: Tue, 20 Feb 2007 11:39:45 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Martin Blapp In-Reply-To: <20070220153632.E4139@godot.imp.ch> Message-ID: References: <20070220153632.E4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 17:02:31 -0000 On Tue, 20 Feb 2007, Martin Blapp wrote: > > Hi, > > After upgrading to clamd 0.90 I found that clamd > doesn't work well with libpthread.so.2. > > Using libmap.conf entries for libthr.so.2 or libc_r.so.6 > worked fine. With both thread libraries clamd works as it > should. > > The effect with libpthread is that clamd works very very slow. > It's only able to scan 1-2 files in 5 seconds instead of > 400-500 in the same time with libthr/libc_r. > > A ktrace shows the following behaviour: > > clamd 0.90 and libpthread.so.2 > > 1735 clamd RET fork 0 Probably because it's fork()ing and trying to call non-async-signal-safe functions without doing an exec()? -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 18:15:45 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1B77216A50A for ; Tue, 20 Feb 2007 18:15:45 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id ACEBD13C49D for ; Tue, 20 Feb 2007 18:15:44 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1KIFdev032807; Tue, 20 Feb 2007 19:15:40 +0100 (CET) (envelope-from mb@imp.ch) Date: Tue, 20 Feb 2007 19:15:39 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: Message-ID: <20070220190347.C4139@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 18:15:45 -0000 Hi Daniel, Any idea how I can track this down ? It's rather easy to reproduce ... fork() is only used in the functions daemonize() and in virusaction(), which is used to call an external programm if one is defined (I don't have one). So fork() should only be called once during going to be a background daemon. Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ On Tue, 20 Feb 2007, Daniel Eischen wrote: > On Tue, 20 Feb 2007, Martin Blapp wrote: > >> >> Hi, >> >>> Probably because it's fork()ing and trying to call >>> non-async-signal-safe functions without doing an exec()? >> >> The strange thing is that I can't see any any fork() calls >> with libc_r or libthr. How's that possible ? Or do you think >> that its stuck at the beginning ? But why does it work then >> for some degree ? > > I dunno, maybe ktrace was confused and it wasn't a fork()? > > -- > DE > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 18:21:04 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B99BC16B6B6 for ; Tue, 20 Feb 2007 18:21:04 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 7DA0513C4B6 for ; Tue, 20 Feb 2007 18:21:04 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.14.0/8.14.0/NETPLEX) with ESMTP id l1KIL2lu028277; Tue, 20 Feb 2007 13:21:02 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-3.0 (mail.ntplx.net [204.213.176.10]); Tue, 20 Feb 2007 13:21:02 -0500 (EST) Date: Tue, 20 Feb 2007 13:21:02 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Martin Blapp In-Reply-To: <20070220190347.C4139@godot.imp.ch> Message-ID: References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 18:21:04 -0000 On Tue, 20 Feb 2007, Martin Blapp wrote: > > Hi Daniel, > > Any idea how I can track this down ? Try commenting out the fork() in daemonize(). Also try setting kern.threads.virtual_cpu=1 if you have more than 1 cpu (this will limit it to 1 KSE). > It's rather easy to reproduce ... > > fork() is only used in the functions daemonize() and in > virusaction(), which is used to call an external programm > if one is defined (I don't have one). > > So fork() should only be called once during going to be > a background daemon. -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 22:18:39 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9245A16C9CF; Tue, 20 Feb 2007 22:18:39 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id 2EAB913C461; Tue, 20 Feb 2007 22:18:38 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1KMIZ6C009234; Tue, 20 Feb 2007 23:18:35 +0100 (CET) (envelope-from mb@imp.ch) Date: Tue, 20 Feb 2007 23:18:34 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: Message-ID: <20070220225303.V4139@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 22:18:39 -0000 Hi, > Also try setting kern.threads.virtual_cpu=1 if you have > more than 1 cpu (this will limit it to 1 KSE). Our boxes have 2 CPUs and HTT is activated. Interesting. With kern.threads.virtual_cpu=1 it works also in the libpthread case, but while there are still some fork() calls it works better. With kern.threads.virtual_cpu=2 it's slower. And With kern.threads.virtual_cpu=4 its really slow. If I run clamd in foreground mode everything works as it should. More interesting, the CPU load drops significantly (500-1000%) ! Something is seriosly broken here. Martin From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 22:29:20 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1F0E716CAFA for ; Tue, 20 Feb 2007 22:29:20 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id D294013C494 for ; Tue, 20 Feb 2007 22:29:19 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.14.0/8.14.0/NETPLEX) with ESMTP id l1KMTHKK020421; Tue, 20 Feb 2007 17:29:17 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-3.0 (mail.ntplx.net [204.213.176.10]); Tue, 20 Feb 2007 17:29:17 -0500 (EST) Date: Tue, 20 Feb 2007 17:29:17 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Martin Blapp In-Reply-To: <20070220225303.V4139@godot.imp.ch> Message-ID: References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 22:29:20 -0000 On Tue, 20 Feb 2007, Martin Blapp wrote: > Hi, > >> Also try setting kern.threads.virtual_cpu=1 if you have >> more than 1 cpu (this will limit it to 1 KSE). > > Our boxes have 2 CPUs and HTT is activated. > > Interesting. With kern.threads.virtual_cpu=1 it works also > in the libpthread case, but while there are still some fork() calls > it works better. With kern.threads.virtual_cpu=2 it's slower. And > With kern.threads.virtual_cpu=4 its really slow. > > If I run clamd in foreground mode everything works as it should. > More interesting, the CPU load drops significantly (500-1000%) ! > > Something is seriosly broken here. Running in foreground with or without kern.threads.virtual_cpu adjusted? If running in foreground works regardless of kern.threads.virtual_cpu (there will still be some inefficiencies here because of the scheduler and KSE overhead/contention), then it is probably due to the application doing a fork() from a threaded application without doing an exec(). This is prohibited by POSIX and libpthread makes no attempt to allow it. -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 22:39:25 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6023516CBE7; Tue, 20 Feb 2007 22:39:25 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id F296F13C4A6; Tue, 20 Feb 2007 22:39:24 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1KMdM4V036771; Tue, 20 Feb 2007 23:39:22 +0100 (CET) (envelope-from mb@imp.ch) Date: Tue, 20 Feb 2007 23:39:22 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: Message-ID: <20070220233322.M4139@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 22:39:25 -0000 Hi, > If running in foreground works regardless of kern.threads.virtual_cpu > (there will still be some inefficiencies here because of the > scheduler and KSE overhead/contention), then it is probably due > to the application doing a fork() from a threaded application > without doing an exec(). This is prohibited by POSIX and > libpthread makes no attempt to allow it. Yes it works regardless of kern.threads.virtual_cpu. Yes, fork() is called without exec(). So what's the proper way ? Start clamd in foreground mode with daemon /usr/local/sbin/clamd And redirect stderr ? Or can this be solved differently ? Martin From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 22:48:26 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6199316C5DA; Tue, 20 Feb 2007 22:48:26 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id DAC2313C481; Tue, 20 Feb 2007 22:48:25 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1KMmNjp048732; Tue, 20 Feb 2007 23:48:24 +0100 (CET) (envelope-from mb@imp.ch) Date: Tue, 20 Feb 2007 23:48:23 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: Message-ID: <20070220234734.H4139@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 22:48:26 -0000 Hi, Useing daemon(3) instead of fork seems to fix the problem. Thank you very much ! Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ On Tue, 20 Feb 2007, Daniel Eischen wrote: > On Tue, 20 Feb 2007, Martin Blapp wrote: > >> Hi, >> >>> Also try setting kern.threads.virtual_cpu=1 if you have >>> more than 1 cpu (this will limit it to 1 KSE). >> >> Our boxes have 2 CPUs and HTT is activated. >> >> Interesting. With kern.threads.virtual_cpu=1 it works also >> in the libpthread case, but while there are still some fork() calls >> it works better. With kern.threads.virtual_cpu=2 it's slower. And >> With kern.threads.virtual_cpu=4 its really slow. >> >> If I run clamd in foreground mode everything works as it should. >> More interesting, the CPU load drops significantly (500-1000%) ! >> >> Something is seriosly broken here. > > Running in foreground with or without kern.threads.virtual_cpu > adjusted? > > If running in foreground works regardless of kern.threads.virtual_cpu > (there will still be some inefficiencies here because of the > scheduler and KSE overhead/contention), then it is probably due > to the application doing a fork() from a threaded application > without doing an exec(). This is prohibited by POSIX and > libpthread makes no attempt to allow it. > > -- > DE > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Tue Feb 20 23:09:20 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AD08F16B04C; Tue, 20 Feb 2007 23:09:20 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id 2A3F013C461; Tue, 20 Feb 2007 23:09:19 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1KN9GMY073964; Wed, 21 Feb 2007 00:09:16 +0100 (CET) (envelope-from mb@imp.ch) Date: Wed, 21 Feb 2007 00:09:16 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: <20070220234734.H4139@godot.imp.ch> Message-ID: <20070221000830.V4139@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Feb 2007 23:09:20 -0000 Sigh, > > Useing daemon(3) instead of fork seems to fix the problem. Of course deamon(3) calls fork, and after a short time I had again the same ktrace output and CPU usage. Martin From owner-freebsd-threads@FreeBSD.ORG Wed Feb 21 01:05:42 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 05D2816CF96; Wed, 21 Feb 2007 01:05:42 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id 9076913C428; Wed, 21 Feb 2007 01:05:41 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1L15Zfq003611; Wed, 21 Feb 2007 02:05:35 +0100 (CET) (envelope-from mb@imp.ch) Date: Wed, 21 Feb 2007 02:05:35 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: <20070221000830.V4139@godot.imp.ch> Message-ID: <20070221020335.Y4139@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Feb 2007 01:05:42 -0000 > Sigh, > >> >> Useing daemon(3) instead of fork seems to fix the problem. > > Of course deamon(3) calls fork, and after a short time I > had again the same ktrace output and CPU usage. Using daemon(3) works some times, but from time to time I see then again the old behaviour, looping kse calls and some forks(). I wonder why those calls never happen in the libthr and libc_r cases. Martin From owner-freebsd-threads@FreeBSD.ORG Wed Feb 21 05:03:19 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9B2DE16DC1B for ; Wed, 21 Feb 2007 05:03:19 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 0864C13C4E1 for ; Wed, 21 Feb 2007 05:03:13 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.14.0/8.14.0/NETPLEX) with ESMTP id l1L53AF5009056; Wed, 21 Feb 2007 00:03:11 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-3.0 (mail.ntplx.net [204.213.176.10]); Wed, 21 Feb 2007 00:03:11 -0500 (EST) Date: Wed, 21 Feb 2007 00:03:10 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Martin Blapp In-Reply-To: <20070221020335.Y4139@godot.imp.ch> Message-ID: References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> <20070221020335.Y4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Feb 2007 05:03:19 -0000 On Wed, 21 Feb 2007, Martin Blapp wrote: >> Sigh, >> >>> >>> Useing daemon(3) instead of fork seems to fix the problem. >> >> Of course deamon(3) calls fork, and after a short time I >> had again the same ktrace output and CPU usage. > > Using daemon(3) works some times, but from time to time > I see then again the old behaviour, looping kse calls > and some forks(). > > I wonder why those calls never happen in the libthr > and libc_r cases. libc_r and libthr don't have to kse initialization and have different locking. Regardless, I think the code (clamd) is wrong. There is no guarantee that fork() from threaded application will work for libthr and libc_r either. If clamd needs to daemonize, it can always exec() itself with an added argument that means "I've already fork()'d, don't fork() again". -- DE From owner-freebsd-threads@FreeBSD.ORG Sat Feb 24 10:30:16 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8B6E716A46F; Sat, 24 Feb 2007 10:30:16 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id 1E47313C441; Sat, 24 Feb 2007 10:30:15 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1OAU9fc016898; Sat, 24 Feb 2007 11:30:11 +0100 (CET) (envelope-from mb@imp.ch) Date: Sat, 24 Feb 2007 11:30:09 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: <20070221020335.Y4139@godot.imp.ch> Message-ID: <20070224103422.V18301@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> <20070221020335.Y4139@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Feb 2007 10:30:16 -0000 Hi, After some busy days I found again time to work on this. I still have no clue what could be responsable for the problems I see. ktrace still shows lot of fork()s, but I can't see where whey are called. Running ktrace with libc_r or libthr I can't see a single call to fork(). Is ktrace lying to me ? Why do I only see fork()s with libpthread ? Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ From owner-freebsd-threads@FreeBSD.ORG Sat Feb 24 15:13:58 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9754516A40A for ; Sat, 24 Feb 2007 15:13:58 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 54FD713C4B7 for ; Sat, 24 Feb 2007 15:13:58 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.14.0/8.14.0/NETPLEX) with ESMTP id l1OFDtXY006411; Sat, 24 Feb 2007 10:13:55 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-3.0 (mail.ntplx.net [204.213.176.10]); Sat, 24 Feb 2007 10:13:55 -0500 (EST) Date: Sat, 24 Feb 2007 10:13:54 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Martin Blapp In-Reply-To: <20070224103422.V18301@godot.imp.ch> Message-ID: References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> <20070221020335.Y4139@godot.imp.ch> <20070224103422.V18301@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Feb 2007 15:13:58 -0000 On Sat, 24 Feb 2007, Martin Blapp wrote: > > Hi, > > After some busy days I found again time to work on this. I still have no clue > what could be responsable for the problems I see. > > ktrace still shows lot of fork()s, but I can't see where whey are called. ktrace is probably lying to you if the code is really only fork()ing once. > Running ktrace with libc_r or libthr I can't see a single call to fork(). > Is ktrace lying to me ? Why do I only see fork()s with libpthread ? The kse upcalls might look like forks to ktrace? -- DE From owner-freebsd-threads@FreeBSD.ORG Sat Feb 24 18:06:36 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E2C9216A405; Sat, 24 Feb 2007 18:06:36 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id 7D38113C4A3; Sat, 24 Feb 2007 18:06:36 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1OI6WBb011459; Sat, 24 Feb 2007 19:06:33 +0100 (CET) (envelope-from mb@imp.ch) Date: Sat, 24 Feb 2007 19:06:32 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: Message-ID: <20070224185741.F18301@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> <20070221020335.Y4139@godot.imp.ch> <20070224103422.V18301@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Feb 2007 18:06:37 -0000 Hi, >> Running ktrace with libc_r or libthr I can't see a single call to fork(). >> Is ktrace lying to me ? Why do I only see fork()s with libpthread ? > > The kse upcalls might look like forks to ktrace? > That could be. Is it easy to fix this for ktrace ? Anyway, that still doesn't solve the problem for me, since clamd with libpthreads is extremly CPU hungry and is slow. I really miss something to profile like strace on solaris to see where it spends most CPU cycles. The problem is not that it doesn't work, the problem is that it works far too slow. Do you have any ideas ? Martin From owner-freebsd-threads@FreeBSD.ORG Sat Feb 24 18:47:46 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A3F9C16A402 for ; Sat, 24 Feb 2007 18:47:46 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 6299613C491 for ; Sat, 24 Feb 2007 18:47:46 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.14.0/8.14.0/NETPLEX) with ESMTP id l1OIlh9x012345; Sat, 24 Feb 2007 13:47:43 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-3.0 (mail.ntplx.net [204.213.176.10]); Sat, 24 Feb 2007 13:47:43 -0500 (EST) Date: Sat, 24 Feb 2007 13:47:43 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Martin Blapp In-Reply-To: <20070224185741.F18301@godot.imp.ch> Message-ID: References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> <20070221020335.Y4139@godot.imp.ch> <20070224103422.V18301@godot.imp.ch> <20070224185741.F18301@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Feb 2007 18:47:46 -0000 On Sat, 24 Feb 2007, Martin Blapp wrote: > > Hi, > >>> Running ktrace with libc_r or libthr I can't see a single call to fork(). >>> Is ktrace lying to me ? Why do I only see fork()s with libpthread ? >> >> The kse upcalls might look like forks to ktrace? >> > > That could be. Is it easy to fix this for ktrace ? I don't know... > Anyway, that still doesn't solve the problem for me, since > clamd with libpthreads is extremly CPU hungry and is slow. > > I really miss something to profile like strace on solaris to > see where it spends most CPU cycles. The problem is not that > it doesn't work, the problem is that it works far too slow. > > Do you have any ideas ? Yes, fix clamd so it doesn't fork() without doing an exec(). POSIX says that behavior is undefined when threaded applications fork() and call non-async-signal-safe functions. -- DE From owner-freebsd-threads@FreeBSD.ORG Sat Feb 24 21:52:07 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 96A2416A400; Sat, 24 Feb 2007 21:52:07 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id E80F213C442; Sat, 24 Feb 2007 21:52:06 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1OLq292025012; Sat, 24 Feb 2007 22:52:03 +0100 (CET) (envelope-from mb@imp.ch) Date: Sat, 24 Feb 2007 22:52:02 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: Message-ID: <20070224225108.D18301@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> <20070221020335.Y4139@godot.imp.ch> <20070224103422.V18301@godot.imp.ch> <20070224185741.F18301@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: rob@debank.tv, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Feb 2007 21:52:07 -0000 I already did that. With libc_r and libthr I don't see a single fork() here. > Yes, fix clamd so it doesn't fork() without doing an exec(). > POSIX says that behavior is undefined when threaded applications > fork() and call non-async-signal-safe functions.