From owner-freebsd-hackers@FreeBSD.ORG Mon Mar 19 19:03:03 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 60AF616A401 for ; Mon, 19 Mar 2007 19:03:03 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id DF22213C4DB for ; Mon, 19 Mar 2007 19:03:02 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from localhost (localhost [127.0.0.1]) by people.fsn.hu (Postfix) with ESMTP id 20C988442B; Mon, 19 Mar 2007 19:38:38 +0100 (CET) Received: from people.fsn.hu ([127.0.0.1]) by localhost (people.fsn.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 22745-08; Mon, 19 Mar 2007 19:38:19 +0100 (CET) Received: from [IPv6:::1] (unknown [192.168.2.3]) by people.fsn.hu (Postfix) with ESMTP id D250784425; Mon, 19 Mar 2007 19:38:18 +0100 (CET) Message-ID: <45FED89A.2050307@fsn.hu> Date: Mon, 19 Mar 2007 19:38:18 +0100 From: Attila Nagy User-Agent: Thunderbird 1.5.0.9 (X11/20070124) MIME-Version: 1.0 To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at fsn.hu Cc: mb@imp.ch Subject: clamav problems with 6-STABLE, with libthr and libpthread X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2007 19:03:03 -0000 Hello, I have: ldd /usr/local/sbin/clamd /usr/local/sbin/clamd: libclamav.so.2 => /usr/local/lib/libclamav.so.2 (0x80063e000) libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x8007a8000) libz.so.3 => /lib/libz.so.3 (0x800999000) libbz2.so.2 => /usr/lib/libbz2.so.2 (0x800aad000) libgmp.so.7 => /usr/local/lib/libgmp.so.7 (0x800bbc000) libthr.so.2 => /usr/lib/libthr.so.2 (0x800cf6000) libc.so.6 => /lib/libc.so.6 (0x800e0d000) clamd -V ClamAV 0.90.1/2873/Mon Mar 19 18:46:10 2007 uname -a FreeBSD mx 6.2-STABLE FreeBSD 6.2-STABLE #1: Tue Jan 30 11:22:54 CET 2007 root@compile:/usr/obj/usr/src/sys/MX amd64 And this in top: PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 2879 vscan 258 96 0 272M 240M umtx 2 222:07 87.55% clamav-milter A ktrace for the process shows: kdump -EH | less 2879 100579 clamav-milter 0.000000 RET _umtx_op 0 2879 100469 clamav-milter 0.000009 CALL _umtx_op(0x801226ac0,0,0x18875,0,0) 2879 100579 clamav-milter 0.000041 CALL _umtx_op(0x801226ac0,0x1,0x188e3,0,0) 2879 100579 clamav-milter 0.000078 RET _umtx_op 0 2879 100469 clamav-milter 0.000082 RET _umtx_op 0 2879 100579 clamav-milter 0.000084 CALL _umtx_op(0x801226ac0,0,0x188e3,0,0) 2879 100469 clamav-milter 0.000091 CALL _umtx_op(0x801226ac0,0x1,0x18875,0,0) 2879 100469 clamav-milter 0.000122 RET _umtx_op 0 2879 100579 clamav-milter 0.000127 RET _umtx_op 0 2879 100579 clamav-milter 0.000141 CALL _umtx_op(0x801226ac0,0x1,0x188e3,0,0) 2879 100469 clamav-milter 0.000131 CALL _umtx_op(0x801226ac0,0,0x18875,0,0) 2879 100744 clamav-milter 0.000161 RET _umtx_op 0 2879 100744 clamav-milter 0.000177 CALL _umtx_op(0x801226ac0,0x1,0x18988,0,0) 2879 100579 clamav-milter 0.000155 RET _umtx_op 0 2879 100469 clamav-milter 0.000198 RET _umtx_op 0 2879 100469 clamav-milter 0.000213 CALL _umtx_op(0x801226ac0,0x1,0x18875,0,0) 2879 100579 clamav-milter 0.000202 CALL _umtx_op(0x801226ac0,0,0x188e3,0,0) 2879 100782 clamav-milter 0.000233 RET _umtx_op 0 2879 100744 clamav-milter 0.000191 RET _umtx_op 0 2879 100782 clamav-milter 0.000251 CALL _umtx_op(0x801226ac0,0x1,0x189ae,0,0) 2879 100744 clamav-milter 0.000266 CALL _umtx_op(0x801226ac0,0,0x18988,0,0) 2879 100782 clamav-milter 0.000283 RET _umtx_op 0 2879 100782 clamav-milter 0.000308 CALL _umtx_op(0x801226ac0,0,0x189ae,0,0) [13210 similar lines skipped] 2879 100553 clamav-milter 0.125179 CALL read(0x38f,0xb83d000,0x1000) 2879 100579 clamav-milter 0.125180 CALL _umtx_op(0x801226ac0,0x1,0x188e3,0,0) 2879 100553 clamav-milter 0.125198 GIO fd 911 read 4096 bytes Although clamav does some work occasionally, it really just spins in the above loops and postfix (which delivers mail to it via milter) just gets connection timeouts. I've seen Martin's e-mails in some places about this issue and there he wrote that this problem occurs only with libpthread. It seems it's not. The previous version of the port worked fine^Wbetter. It seems that the problem has a straight relationship with the number of processors, because those machines which has less of them (2 cores) timeout and get stuck far less (or don't timeout at all) than the others, which has 4 (two dual core xeon CPUs). The problem was even worse with a dual quad core system, where the previous port (both with libthr and libpthreads) was useless with clamav, because of the above problem. Do you have any idea about the solution? ps: I run clamav in foreground mode, most of the machines have non-HTT processors (Xeon 51XX for the most problematic ones, which has two cores, but no HTT) Thanks, -- Attila Nagy e-mail: Attila.Nagy@fsn.hu Free Software Network (FSN.HU) phone: +3630 306 6758 http://www.fsn.hu/