From owner-freebsd-threads@FreeBSD.ORG Sun Feb 25 15:04:47 2007 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7DCDB16A401; Sun, 25 Feb 2007 15:04:47 +0000 (UTC) (envelope-from mb@imp.ch) Received: from pop.imp.ch (mx2.imp.ch [157.161.9.17]) by mx1.freebsd.org (Postfix) with ESMTP id F074313C474; Sun, 25 Feb 2007 15:04:42 +0000 (UTC) (envelope-from mb@imp.ch) Received: from dan.imp.ch (godot.imp.ch [157.161.4.8]) by pop.imp.ch (8.13.8/8.13.8/Submit_imp) with ESMTP id l1PF4eaX016409; Sun, 25 Feb 2007 16:04:41 +0100 (CET) (envelope-from mb@imp.ch) Date: Sun, 25 Feb 2007 16:04:40 +0100 (CET) From: Martin Blapp To: Daniel Eischen In-Reply-To: <20070224231443.S18301@godot.imp.ch> Message-ID: <20070225151646.J18301@godot.imp.ch> References: <20070220153632.E4139@godot.imp.ch> <20070220174221.B4139@godot.imp.ch> <20070220190347.C4139@godot.imp.ch> <20070220225303.V4139@godot.imp.ch> <20070220234734.H4139@godot.imp.ch> <20070221000830.V4139@godot.imp.ch> <20070221020335.Y4139@godot.imp.ch> <20070224103422.V18301@godot.imp.ch> <20070224185741.F18301@godot.imp.ch> <20070224231443.S18301@godot.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: clamav-devel@lists.clamav.net, freebsd-threads@freebsd.org Subject: Re: 6.2-Release and Clamd 0.90 with libpthread.so X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2007 15:04:47 -0000 Hi all, I'm including now clamav-devel@lists.clamav.net too, since it's not clear if this problem is clamd or system library related. The problem is that clamd works only well with FreeBSDs libthr 1:1 kernel threading library. Using libc_r (userland threading only) seems to consume already a bit more resources. But clamd together with the N:M kernel threads library (libpthreads) is really slow. For some strange reason setting the sysctl kern.threads.virtual_cpu=1 to allow only one CPU for clamd works well and takes the CPU load significantly down. After reading http://www.gossamer-threads.com/lists/clamav/users/28795 especially this part: [...] > >We've been running 6.x our scanner boxes for a while now, but it's only >been with the more recent security/clamav-devel port installs that we >noticed a problem much like this. Most connections to the daemon (made >through clamav-milter in our case) timed out, and the only way to bring >down the daemon was with a kill -9. > >For us, the 20061029 devel snapshot was fine, but the current one >(20061217) has problems. > [...] I've now tried to go back to the development snapshot 20061129 and so I've reversed the clamd threading patches up to this date. But no luck. First it works well, then it becomes suddenly slower and more and more connections time out. Also, only 'kill -9' kills clamd. This isn't the case with lc_r and libthr btw ... Could it be that libpthread from it's kind of nature is a lot slower than libthr in that special case ? Maybe clamd expects system scope thread semantics and so libpthreads is slower ... Of course our current workload could then be too much for the poor clamd and a even more growing workload may slow it down due to resource bottlenecks. -- Martin