From owner-freebsd-questions@FreeBSD.ORG Mon Jun 30 19:21:44 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B126106567B for ; Mon, 30 Jun 2008 19:21:44 +0000 (UTC) (envelope-from alex@schnarff.com) Received: from mho-01-bos.mailhop.org (mho-01-bos.mailhop.org [63.208.196.178]) by mx1.freebsd.org (Postfix) with ESMTP id 3E8908FC1C for ; Mon, 30 Jun 2008 19:21:44 +0000 (UTC) (envelope-from alex@schnarff.com) Received: from c-76-114-208-110.hsd1.va.comcast.net ([76.114.208.110] helo=schnarff.com) by mho-01-bos.mailhop.org with esmtpa (Exim 4.68) (envelope-from ) id 1KDNda-000Lg1-Ea for freebsd-questions@freebsd.org; Mon, 30 Jun 2008 17:58:02 +0000 Received: (qmail 2579 invoked by uid 67); 30 Jun 2008 17:58:01 -0000 Received: from sf-nat.sourcefire.com (sf-nat.sourcefire.com [64.214.53.2]) by mail.schnarff.com (Horde) with HTTP for ; Mon, 30 Jun 2008 13:58:01 -0400 X-Mail-Handler: MailHop Outbound by DynDNS X-Originating-IP: 76.114.208.110 X-Report-Abuse-To: abuse@dyndns.com (see http://www.mailhop.org/outbound/abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18+yhf6Cl3bCAed0exEdVUkGdQLeV44AR8= Message-ID: <20080630135801.h2jz2aibb44gw0kg@mail.schnarff.com> Date: Mon, 30 Jun 2008 13:58:01 -0400 From: alex@schnarff.com To: freebsd-questions@freebsd.org References: <20080630165205.GA3033@lpthe.jussieu.fr> <48691D7C.2090804@FreeBSD.org> In-Reply-To: <48691D7C.2090804@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.0.4) Subject: Re: Too Much Context Switching? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2008 19:21:44 -0000 >>> In 6.x. the default thread library is quite inefficient although it >>> can make use of multiple CPUs (again, providing the application is >>> giving them work to do). For multi-threaded performance you will >>> be better off switching to the libthr library (see libmap.conf(5)) >>> or updating to 7.0 (where it is the default). This isn't likely to >>> be the underlying issue if you are trying to debug a loss of >>> performance relative to the same configuration in the past though. >> >> Indeed Plone is written in python, and python has a "Big Giant Lock" >> inside which insures that only one thread can execute, in order to >> protect the python structures. This lock is only released under special >> circumstances, such as doing IO. Hence it is necessary to run several >> instances of python programs and do synchronization work, if one wants >> to make use of several CPUs, or use python threads, and immediately >> make some IOs, or similar techniques. It may be that using Jython, if >> possible, yields better threading behavior. When doing some work >> according to these ideas, i had found quite severe contention, and this >> was not cured when switching native threading libraries (libksd, libthr, >> etc.). The problem is really inside python. > > Yep, it could be that -- what confuses me though is that it is claimed > that performance suddenly regressed. If so then this cannot be the > underlying cause. > It's actually been a long, slow, steady degradation of performance as best I can tell, that's recently just reached proportions that are so ridiculous that it's gone from "this sucks but I can deal" to "this is completely unusable." The system has been slow from the start, just not this slow. I guess I'll need to investigate this...and while I know that Python is somewhat off-topic, if anyone here has any suggestions on where to start, they'd be much appreciated. :-) Alex