From owner-freebsd-questions@FreeBSD.ORG Thu Jul 3 19:14:35 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD8451065673 for ; Thu, 3 Jul 2008 19:14:35 +0000 (UTC) (envelope-from lreid@cs.okstate.edu) Received: from a.cs.okstate.edu (a.cs.okstate.edu [139.78.113.1]) by mx1.freebsd.org (Postfix) with ESMTP id B6EC08FC14 for ; Thu, 3 Jul 2008 19:14:35 +0000 (UTC) (envelope-from lreid@cs.okstate.edu) Received: from [127.0.0.1] (localhost [127.0.0.1]) by a.cs.okstate.edu (Postfix) with ESMTP id CC68D154CED; Thu, 3 Jul 2008 14:14:34 -0500 (CDT) Message-ID: <486D2518.1050501@cs.okstate.edu> Date: Thu, 03 Jul 2008 14:14:32 -0500 From: Reid Linnemann User-Agent: Thunderbird 2.0.0.6 (X11/20070926) MIME-Version: 1.0 To: alex@schnarff.com References: <20080630165205.GA3033@lpthe.jussieu.fr> <48691D7C.2090804@FreeBSD.org> <20080630135801.h2jz2aibb44gw0kg@mail.schnarff.com> In-Reply-To: <20080630135801.h2jz2aibb44gw0kg@mail.schnarff.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org Subject: Re: Too Much Context Switching? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2008 19:14:35 -0000 Written by alex@schnarff.com on 06/30/08 12:58>> >>>> In 6.x. the default thread library is quite inefficient although it >>>> can make use of multiple CPUs (again, providing the application is >>>> giving them work to do). For multi-threaded performance you will be >>>> better off switching to the libthr library (see libmap.conf(5)) or >>>> updating to 7.0 (where it is the default). This isn't likely to be >>>> the underlying issue if you are trying to debug a loss of >>>> performance relative to the same configuration in the past though. >>> >>> Indeed Plone is written in python, and python has a "Big Giant Lock" >>> inside which insures that only one thread can execute, in order to >>> protect the python structures. This lock is only released under special >>> circumstances, such as doing IO. Hence it is necessary to run several >>> instances of python programs and do synchronization work, if one wants >>> to make use of several CPUs, or use python threads, and immediately >>> make some IOs, or similar techniques. It may be that using Jython, if >>> possible, yields better threading behavior. When doing some work >>> according to these ideas, i had found quite severe contention, and this >>> was not cured when switching native threading libraries (libksd, libthr, >>> etc.). The problem is really inside python. >> >> Yep, it could be that -- what confuses me though is that it is claimed >> that performance suddenly regressed. If so then this cannot be the >> underlying cause. >> > > It's actually been a long, slow, steady degradation of performance as > best I can tell, that's recently just reached proportions that are so > ridiculous that it's gone from "this sucks but I can deal" to "this is > completely unusable." The system has been slow from the start, just not > this slow. I guess I'll need to investigate this...and while I know that > Python is somewhat off-topic, if anyone here has any suggestions on > where to start, they'd be much appreciated. :-) > > Alex As far as degradations-over-time are concerned, don't overlook your ZODB. If you don't pack it regularly and it grows to some ridiculous size you can be in for a world of hurt.